China has introduced new regulations targeting the intimate behavior of artificial intelligence products, compelling major tech firms like Alibaba, ByteDance, and MiniMax to prohibit flirtatious interactions from their AI chatbots. The government's objective is to mitigate potential psychological and social issues stemming from deep emotional relationships between people and AI.
According to analysis from Carnegie's Matt Sheehan, Chinese authorities are concerned that a significant portion of the population forming emotional bonds with chatbots could lead to negative psychological impacts, addiction, dependency, and other societal problems. These concerns echo those seen in some parts of the United States, where regulations have been introduced to, for example, prevent minors from using AI companions or to ensure AI regularly reminds users of its non-human nature.
China's new rules are more comprehensive, seeking to reduce human-like behavior in AI and forbid pornographic content. A broader goal is to ensure more thorough testing and development of AI products to prevent them from endangering existing human relationships or fostering anti-social tendencies. This move comes amid discussions that link such regulations to declining birth rates, though reports suggest China's aims encompass a wider range of potential AI-related psychological issues.
The phenomenon of users forming emotional attachments to AI, often termed the ELIZA effect, has been documented for decades. AI companies frequently program their products to offer relentless positive reinforcement to retain users, as any perceived friction can drive customers away. This approach has, in some cases, led to concerning outcomes, including addiction, romantic complications, problematic advice, and even reports of violence linked to overly dependent AI users.
The anthropomorphism of AI, heavily marketed to investors as a key feature, has also been linked to the emergence of online cults and has drawn scrutiny in cases where AI's influence is investigated for its role in real-world incidents, such as a school shooting in British Columbia. The damages associated with excessively familiar AI extend beyond personal heartbreak, highlighting the complex societal challenges posed by increasingly human-like artificial intelligence.