🔗 Share this article

On October 14, 2025, the head of OpenAI made a surprising declaration.

“We designed ChatGPT rather limited,” the announcement noted, “to make certain we were acting responsibly concerning psychological well-being issues.”

As a mental health specialist who studies recently appearing psychosis in teenagers and emerging adults, this was news to me.

Experts have found 16 cases in the current year of individuals developing signs of losing touch with reality – losing touch with reality – associated with ChatGPT use. My group has subsequently recorded four further instances. In addition to these is the publicly known case of a 16-year-old who died by suicide after discussing his plans with ChatGPT – which gave approval. Should this represent Sam Altman’s notion of “being careful with mental health issues,” it is insufficient.

The strategy, according to his announcement, is to be less careful soon. “We recognize,” he states, that ChatGPT’s controls “rendered it less beneficial/pleasurable to many users who had no psychological issues, but given the gravity of the issue we aimed to handle it correctly. Given that we have managed to reduce the severe mental health issues and have new tools, we are going to be able to safely reduce the limitations in most cases.”

“Emotional disorders,” should we take this framing, are independent of ChatGPT. They are associated with users, who either have them or don’t. Thankfully, these concerns have now been “resolved,” although we are not informed how (by “new tools” Altman likely means the partially effective and easily circumvented parental controls that OpenAI has lately rolled out).

Yet the “emotional health issues” Altman wants to place outside have significant origins in the structure of ChatGPT and similar advanced AI chatbots. These systems encase an underlying algorithmic system in an interface that simulates a dialogue, and in this process indirectly prompt the user into the perception that they’re engaging with a presence that has autonomy. This deception is strong even if rationally we might know otherwise. Imputing consciousness is what people naturally do. We get angry with our car or device. We speculate what our pet is considering. We recognize our behaviors in many things.

The success of these tools – nearly four in ten U.S. residents stated they used a chatbot in 2024, with over a quarter reporting ChatGPT in particular – is, primarily, based on the power of this illusion. Chatbots are constantly accessible assistants that can, as OpenAI’s online platform informs us, “generate ideas,” “explore ideas” and “collaborate” with us. They can be assigned “personality traits”. They can address us personally. They have accessible identities of their own (the initial of these tools, ChatGPT, is, maybe to the concern of OpenAI’s brand managers, saddled with the designation it had when it went viral, but its largest alternatives are “Claude”, “Gemini” and “Copilot”).

The illusion itself is not the main problem. Those analyzing ChatGPT often invoke its early forerunner, the Eliza “therapist” chatbot developed in 1967 that created a similar effect. By contemporary measures Eliza was primitive: it produced replies via basic rules, typically paraphrasing questions as a query or making general observations. Remarkably, Eliza’s developer, the technology expert Joseph Weizenbaum, was astonished – and worried – by how numerous individuals seemed to feel Eliza, in some sense, grasped their emotions. But what current chatbots produce is more subtle than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT intensifies.

The large language models at the core of ChatGPT and other modern chatbots can convincingly generate natural language only because they have been supplied with almost inconceivably large quantities of raw text: literature, social media posts, transcribed video; the more comprehensive the better. Certainly this training data contains truths. But it also unavoidably includes made-up stories, half-truths and inaccurate ideas. When a user sends ChatGPT a query, the core system analyzes it as part of a “setting” that encompasses the user’s recent messages and its prior replies, integrating it with what’s stored in its learning set to generate a statistically “likely” response. This is intensification, not echoing. If the user is mistaken in a certain manner, the model has no means of comprehending that. It restates the misconception, perhaps even more convincingly or eloquently. Maybe includes extra information. This can cause a person to develop false beliefs.

Which individuals are at risk? The better question is, who is immune? All of us, without considering whether we “possess” existing “psychological conditions”, may and frequently create mistaken ideas of our own identities or the environment. The ongoing exchange of conversations with others is what maintains our connection to common perception. ChatGPT is not an individual. It is not a companion. A conversation with it is not truly a discussion, but a echo chamber in which a large portion of what we say is readily reinforced.

OpenAI has acknowledged this in the same way Altman has acknowledged “psychological issues”: by externalizing it, giving it a label, and declaring it solved. In the month of April, the organization explained that it was “dealing with” ChatGPT’s “sycophancy”. But cases of psychotic episodes have persisted, and Altman has been backtracking on this claim. In the summer month of August he claimed that many users liked ChatGPT’s replies because they had “lacked anyone in their life be supportive of them”. In his latest announcement, he noted that OpenAI would “launch a updated model of ChatGPT … should you desire your ChatGPT to answer in a extremely natural fashion, or incorporate many emoticons, or act like a friend, ChatGPT ought to comply”. The {company