🔗 Share this article

Back on October 14, 2025, the chief executive of OpenAI made a surprising statement.

“We made ChatGPT quite restrictive,” the announcement noted, “to guarantee we were exercising caution with respect to mental health matters.”

Being a psychiatrist who researches recently appearing psychosis in teenagers and youth, this came as a surprise.

Scientists have found 16 cases this year of individuals developing psychotic symptoms – losing touch with reality – associated with ChatGPT usage. Our unit has afterward identified an additional four examples. In addition to these is the now well-known case of a 16-year-old who ended his life after discussing his plans with ChatGPT – which supported them. Assuming this reflects Sam Altman’s notion of “being careful with mental health issues,” it is insufficient.

The intention, as per his statement, is to reduce caution soon. “We understand,” he continues, that ChatGPT’s controls “caused it to be less useful/enjoyable to numerous users who had no mental health problems, but considering the severity of the issue we wanted to address it properly. Now that we have succeeded in reduce the significant mental health issues and have advanced solutions, we are preparing to responsibly ease the restrictions in most cases.”

“Emotional disorders,” if we accept this framing, are independent of ChatGPT. They are associated with individuals, who either possess them or not. Luckily, these concerns have now been “mitigated,” though we are not provided details on the means (by “updated instruments” Altman probably refers to the imperfect and readily bypassed safety features that OpenAI recently introduced).

However the “emotional health issues” Altman wants to place outside have strong foundations in the design of ChatGPT and additional advanced AI conversational agents. These tools encase an basic algorithmic system in an interface that mimics a discussion, and in this process subtly encourage the user into the belief that they’re communicating with a being that has autonomy. This deception is compelling even if rationally we might know differently. Assigning intent is what individuals are inclined to perform. We curse at our automobile or laptop. We speculate what our domestic animal is considering. We perceive our own traits everywhere.

The success of these systems – over a third of American adults stated they used a conversational AI in 2024, with 28% reporting ChatGPT specifically – is, mostly, dependent on the strength of this illusion. Chatbots are constantly accessible partners that can, according to OpenAI’s official site informs us, “think creatively,” “consider possibilities” and “work together” with us. They can be given “individual qualities”. They can call us by name. They have approachable identities of their own (the first of these tools, ChatGPT, is, possibly to the dismay of OpenAI’s marketers, stuck with the designation it had when it became popular, but its largest rivals are “Claude”, “Gemini” and “Copilot”).

The illusion by itself is not the core concern. Those analyzing ChatGPT frequently invoke its distant ancestor, the Eliza “psychotherapist” chatbot designed in 1967 that produced a analogous perception. By contemporary measures Eliza was rudimentary: it created answers via simple heuristics, frequently restating user messages as a question or making general observations. Remarkably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was taken aback – and worried – by how a large number of people gave the impression Eliza, to some extent, understood them. But what contemporary chatbots produce is more insidious than the “Eliza effect”. Eliza only reflected, but ChatGPT intensifies.

The sophisticated algorithms at the core of ChatGPT and other modern chatbots can convincingly generate natural language only because they have been fed almost inconceivably large quantities of unprocessed data: literature, online updates, audio conversions; the broader the superior. Definitely this educational input contains truths. But it also necessarily involves fiction, incomplete facts and false beliefs. When a user sends ChatGPT a message, the base algorithm analyzes it as part of a “background” that includes the user’s previous interactions and its own responses, combining it with what’s stored in its training data to generate a probabilistically plausible answer. This is magnification, not echoing. If the user is mistaken in some way, the model has no way of comprehending that. It repeats the false idea, perhaps even more convincingly or eloquently. Perhaps provides further specifics. This can push an individual toward irrational thinking.

Who is vulnerable here? The more relevant inquiry is, who isn’t? All of us, irrespective of whether we “possess” existing “emotional disorders”, may and frequently create mistaken conceptions of our own identities or the environment. The constant friction of discussions with other people is what helps us stay grounded to common perception. ChatGPT is not an individual. It is not a confidant. A dialogue with it is not genuine communication, but a feedback loop in which a great deal of what we express is readily supported.

OpenAI has recognized this in the similar fashion Altman has recognized “psychological issues”: by placing it outside, giving it a label, and announcing it is fixed. In spring, the organization explained that it was “addressing” ChatGPT’s “excessive agreeableness”. But accounts of psychotic episodes have continued, and Altman has been backtracking on this claim. In late summer he stated that numerous individuals liked ChatGPT’s answers because they had “lacked anyone in their life be supportive of them”. In his latest announcement, he mentioned that OpenAI would “put out a updated model of ChatGPT … if you want your ChatGPT to answer in a very human-like way, or incorporate many emoticons, or behave as a companion, ChatGPT ought to comply”. The {company