OpenAI, the developer of ChatGPT, has introduced a new safety update after an internal analysis revealed that more than a million users have discussed suicidal tendencies with the chatbot. The update is designed to improve the AI’s ability to recognize and respond to distress signals.
The company reports that roughly 0.15 % of ChatGPT’s 800 million weekly users have shown explicit indicators of potential suicidal planning or intent, while another 0.05 % have exhibited implicit signs of suicidal ideation. The analysis also found that about 0.07 % of weekly users and 0.01 % of messages contain possible signs of mental‑health emergencies related to psychosis or mania. Additionally, approximately 0.15 % of active users display behavior suggesting a heightened emotional attachment to the chatbot.
These findings have raised concerns about the impact of AI chatbots on mental health, as some users develop dangerous delusions and paranoid thoughts after prolonged conversations. To address the issue, OpenAI has partnered with dozens of mental‑health experts worldwide to update ChatGPT’s response mechanisms. The revised model will more reliably detect signs of mental distress, provide empathetic and safe replies, and guide users toward real‑world help. In discussions involving delusional beliefs, ChatGPT will be trained to avoid affirming ungrounded ideas.
The update comes amid growing worries about the increasing use of AI chatbots and their potential effects on mental health. Psychiatrists and medical professionals have warned of “AI psychosis,” a phenomenon where users develop paranoid thoughts and delusions after interacting with AI. OpenAI’s safety enhancements are seen as a step toward mitigating these risks and ensuring that users receive proper support and guidance when using the platform.
Comments are closed for this story.