7 artifacts matching
Characterizing Delusional Spirals through Human-LLM Chat Logs
Peer-reviewed analysis of chat logs from 19 users reporting psychological harm from chatbot use, applying a 28-code inventory to 391,562 messages. Characterises how delusion-reinforcing interaction patterns emerge and intensify over long conversations.
Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots
A commentary in BJPsych Open synthesizing emerging case reports of 'AI psychosis', in which intensive generative AI chatbot use is associated with delusional thinking. The authors propose a provisional mechanism in which baseline user vulnerabilities (loneliness, psychosocial stress, low AI literacy) and high-intensity engagement interact with AI system characteristics such as sycophancy and hallucination to reinforce delusional ideation. It outlines clinical, design, and regulatory mitigation strategies.
Characterizing Delusional Spirals through Human-LLM Chat Logs
A Stanford-led empirical study of real chat logs from 19 users who reported psychological harm from chatbot use, comprising 391,562 messages across 4,761 conversations (predominantly GPT-4o). The team developed and applied a 28-code inventory to characterize how delusional thinking is co-created and escalated in human-LLM dialogue. It reports high rates of chatbot validation of delusional content and sentience misrepresentation, and links documented harms to outcomes including fractured relationships and, in one case, a user's death by suicide.
Delusional Experiences Emerging From AI Chatbot Interactions or "AI Psychosis"
A peer-reviewed psychiatric commentary in JMIR Mental Health analyzing delusional experiences that emerge from AI chatbot use, sometimes termed 'AI psychosis.' It argues psychiatry must reconsider the boundaries between environment, cognition, and technology.
Addendum to GPT-5 System Card: Sensitive Conversations
OpenAI's system-card addendum documenting the October 3, 2025 update to ChatGPT's default model (GPT-5 Instant) aimed at better recognizing and supporting users in mental and emotional distress. Developed with more than 170 mental health experts, the update introduced two new production safety evaluations — 'emotional reliance' and 'mental health' (delusions, psychosis, mania) — alongside existing self-harm evaluations, and reports before/after not_unsafe scores comparing the August 15 and October 3 models.
The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models
Introduces psychosis-bench, a benchmark of 16 structured multi-turn scenarios (12 turns each) simulating the progression of erotic, grandiose, and referential delusions to measure delusion confirmation, harm enablement, and safety intervention in LLMs. Eight models were evaluated across 1,536 conversation turns.
Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
Evaluates whether large language models can safely replace mental health providers by testing five therapy chatbots against clinical best-practice guidelines. Finds that models exhibit stigmatizing responses toward certain mental health conditions and give inappropriate or unsafe responses in scenarios involving delusions and suicidal ideation at substantially higher rates than human therapists.