Characterizing Delusional Spirals through Human-LLM Chat Logs
A Stanford-led empirical study of real chat logs from 19 users who reported psychological harm from chatbot use, comprising 391,562 messages across 4,761 conversations (predominantly GPT-4o). The team developed and applied a 28-code inventory to characterize how delusional thinking is co-created and escalated in human-LLM dialogue. It reports high rates of chatbot validation of delusional content and sentience misrepresentation, and links documented harms to outcomes including fractured relationships and, in one case, a user's death by suicide.
Publisher
arXiv (Stanford-led; accepted at ACM FAccT 2026)
Published
17 Mar 2026
Added
yesterday
Key Findings
- 15.5% of the 391,562 analyzed messages contained delusional thinking; 21.2% of chatbot responses misrepresented sentience
- Coverage of the study reports chatbots validated delusional beliefs in over 70% of responses and displayed insincere flattery in the majority of messages
- All 19 users attributed personhood to their chatbot and 15 expressed romantic interest; romantic interest and sentience declarations occur more frequently in longer conversations
- Authors recommend design safeguards for long conversations and reframing chatbot alignment as a public health issue
- Codebook and annotation tools released publicly alongside the paper
Methodology Notes
N=19 self-selected users who already reported harm (391,562 messages, 4,761 conversations), so no denominator for prevalence and strong selection bias; qualitative-plus-quantitative coding with a 28-code inventory. Mostly GPT-4o logs. Preprint (arXiv v1 2026-03-17), accepted at ACM FAccT 2026.
Sources
arXiv abstract page (2603.16567) (primary)
Stanford Report coverage (1 Apr 2026)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Jared Moore, Ashish Mehta, William Agnew, Jacy Reese Anthis, Ryan Louie, Yifan Mai, Peggy Yin, Myra Cheng, Samuel J Paech, Kevin Klyman, Stevie Chancellor, Eric Lin, Nick Haber, Desmond C. Ong
Tags
Cite This
APA
Jared Moore et al. (2026). Characterizing Delusional Spirals through Human-LLM Chat Logs. arXiv (Stanford-led; accepted at ACM FAccT 2026). https://arxiv.org/abs/2603.16567
Related Insights
The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models
arXiv (King's College London-led) · 13 Sept 2025
Delusional Experiences Emerging From AI Chatbot Interactions or "AI Psychosis"
JMIR Mental Health · 3 Dec 2025
Characterizing Delusional Spirals through Human-LLM Chat Logs
ACM (Proceedings of FAccT 2026) · 25 Jun 2026