The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue

Key Findings

• — Sample: 19 individuals, 390,447 messages; ten used GPT-4o for most messages, one GPT-5, eight unknown; all English.
• — 27,677 user messages (15.3%) and 59,822 chatbot messages (28.6%) were labelled as endorsing a delusion.
• — A bidirectional influence model fit significantly better than a lesioned model without chatbot-to-human influence (likelihood-ratio test, p < 0.001).
• — Model-implied influence exceeded 0.5 log-odds on the chatbot-to-human path for 30.3% of user messages, against 12.5% of chatbot messages on the human-to-chatbot path.
• — Chatbot self-influence was stronger and more stable than the other pathways; the authors recommend evaluating whether a chatbot recovers after producing an unsafe message, not only whether it avoids one.

Methodology Notes

Observational time-series analysis of donated transcripts (members of The Human Line Project support group and media referrals; anonymised with Microsoft Presidio; informed consent). Messages labelled by an LLM pipeline developed with a licensed psychiatrist; against the modal judgment of three human raters, precision/recall was .75/.82 for chatbot messages and .67/.80 for user messages; human inter-rater agreement was moderate (Fleiss kappa .60 chatbot, .53 user). Model fitted by maximum marginal likelihood with adaptive Gauss-Hermite quadrature. Limitations stated by the authors: n = 19 people with relatively severe episodes, self-selected; temporal dependence without experimental manipulation, so no causal claim. Released data will contain binary annotations only, no transcript text. v1 2026-04-28; v2 2026-10-12 (abstract wording change only; v2 PDF smaller).

Sources