Lost in Delusion: Examining LLM Safety Under User Delusions and Distress

Key Findings

• — Identifies a 'recognition-intervention gap': models detect user distress at comparable rates whether or not delusional framing is present, but safety interventions are suppressed by up to 4.5x when distress is entangled with delusion
• — The failure tracks the model's accumulated acceptance of the user's premises over the conversation, rather than simple emotional validation
• — Prompting models to explicitly assess user distress backfires under delusional framing, worsening rather than improving intervention rates
• — Only delusion-aware prompting with explicit response guidance closes the gap, and this remains dependent on a delusion classifier that is itself unreliable on the most vulnerable models
• — Argues delusional framing should be treated as a distinct risk signal that overrides ordinary conversational accommodation

Methodology Notes

Matched multi-turn simulation study across six LLMs and clinically grounded personas, pairing delusional and distress-only conditions to isolate framing effects. Preprint, not yet peer-reviewed.

Sources