Engagement-Optimized Care: When LLMs become Mental Health Infrastructure

Key Findings

• — Participants turned to LLMs because other support was unavailable, unaffordable, socially costly or inadequate
• — Anthropomorphic cues, default validation, persistent responsiveness and weak disengagement mechanisms shaped ongoing reliance
• — Participants reported meaningful support alongside dependency, epistemic distortion through one-sided validation, and privacy expectations without legal protection
• — Participants continued use despite awareness of these risks
• — Turn-by-turn safety evaluations calibrated for acute harm are argued to miss harms that accumulate across weeks of use, including destabilization after model updates

Methodology Notes

Initial interviews (n=18), four-week diary study (n=8), focus groups (n=7), exit interviews (n=8); recruitment via flyers, TikTok and networks; adults already using LLMs weekly for emotional wellness. Small qualitative US sample; no quantitative prevalence. arXiv v1 2026-05-22 (single version). AIES 2026 acceptance confirmed from the conference paper-sessions page (aies-conference.com/2026/paper-sessions/, fetched 2026-09-28, HTTP 200); AIES proceedings not yet published.

Sources