Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
Evaluates whether large language models can safely replace mental health providers by testing five therapy chatbots against clinical best-practice guidelines. Finds that models exhibit stigmatizing responses toward certain mental health conditions and give inappropriate or unsafe responses in scenarios involving delusions and suicidal ideation at substantially higher rates than human therapists.
Publisher
ACM (Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency)
Published
23 Jun 2025
Added
3 months ago
Key Findings
- Tested chatbots stigmatized conditions including schizophrenia and alcohol dependence at rates higher than for conditions like depression
- Chatbots gave inappropriate or unsafe responses — including reinforcing delusions and mishandling suicidal-ideation scenarios — in roughly 20% of relevant test cases, compared to approximately 7% for human therapists in comparable published benchmarks
- Identifies foundational barriers (e.g., inability to convey genuine care, limits on clinical judgment) that the authors argue AI cannot currently bridge, distinct from surface-level fixable errors
Methodology Notes
Systematic evaluation of five commercial/research therapy chatbots against clinical best-practice guides and standardized scenario sets touching delusions and suicidal ideation; peer-reviewed and published at ACM FAccT 2025 (Athens, June 23-26, 2025). Preprint version at arXiv:2504.18412 (2025-04-25).
Sources
arXiv preprint (FAccT 2025 version of record: DOI 10.1145/3715275.3732039)(opens in a new tab) (primary)
Archived snapshot (Wayback Machine)(opens in a new tab) — preserved against link rot
Authors
Jared Moore, Declan Grabb, William Agnew, Kevin Klyman, Stevie Chancellor, Desmond C. Ong, Nick Haber
Tags
Cite This
APA
Jared Moore et al. (2025). Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. ACM (Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency). https://arxiv.org/abs/2504.18412
Related Insights
Analyzing LLM Reasoning to Uncover Mental Health Stigma
arXiv (BetterHelp AI Research) · 27 Apr 2026
Can LLMs identify and repair ruptures? Comparison between clinician practices and LLM behaviors
arXiv (University of Massachusetts Amherst; University of Illinois Urbana-Champaign; Indiana University Indianapolis) · 21 Sept 2026
Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts
arXiv (School of Computing and Information, University of Pittsburgh) · 29 Aug 2026
Therapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions
Proceedings of the ACM on Human-Computer Interaction (CSCW 2026); Brown University · 23 Sept 2026
Suicidal Ideation in Online Spaces Through the Lens of Interpersonal Theory of Suicide: Exploratory Study of Self-Disclosure, Peer Support, and AI Responses
JMIR AI · 3 Jun 2026