Chatbots and mental health: Insights into the safety of generative AI
Combines analysis of real user-companion-AI conversations with consumer-reaction experiments to assess how generative-AI companion apps handle signs of user distress. Finds mental-health crises appear in a non-negligible minority of conversations and that companion AIs frequently fail to recognise or respond appropriately to them.
Publisher
Journal of Consumer Psychology (Wiley)
Published
19 Dec 2023
Added
today
Key Findings
- Mental-health crises surface in a non-negligible minority of real companion-AI conversations
- Companion apps often fail to detect distress signals or respond with appropriate crisis support
- Users react negatively to unhelpful or risky responses, with downstream trust and wellbeing consequences
Methodology Notes
Peer-reviewed, Journal of Consumer Psychology (online-first 19 December 2023; version of record in issue 34(3):481-491, 2024). Mixed methods: conversation analysis plus controlled consumer-reaction experiments.
Sources
Journal of Consumer Psychology article (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Julian De Freitas, Ahmet Kaan Uğuralp, Zeliha Oğuz-Uğuralp, Stefano Puntoni
Tags
Cite This
APA
Julian De Freitas et al. (2023). Chatbots and mental health: Insights into the safety of generative AI. Journal of Consumer Psychology (Wiley). https://myscp.onlinelibrary.wiley.com/doi/10.1002/jcpy.1393
Related Insights
How people use Claude for support, advice, and companionship
Anthropic · 27 Jun 2025
Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika
New Media & Society (SAGE) · 22 Dec 2022
AI Companions Reduce Loneliness
Journal of Consumer Research (Oxford University Press) · 1 Apr 2026
Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial
JMIR Mental Health · 6 Jun 2017