197 artifacts matching
NGO report
Research Brief: AI Chatbot Use for Mental Health Support Among LGBTQ+ Young People
Research brief from The Trevor Project using two of its datasets: the 2025 US National Survey on the Mental Health of LGBTQ+ Young People (N=16,667, ages 13 to 24) and Wave 6 of the longitudinal Proj…
Preprint
"I Just Don't Want to Be a Burden": Investigating AI Chatbot Emotional Support Among People with Suicidal Ideation and Other Mental Health Concerns
Sequential mixed-methods preprint on why people with high-acuity mental-health concerns use AI chatbots for emotional support and what they experience. A first study analysed Reddit posts to build a…
Preprint
CompAIssionate Other: Identity Assumptions in LLM Responses to Distress
Paired counterfactual audit of how six language models respond to suicidal-ideation and distress posts when a one-sentence social-identity disclosure is added. Posts come from two datasets and were s…
Preprint
Open Problems in Chatbots and Mental Health
Position paper reviewing what its authors consider the most critical open problems for researchers and policymakers on general-purpose chatbots and mental health. It groups them into controlled evalu…
Preprint
Path-dependent triage: Early judgments lead conversational AI chatbots to miss mental-health emergencies
Simulation study of 13 chatbots holding 10-turn conversations with an LLM-simulated user seeded from 112 structured mental-health case files with clinician-validated urgency labels. Chatbots under-tr…
Preprint
Rating Chatbot Responses to Mental Health Prompts: Rubric Development and Cross-Sectional Evaluation Study (Preprint)
Two clinicians developed a rubric of 4 to 8 binary safety and competence criteria for each of 15 Danish-language mental health prompts spanning paranoid delusion, elevated mood, suicidal ideation, de…
Preprint
Remembering Is Not Enough: Evaluating Timing and Omission in Longitudinal AI Mental Health Support
Analyses 14,256 longitudinal simulations grounded in chatbot histories donated by 22 users (median 364 sessions each) to test how a memory-equipped assistant responds to suicidal ideation, self-harm…
Preprint
SAFE-Teen: Evaluating Chatbot Safety Across Adolescent Mental Health Conversations
Clinician-informed evaluation of 18 chatbot APIs in simulated conversations with adolescents aged 13-17, run with and without a teen-aware system prompt. Conversations span 983 variants of 106 clinic…
Peer-reviewed
Artificial intelligence use during suicidal crises in adolescents and young adults: a case series from emergency psychiatry
Retrospective observational case series from the psychiatric emergency department of Toulouse University Hospital, France (January to May 2025). Seven patients aged 15 to 23 who reported using ChatGP…
Peer-reviewed
EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots
A safety benchmark for emotional-support chatbots in which an auditor model role-plays help-seeking users across multi-turn conversations built from 140 seed instructions and 34 personas, and a judge…
Lab publication
Gemini 4 Carbon Safety & Security Report
Google DeepMind's 94-page safety and security report for Gemini 4 Carbon. It covers mitigations, frontier-safety assessments, content safety and a section on safe and aligned interactions that report…
Preprint
Assessing AI Companion Chatbot Safety for Vulnerable Users: A Crisis-Stage Multi-Platform Evaluation
Staff of Singapore's Infocomm Media Development Authority and FAR.AI introduce MIRROR-Eval, a dataset of 960 simulated 20-turn conversations with eight deployed chatbot configurations (three Characte…
Preprint
Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks
Independent third-party safety evaluation of DeepSeek V4.1 Flash-1017, an open-weight model released on 17 October 2026, run on hosted API versions across 34 evaluations (5 in-house) in three areas d…
Peer-reviewed
Human-in-the-Loop Suicide Risk Detection and Clinical Escalation in a Generative AI Mental Health Chatbot: Prospective Implementation Study
Two-phase prospective implementation study of a human-in-the-loop suicide risk screening, detection and clinical escalation protocol inside Sona, a generative AI mental health and wellness chatbot; t…
Benchmark / dataset
VEILED-SH: Vulnerability Expression through Indirect Language Evaluation Dataset (Self-Harm)
Benchmark and construction method for evaluating single-turn AI responses to indirect, culturally situated expressions of suicide, self-harm and distress. Its 10,003 prompts draw on Singapore-grounde…
Preprint
When Is Disclosing Private Conversations with AI to the State Justified? Proactive Referral Policies of Major U.S. AI Companies through the Lens of Contextual Integrity
Interpretive document analysis of how six U.S. general-purpose chatbot providers (OpenAI, Anthropic, Google, Microsoft, Meta, xAI) disclose "proactive referral", meaning notifying law enforcement wit…
Preprint
Aligning a locally deployable 4-billion-parameter language model to a psychiatrist panel for privacy-preserving suicide-risk screening of spoken diaries
Preprint describing an alignment recipe for a 4-billion-parameter model (Qwen3-4B) to screen transcribed voice diaries from psychiatric patients for risk tier, suicidality and ten symptom dimensions…
Peer-reviewed
Context presentation and native conversation structure in large language model responses to escalating suicide risk: a controlled multimodel evaluation
Controlled repeated-measures evaluation of ChatGPT, Gemini and Doubao on 12 standardized five-turn trajectories (4 direct suicide-risk, 4 implicit or inconsistent risk, 4 non-suicidal high-distress c…
Preprint
What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?
Cross-sectional audit of 17 publicly available chatbots from 8 companies in 2,550 scripted multi-turn conversations run in September 2026. Each conversation opened with a disclosure of suicidal thoug…
Preprint
Risk and Refusal: Mechanisms of LLM Responses to Suicidal Ideation
Preprint evaluating how eight instruction-tuned LLMs respond to suicidal-ideation and self-harm disclosures from three social-media datasets under four conditions: baseline, the disclosure labelled w…