Human-in-the-Loop Suicide Risk Detection and Clinical Escalation in a Generative AI Mental Health Chatbot: Prospective Implementation Study
Two-phase prospective implementation study of a human-in-the-loop suicide risk screening, detection and clinical escalation protocol inside Sona, a generative AI mental health and wellness chatbot; the study was funded by Digital Therapeutics, Inc, which employs the authors. Phase 1 escalated positive PHQ-9 item 9 screens to a clinician crisis team. Phase 2 added automated post-conversation transcript review by a large language model routing classifier that raised alerts for clinician outreach and Ask Suicide-Screening Questions (ASQ) assessment. The routing classifier was also tested on the held-out test set of the clinician-annotated CRADLE-Dialogue benchmark.
Publisher
JMIR Formative Research (JMIR Publications)
Published
23 Oct 2026
Added
today
Key Findings
- Phase 1 (April 2026): 81 of 624 participants (13%) endorsed PHQ-9 item 9; clinician outreach began a mean 11.2 minutes after the alert (SD 13.2; range 1-54) and 25 of 81 (31%) were reached
- Phase 2 (July 2026, participants with at least mild PHQ-9 or GAD-7 symptoms): 133 of 650 (20.5%) screened positive at baseline; over 30 days, 284 risk-alert events (234 PHQ-9-triggered, 50 conversational) arose among 172 participants (26.5%), and clinicians reached 73 of 172 (42.4%)
- Across both phases 71 complete ASQ assessments were documented for 53 participants: 30 (42%) negative, 40 (56%) nonacute positive and 1 (1%) acute positive, which was escalated to local crisis services for a welfare check
- Of the 99 phase-2 participants never reached by a clinician, 55 (56%) went on to converse with the chatbot after their first alert day
- On the CRADLE-Dialogue held-out test set the unchanged production classifier alerted on all 58 dialogues with ongoing active suicidal ideation (sensitivity 100%, 95% CI 93.8%-100%) and did not alert on 307 of 309 strict-negative dialogues (specificity 99.4%, 95% CI 97.7%-99.8%)
Methodology Notes
Single-arm implementation study nested in an open-label feasibility trial of the Sona chatbot. US English-speaking adults recruited through Prolific and CloudResearch who were employed or insured; phase 1 recruited April 2026 (624 completed baseline) and phase 2 July 2026 (650 completed baseline). The conversational pathway sent each completed transcript to an LLM routing classifier (Claude Opus 4.7) instructed to label CRISIS_988 when there was evidence of suicidal ideation, current or imminent self-harm, or imminent harm to others; alerts were triage triggers, and risk stratification was done by clinicians with the ASQ. The parent study received WCG IRB exemption under 45 CFR 46.104(d)(3). Descriptive feasibility outcomes only; no control arm and no clinical-outcome measures. All three authors are employed by and hold stock options in Digital Therapeutics, Inc, which funded the study. Submitted 2 July 2026, accepted 9 October 2026, published 23 October 2026.
Sources
https://formative.jmir.org/2026/1/e106118(opens in a new tab) (primary)
PubMed record(opens in a new tab)
Archived snapshot (Wayback Machine)(opens in a new tab) — preserved against link rot
Topics
Authors
Suzette Glasner, Darcy Michero, Matthew Stewart
Tags
Cite This
APA
Suzette Glasner, Darcy Michero, Matthew Stewart. (2026). Human-in-the-Loop Suicide Risk Detection and Clinical Escalation in a Generative AI Mental Health Chatbot: Prospective Implementation Study. JMIR Formative Research (JMIR Publications). https://formative.jmir.org/2026/1/e106118