A machine learning approach to identifying suicide risk among text-based crisis counseling encounters
Develops a transformer-based model on 5,992 SafeUT crisis-counseling encounters to detect conversation-level suicide risk, benchmarked against a tf-idf baseline. Reports strong discrimination and better sensitivity on higher-risk cases despite noisy human counsellor labels, and positions the model as decision support.
Key Findings
- Transformer model reached ROC AUC ~90.4%, outperforming a tf-idf baseline
- Greater sensitivity to genuine suicide risk than the baseline, though with a non-trivial false-negative rate
- Manual review found the model flagged real risk indicators that counsellors sometimes missed
Methodology Notes
Peer-reviewed, Frontiers in Psychiatry (23 March 2023). RoBERTa-based classification on 5,992 SafeUT crisis-text encounters with human counsellor labels; conversation-level risk prediction.
Sources
Frontiers in Psychiatry (PMC full text) (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Meghan Broadbent, Mattia Medina Grespan, Katherine Axford, Xinyao Zhang, Vivek Srikumar, Brent Kious, Zac Imel
Tags
Cite This
APA
Meghan Broadbent et al. (2023). A machine learning approach to identifying suicide risk among text-based crisis counseling encounters. Frontiers in Psychiatry. https://pmc.ncbi.nlm.nih.gov/articles/PMC10076638/
Related Insights
Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment
Psychiatric Services (American Psychiatric Association); RAND-led author team · 26 Aug 2025
Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
arXiv · 11 May 2025
Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines (PsyCrisisBench)
arXiv (Chinese research team) · 2 Jun 2025
Risk prediction using natural language processing of electronic mental health records in an inpatient forensic psychiatry setting
Journal of Biomedical Informatics (Elsevier) · 1 Oct 2018