Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse
A peer-reviewed, three-study evaluation of GPT-4 and Claude on detecting suicidality, intimate partner violence, and substance misuse from lived-experience vignettes, including a supervised multi-agent risk-assessment chatbot. Reports accuracy and severity alignment across the three risk domains.
Publisher
PLOS Digital Health
Published
27 Apr 2026
Added
yesterday
Key Findings
- Strong overall accuracy and severity alignment across the three psychosocial risk domains, with suicide the hardest to assess
- A supervised multi-agent risk-assessment chatbot performed well but showed occasional protocol gaps
- Covers suicide risk, intimate partner violence, and substance misuse in a single multi-risk evaluation with clinical framing
Methodology Notes
Peer-reviewed (PLOS Digital Health, DOI 10.1371/journal.pdig.0001352). Three linked studies using lived-experience vignettes; open access.
Sources
PLOS Digital Health article (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Laura M. Vowels, Pranika Vohra, Danyang Li, Pegah Zeinoddin, Alex Elswick, Tiffany Marcantonio, Nathan D. Wood, Matthew J. Vowels
Tags
Cite This
APA
Laura M. Vowels et al. (2026). Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse. PLOS Digital Health. https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001352
Related Insights
Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment
Psychiatric Services (American Psychiatric Association); RAND-led author team · 26 Aug 2025
VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health
arXiv (Spring Health / Slingshot AI-affiliated author team) · 4 Feb 2026
Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs
arXiv (ELLIS Alicante-led) · 29 Sept 2025
Risk prediction using natural language processing of electronic mental health records in an inpatient forensic psychiatry setting
Journal of Biomedical Informatics (Elsevier) · 1 Oct 2018
CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection
arXiv (Emory University-led) · 27 Oct 2025
Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
arXiv · 11 May 2025
Classifying the Information Needs of Survivors of Domestic Violence in Online Health Communities Using Large Language Models: Prediction Model Development and Evaluation Study
Journal of Medical Internet Research (JMIR) · 12 May 2025
Detecting Patterns of Intimate Partner Violence Using Qualitative Analyses and Machine Learning Algorithms
Prevention Science (Springer) · 6 May 2026
Are Natural Language Processing Tools Ready for Predicting Violence Toward Self or Others?
Psychiatric Annals (SLACK Incorporated) · 1 Apr 2026