Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
Tests six LLMs on classifying posts across the Columbia-Suicide Severity Rating Scale (C-SSRS) 7-point severity ladder, comparing model outputs with human annotations. Assesses automated suicide-risk screening and characterises misclassification patterns.
Publisher
arXiv
Published
11 May 2025
Added
3 months ago
DOI
—
Key Findings
- Claude and GPT models aligned closely with human C-SSRS annotations; ordinal error varied across models
- Models can approximate C-SSRS severity classification but make consequential misclassifications
- Authors stress human oversight remains necessary for any deployment
Methodology Notes
Preprint (arXiv 2505.13480, v1 11 May 2025). Six LLMs evaluated on C-SSRS 7-point severity classification against human annotations.
Sources
arXiv abstract(opens in a new tab) (primary)
Archived snapshot (Wayback Machine)(opens in a new tab) — preserved against link rot
Authors
Avinash Patil, Siru Tao, Amardeep Gedhu
Tags
Cite This
APA
Avinash Patil, Siru Tao, Amardeep Gedhu. (2025). Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale. arXiv. https://arxiv.org/abs/2505.13480
Related Insights
Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment
Psychiatric Services (American Psychiatric Association); RAND-led author team · 26 Aug 2025
Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse
PLOS Digital Health · 27 Apr 2026
A machine learning approach to identifying suicide risk among text-based crisis counseling encounters
Frontiers in Psychiatry · 23 Mar 2023
Assessment of Suicidal Intention: The Scale for Suicide Ideation
Journal of Consulting and Clinical Psychology (American Psychological Association) · 1 Jan 1979
The Columbia–Suicide Severity Rating Scale: Initial Validity and Internal Consistency Findings From Three Multisite Studies With Adolescents and Adults
American Journal of Psychiatry (American Psychiatric Association) · 1 Dec 2011
Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
IEEE (2025 IEEE International Conference on Future Machine Learning and Data Science, FMLDS) · 2 Nov 2025
Ground Truths in Suicide Research: The Current State of AI-Based Suicide Detection in Social Media
Association for Computational Linguistics (Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology, CLPsych 2026) · 1 Jul 2026
Conversational trajectory degrades large language model detection of suicidal ideation relative to clinicians: a preregistered study
medRxiv (Beth Israel Deaconess / Harvard digital-psychiatry group) · 14 Jul 2026
From Symptom Networks to Conversation Networks: A Cross-Sectional Study Mapping the Topology of Suicide-Related Clinical Dialogue
medRxiv (preprint); University Hospital Frankfurt; UKP Lab, Technical University of Darmstadt · 29 Sept 2026
Preparing AI chatbots to respond to patient distress and suicidality in high-risk healthcare settings
npj Digital Medicine (Nature Portfolio) · 22 Sept 2026