CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection
Introduces a clinician-annotated benchmark for detecting seven clinically-defined crisis and safety-risk types (including suicidal ideation, sexual assault, domestic violence, child abuse, and sexual harassment) in text. Comprises 600 clinician-annotated evaluation examples, 420 development examples, and roughly 4,000 ensemble-labelled training instances, with temporal labels.
Publisher
arXiv (Emory University-led)
Published
27 Oct 2025
Added
today
DOI
—
Key Findings
- Covers seven clinically-defined crisis/safety-risk categories in a single benchmark
- Provides 600 clinician-annotated evaluation examples plus development and ensemble-labelled training splits
- Incorporates temporal labels for crisis detection, distinguishing it from narrower single-risk benchmarks
Methodology Notes
Preprint/benchmark (arXiv 2510.23845, v1 27 October 2025; v2 January 2026). Accepted to EACL 2026. Clinician-annotation methodology; annotations aligned to clinical guidelines.
Sources
arXiv abstract (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Grace Byun, Rebecca Lipschutz, Sean T. Minton, Abigail Lott, Jinho D. Choi
Tags
Cite This
APA
Grace Byun et al. (2025). CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection. arXiv (Emory University-led). https://arxiv.org/abs/2510.23845
Related Insights
Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs
arXiv (ELLIS Alicante-led) · 29 Sept 2025
VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health
arXiv (Spring Health / Slingshot AI-affiliated author team) · 4 Feb 2026
Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines (PsyCrisisBench)
arXiv (Chinese research team) · 2 Jun 2025
Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse
PLOS Digital Health · 27 Apr 2026
Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models
JMIR Mental Health · 11 Jun 2026