An AI-based mental health guardrail and dataset for identifying psychiatric crises in text-based conversations

Key Findings

• — VMHG reached 0.990 sensitivity and 0.992 specificity (F1 0.939) on the Verily crisis dataset, outperforming the OpenAI omni-moderation-latest and NVIDIA NeMo Guardrails baselines
• — Category-level sensitivity ranged 0.917-0.992 with specificity of at least 0.978 across the psychiatric-crisis categories evaluated
• — Introduces a clinician-labeled dataset of 1,800 simulated crisis messages; data and code are available on researcher request rather than openly released

Methodology Notes

Published in npj Digital Medicine (volume 9, article 407; DOI 10.1038/s41746-026-02579-5, 2026-04-03). Author team from Verily Life Sciences with John Torous (Beth Israel Deaconess/Harvard). Evaluation uses simulated rather than real-user messages; both evaluation datasets are clinician-labeled; comparison baselines are general-purpose content-moderation guardrails.

Sources