Risk and Refusal: Mechanisms of LLM Responses to Suicidal Ideation

Key Findings

• — At baseline, 17.8-63.9% of responses to r/SuicideWatch posts were flagged for risk (13.4-45.8% on SH-Detection, 14.8-24.3% on X-Sensitive), driven mainly by inattention to the disclosure as well as refusal to engage.
• — Supplying the SI classification lowered refusal but significantly raised flagged risk for four of eight models on the Reddit set (for example GPT-4o-mini 39.4% to 56.7%, Llama-3.1-8B 55.5% to 71.8%); for Gemma-3-27B on X-Sensitive, information inaccuracy rose from 13.6% to 63.2%, including non-credible support resources.
• — The clinician-developed protocol significantly reduced response risk in 22 of 24 model-dataset comparisons; with classification plus guidance, flagged risk fell to 1.9-3.3% on the Reddit set with significant reductions in all 24 comparisons.
• — A refusal direction estimated from generic harmful and harmless instructions predicted refusal of SI disclosures (AUROC 0.73-0.90 for Phi-4-mini, 0.88-0.93 for Llama-3.1-8B); ablating it reduced explicit SI acknowledgment and raised flagged risk in five of six model-dataset comparisons (Phi-4-mini on Reddit SI 44.7% to 71.4%).

Methodology Notes

Datasets: 1,504 r/SuicideWatch posts with labels from an Interpersonal Theory of Suicide classifier, a 1,500-post sample of SH-Detection, and X-Sensitive. Models: GPT-4o-mini, MedGemma-1.5-4B, Phi-4-mini, Qwen3.5-9B, Gemini-3.8-Flash, Llama-3.1-8B, Gemma-3-27B and gpt-oss-20b; mechanistic work on the six open-weight models. Refusal and RubRIX (29 binary items, five dimensions) scored by separate GPT-5-nano judges blind to model and condition; Holm-corrected tests. Single-turn responses to public posts, mostly small open models; no human agreement figure for the refusal or RubRIX judges was found in the main text. v1 submitted 2026-10-15 22:34 UTC; announced in the Monday 19 Oct mailing (abstract and HTML pages were already live at 01:40 UTC 10-19). Verified on the arXiv abstract page and HTML full text (v1, HTTP 200). Curator re-verified 2026-10-19 on the arXiv abstract page (v1 submitted 15 Oct 2026) and HTML full text: all quoted figures read in Section 4.

Sources