Aligning a locally deployable 4-billion-parameter language model to a psychiatrist panel for privacy-preserving suicide-risk screening of spoken diaries

Key Findings

• — 450 transcribed voice diaries from 37 psychiatric patients were annotated independently by three psychiatrists
• — Panel-aligned 4B model: quadratic-weighted kappa 0.571 for risk tier and 0.843 for suicidality, versus 0.476/0.439 unaligned 4B, 0.507/0.511 zero-shot 8B, 0.495/0.474 embedding baseline and 0.586/0.801 for the best frontier arm
• — High-risk sensitivity was 0.684, resting on 7 patients
• — Preference optimization over all under-call pairs reached sensitivity 0.895 but collapsed agreement to kappa 0.310; restricting to high-risk-miss pairs amplified fourfold gave sensitivity 0.789 and kappa 0.596
• — Training on majority-vote labels instead of the panel distribution cost 0.26 sensitivity

Methodology Notes

QLoRA fine-tuning under patient-grouped 5-fold cross-validation; patient-clustered intervals; parse failures counted as missed alarms. Small sample (37 patients; 7 drive the high-risk sensitivity estimate). Affiliations and country not verified (SSRN page 403 on direct and r.jina.ai). Verified from the Crossref-deposited abstract only; SSRN posting date not readable, 2026-10-18 is the DOI creation date. Published date is the Crossref DOI creation date (2026-10-18T04:16 UTC); the SSRN posting date could not be read.

Sources