RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions

Key Findings

• — Five empirically derived risk dimensions for caregiving-support responses: Inattention, Bias and Stigma, Information Inaccuracy, Uncritical Affirmation, and Epistemic Arrogance.
• — Six state-of-the-art LLMs were evaluated on more than 20,000 caregiver queries (an ADRD caregiver set from r/Alzheimers and ALZConnected and a general caregiver set from r/CaregiverSupport).
• — Rubric-guided refinement reduced risk components by 45% to 98% after a single iteration across models.
• — Evaluator validity was checked against authors, external annotators and clinician co-authors, with agreement in the high 80s to mid 90s per the benchmarks beat's read of the PDF.
• — The authors argue general-risk frameworks (toxicity, hallucination, policy violation) miss the relational risks of caregiving contexts and release the datasets for contextual risk evaluation.

Methodology Notes

Rubric development grounded in care ethics, evaluation of six LLMs (including GPT-4o-mini and Claude Sonnet 4 alongside small open and medical models) on roughly 20,000 real caregiver posts, rubric-guided response refinement, multi-rater validation including clinicians. Published in Findings of ACL 2026 (July 2026), pages 35620 to 35638, DOI 10.18653/v1/2026.findings-acl.1774. Verified at the ACL Anthology on 2026-09-15; per-model and per-dimension figures beyond the abstract come from the beat's PDF read.

Sources