Skip to main content
Preprint Credible — Major labs, established NGOs, reputable named-author preprints

CompAIssionate Other: Identity Assumptions in LLM Responses to Distress

Paired counterfactual audit of how six language models respond to suicidal-ideation and distress posts when a one-sentence social-identity disclosure is added. Posts come from two datasets and were screened with a C-SSRS-based LLM rater. The authors find small aggregate differences in refusal, risk recognition and crisis-resource provision, but several models attribute distress to presumed identity-related adversity more often when marginalized identities are disclosed.

Read at arxiv.org (opens the original source in a new tab) Archived copy(opens in a new tab)

Publisher

University of Illinois Urbana-Champaign (arXiv preprint)

Published

7 Nov 2026

Added

3 days ago

DOI

—

Key Findings

  • Across the six models and both datasets, refusal rates after marginalized and majority disclosures differed by at most 2% in all but one case, and no model increased refusal for a marginalized disclosure.
  • Gemini-3.8-Flash linked immigrant writers' distress to their identity in 14% of responses on Reddit SI and 34% on SH-Detection (Qwen3.5-9B: 11% and 20%).
  • Responses to LGBTQ+ disclosures included an identity-specific resource in 15 to 28% of cases and responses to Muslim and Black disclosures in 8 to 16%, whereas Latino, disabled and immigrant writers received one in fewer than 3% of responses.

Methodology Notes

Preprint (arXiv 2611.10808, submitted 7 November 2026), not peer reviewed. Models: GPT-4o-mini, Gemini-3.8-Flash, MedGemma-1.5-4B, Phi-4-mini, Qwen3.5-9B, Llama-3.1-8B. Posts screened by an LLM rater applying C-SSRS items in two passes (Cohen's kappa 0.86 to 0.91), yielding 654 suicidal-ideation and 153 distress posts from Reddit SI plus 500 posts from SH-Detection; 16 identity and control conditions and two prompting protocols, giving 25,824 responses per model on Reddit SI and 16,000 on SH-Detection. A small manual check of flagged attributions is reported by the authors. Figures verified by the curator in the arXiv HTML full text on 2026-11-12.

Topics

Suicide risk assessmentCrisis detectionVulnerable usersModel behaviorEval methodology

Authors

Drishti Goel, Ethan M. Nguyen, Violeta J. Rodriguez, Koustuv Saha

Tags

counterfactual-auditidentityc-ssrsequitycompassionate-othering

Cite This

APA

Drishti Goel et al. (2026). CompAIssionate Other: Identity Assumptions in LLM Responses to Distress. University of Illinois Urbana-Champaign (arXiv preprint). https://arxiv.org/abs/2611.10808