Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries
An audit of what three free consumer generative-search products (ChatGPT, Perplexity, Google AI Overview) cite when answering mental-health questions. Twenty English questions were run under two prompt conditions, with a subset of three questions also translated into six further languages of varying resource tiers. Every citation was classified with a nine-category organisational typology applied by a deterministic classifier validated against human coding, and the typology, classifier and annotated corpus are released.
Publisher
arXiv (Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School; Pontificia Universidad Javeriana; Tufts University School of Medicine)
Published
31 Aug 2026
Added
today
DOI
—
Key Findings
- 15,942 citations were recorded across 1,140 responses and 1,713 unique domains
- Citations were heavily concentrated: the ten most-cited domains accounted for 43.6% of English citations, with government, commercial-health and academic sources closely matched at roughly 22% each
- Platforms differed little in typical citation volume but sharply in consistency and favoured source types; explicitly requesting sources shifted composition only modestly
- Non-English queries surfaced fewer citations and were routed to language-appropriate resources at significantly lower rates
Methodology Notes
arXiv 2609.00319, v1 submitted 2026-08-31 (announced 2026-09-02); no DOI for the paper (the dataset carries DOI 10.57967/hf/10067). Affiliations read from the PDF title block: nine authors at the Division of Digital Psychiatry, Beth Israel Deaconess Medical Center / Harvard Medical School, plus Pontificia Universidad Javeriana (Bogotá) and Tufts University School of Medicine. Free consumer tiers only; twenty seed questions; the multilingual arm covers only three of the twenty questions; classification is by a human-validated deterministic classifier rather than per-citation human review. Code (github.com/mindbench-ai/search-source-audit) and data (Hugging Face MindBench/search-source-audit, CC-BY-4.0, languages en/es/ja/uk/hi/ne/tw) verified live.
Sources
Authors
Phuong Anh Nguyen, Jill Noorily, Matthew Flathers, Haruka Notsu, Laura Ospina-Pinillos, Tommy Nguyen, Samantha Clark, Aoife Keane, Grace Thompson, John Torous
Tags
Cite This
APA
Phuong Anh Nguyen et al. (2026). Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries. arXiv (Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School; Pontificia Universidad Javeriana; Tufts University School of Medicine). https://arxiv.org/abs/2609.00319
Related Insights
Public use of a generalist LLM chatbot for health queries
Nature Health · 16 Apr 2026
From Diagnoses to Treatments, Why Americans Use AI Chatbots for Health
Pew Research Center · 25 Aug 2026
HealthBench-Psych: A Mental Health Subset of OpenAI's HealthBench
arXiv (Division of Digital Psychiatry, Beth Israel Deaconess Medical Center) · 25 Aug 2026
Google Search: AI Overview & AI Mode — AI Risk Assessment
Common Sense Media Youth AI Safety Institute · 14 Jul 2026