Google Search: AI Overview & AI Mode — AI Risk Assessment

Key Findings

• — Across 652 test prompts spanning 13 clinical conditions (psychosis, eating disorders, suicide, self-harm, mania, depression, anxiety, substance use, PTSD, mood, ADHD, OCD, ODD), 214 prompts per feature were identified as clearly warranting a hotline or medical referral; AI Overview supplied an appropriate resource in 58% of those moments and AI Mode in 77%
• — On explicit suicidal ideation specifically, AI Mode produced a hotline or medical referral in 96% of cases while AI Overview missed or responded inappropriately in documented examples
• — The organisation sets an explicit numerical minimum of 95% crisis detection, argued from the upper range of documented crisis-line effectiveness, and both features fall below it
• — The report rates the product Unacceptable Risk on two of eight principles (Keep Kids & Teens Safe; Put People First) and High Risk on five more, with only Use Data Responsibly at Moderate Risk
• — Neither AI Overview nor AI Mode can be disabled in Search, unlike the standalone Gemini chatbot, which parents and schools can turn off via Family Link or the Google Admin Console
• — In the Academic Integrity plan (180 prompts), AI Mode completed 100% of the homework assignments tested
• — Testers found no meaningful difference in product behaviour between the 11-year-old supervised account and the 15-year-old teen account

Methodology Notes

Structured product testing, 19 May to 1 July 2026, seven test plans: Mental Health (652 prompts), Developmental Appropriateness (502), Historical Facts (278), Academic Integrity (180) and three others, totalling over 2,600 interactions and over 2,100 analysed source citations. All testing was conducted in the United States from accounts based in the San Francisco Bay Area, and Google Search personalises on location and account signals, so results may not generalise to other locales. The assessment excludes the standalone Gemini app, third-party integrations, API access, and long-term effects of use. Publication date taken from the PDF filename (csm-ai-risk-assessment-google-search-07142026_0.pdf) and PDF Title metadata ('Google Search Full-Risk-Assessment 07142026'); the landing page's 'Updated Jul 14, 2026' label is a page-edit date and is not a reliable date source in general. Funding disclosure carried in the document: the Institute is funded by philanthropy and by industry, including makers of some of the technologies it evaluates, while asserting editorial independence.

Sources