Disclaim and Diagnose: Consumer-Facing Artificial Intelligence Chatbots and Medical Device Criteria: Cross-Sectional Content Analysis

Key Findings

• — Across 72 tool-vignette instances, 66 (91.7%) provided diagnostic information: a primary diagnosis in 15 (20.8%), a possible diagnosis in 20 (27.8%) and a differential diagnosis in 31 (43.1%)
• — Treatment content appeared in 51 (70.8%) instances, including prescription medication recommendations in 36 (50.0%) and over-the-counter medication in 31 (43.1%)
• — Disclaimers were present in 55 (76.4%) instances; the most common stated that the tool does not provide a diagnosis or medical advice (39/72, 54.2%), and 6 (8.3%) disavowed liability
• — For the potential-depression vignette, 22 of 24 tools gave diagnostic information (16 as a differential diagnosis) and 12 of 24 recommended prescription medication

Methodology Notes

Cross-sectional content analysis conducted in July 2026; vignettes structured to follow American Medical Association guidance on safe chatbot use for health; tools included general assistants (Claude, Gemini and Grok are quoted) and symptom checkers (Ada Health, MySymptom AI). One author coded all responses with an a priori codebook, consulting the second author on ambiguous cases; no inter-rater reliability reported. Reporting followed the CHART guideline. Accuracy of the advice was not assessed. Funded by NIDA grant R01DA04853. JMIR Preprints date stamp 20 October 2026; open peer review period from 21 October 2026. Verified from the submitted-manuscript text (11 pages) read through r.jina.ai on 2026-10-22; the JMIR preprint page returns a 202 WAF response directly.

Sources