Generative Large Language Models in Mental Health Care Settings: Systematic Review and Meta-Analysis

Key Findings

• — 66 studies were included: 37 vignette or simulation studies, 22 retrospective studies and 7 prospective studies.
• — Applications: screening and diagnosis (29 studies), clinical decision support (14), treatment support (10), documentation and monitoring (6), patient education (4) and patient engagement (3).
• — Meta-analysis of 8 screening and diagnosis studies: GPT-4 pooled sensitivity 0.83 (95% CI 0.38 to 0.97; 95% prediction interval 0.02 to 1.00) versus GPT-3.5 0.70 and GPT-3 0.61; GPT-4 specificity 0.77 (95% CI 0.52 to 0.91; prediction interval 0.10 to 0.99).
• — Certainty of evidence was generally low across domains; only documentation and monitoring reached moderate certainty.
• — The authors conclude the evidence is too heterogeneous, indirect and uncertain to support routine unsupervised use, particularly for diagnosis, risk assessment, crisis response or therapeutic interaction, and that broad accessibility should not be mistaken for clinical readiness.

Methodology Notes

Systematic review and meta-analysis; quantitative synthesis restricted to OpenAI models and to studies reporting sensitivity and specificity; prediction intervals span nearly 0 to 1, so pooled estimates are unstable. Limitations named by the authors: indirectness from vignette designs, uncertain representativeness, inconsistent outcome reporting, sparse prospective real-world evaluation. Published 2026-08-31 in JMIR AI volume 5. The publisher's article page returns an anti-bot interstitial (HTTP 202) from the sweep host, so verification used the Crossref record, which carries the publisher-deposited structured abstract; funding and competing-interest statements were not visible through that route.

Sources