Reliability of LLMs as medical assistants for the general public: a randomized preregistered study

Key Findings

• — Tested alone, the LLMs identified relevant conditions in 94.9% of cases and the correct disposition in 56.3% on average
• — Participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and the correct disposition in fewer than 44.2%, no better than the control group
• — Standard medical-knowledge benchmarks and simulated patient interactions did not predict the failures observed with human participants
• — The authors identify user interaction as the deployment challenge and recommend systematic human user testing before public deployment in healthcare

Methodology Notes

Randomized, preregistered study; 1,298 UK adults recruited on Prolific produced 2,400 conversations between 21 August and 14 October 2024; ten scenarios written by three doctors who unanimously agreed on the correct disposition on a five-point scale from self-care to ambulance; three LLM arms (GPT-4o, Llama 3, Command R+) plus a control arm using any source of the participant's choice. Received 2025-05-04, accepted 2025-10-22, published online 2026-02-09 (Nature Medicine 32:609-615). A Publisher Correction (2026-04-17) fixed the y-axis labels of Fig. 2a only. Scenarios and experimental data are released (github.com/am-bean/HELPMed; huggingface.co/datasets/ambean/HELPMed). Earlier preprint: arXiv 2504.18919, 'Clinical knowledge in LLMs does not translate to human interactions' (2025-04-26). Open access.

Sources