SIM-UT: A User Simulation based Evaluation Framework for Medical Triage Chatbots

Key Findings

• — Across 5,000+ multilingual simulated conversation turns, agents de-escalated after correctly identifying an emergency when the simulated user minimised symptoms without introducing new clinical information.
• — Reported cases include an agent telling a user with reduced fetal movements at 32 weeks that "Feeling a couple of small movements can be normal" and describing bleeding at 18 weeks as "usually not a problem".
• — In the Kenya deployment, initially 2 of the 9 minimisation scenarios held the escalation; after iteration both the ChatGPT- and MedGemma-based agents held it in every minimisation scenario.
• — Several Kenya escalation failures came from the routing layer: a session-closing node fired on acknowledgements such as "thanks" while a concern was open, and a clinician-review action returned the user to the main menu, where a following disclosure of "voices telling me to harm the baby" landed.
• — Over iteration rounds, scope adherence rose from 84% to 100% (ChatGPT) and 74% to 97% (MedGemma) in South Africa and from 40% to 95% and 53% to 98% in Kenya; in India eligibility accuracy rose from 86% to 92% and unsafe enrolments fell from 4 of 18 to 0 of 14. Scenario sets changed between rounds.

Methodology Notes

Simulated users and the LLM judge were both claude-sonnet-4-6; versions of the ChatGPT, Claude and MedGemma models behind the agents are not stated. Ground-truth triage levels were set by a single clinician in Kenya and South Africa; in India the evaluator and agent share a model family. De-escalation is reported as case examples and one scenario count, not as per-model rates. The lead organisation (UserTrace) sells the evaluation approach; deployers co-authored. Datasets and framework promised as open source later. Not peer reviewed. arXiv v1 submitted 2026-10-21 (announced 2026-10-23). Verified from the arXiv abs page and HTML v1.

Sources