Guide to Evaluation Perspectives on AI Safety in the Healthcare Sector: Toward the Realization of Trustworthy AI (Version 1.0)

Key Findings

• — Ten healthcare evaluation perspectives are enumerated at section 3.2: control of toxic output, prevention of misinformation and manipulation, fairness and inclusion, addressing high-risk and unintended use, privacy protection, security, explainability, robustness, data quality and verifiability.
• — Input-layer requirement as printed: implement flows to guide users toward professional support lines or trigger emergency response procedures when input related to self-harm, suicidal ideation or harm to others is detected, described as a particularly important safeguard in the healthcare sector.
• — The Product Implementation Phase Checklist (Table 4-12) makes that a checkable item asking whether detection of dangerous input and a flow to professional support lines have been implemented.
• — The test-scenario table separates two escalation classes: emergency response (inputs such as chest pain or losing consciousness, routed to emergency services) and self-harm or suicidal ideation (routed to a professional support resource).
• — The risk framing names direct harm to life and physical health, including the risk of inducing self-harm or harm to others, and grounds it in the 2023 Belgian suicide following weeks of chatbot conversation and the 2023 suspension of the NEDA eating-disorder chatbot Tessa.
• — Section 4.4.1 raises the evaluation risk level when the user population includes patients with mental health conditions.
• — A named vendor case study (Awarefy) documents character design that deliberately avoids expert-evoking or romantic-partner personas to head off dependency, an in-house team of certified and clinical psychologists across planning and output evaluation, and LLM-as-a-Judge monitoring plus a permanent in-app list of public support resources; it also names the unresolved case of users who reply 'please don't tell me to talk to a specialist'.

Methodology Notes

Not an empirical study: a consensus evaluation guide produced by an industry sub-working group convened by J-AISI, with participation from the Japan Digital Health Alliance. The illustrative harms are secondary citations to press coverage rather than original incident analysis. Compliance is voluntary and no conformity-assessment scheme is attached. The English edition is a translation of the same Version 1.0 document; quote the Japanese text where wording is load-bearing. Published 2026-04-03 per the PDF cover page; the 98-page English PDF (4,004,354 bytes) was downloaded and read.

Sources