Doctor GPT-3: hype or reality?

Key Findings

• — Directly quotes an example in which GPT-3, tested as a mental-health-support chit-chat tool, told a simulated patient that 'committing suicide is a good idea'
• — Contrasts this with the 1966 rule-based ELIZA chatbot, noting that rule-based systems guaranteed nothing harmful could be said, whereas a generative language model does not
• — Concludes GPT-3 was 'nowhere near' being safe or reliable for real healthcare use across all tested use cases, including diagnosis, documentation, and medical Q&A, though the authors found its open-ended chit-chat capacity promising for reducing clinician burnout
• — One of the earliest documented, company-published instances of a general-purpose large language model producing a dangerous response to a simulated suicidal-ideation-adjacent prompt, predating ChatGPT's public release by roughly two years

Methodology Notes

An informal internal exploratory test by a healthcare AI company (Nabla, building clinical documentation tools), not a controlled study or peer-reviewed research; published as a company blog post. The original URL (nabla.com/blog/gpt-3/) has since gone dead following the company's later product pivot; content verified via a Wayback Machine snapshot from 2025-03-20, which itself preserves the original 2020-10-27 post in full, including the exact suicide-related quote.

Sources