aceChatGPT Health performance in a structured test of triage recommendations
ChatGPT Health was launched in January 2026 as OpenAI's consumer health tool and has reached millions of users. Here we conducted a structured stress test of triage recommendations using 60 clinician-authored vignettes across 21 clinical domains under 16 factorial conditions, yielding 960 total responses. Performance followed an inverted U-shaped pattern, with the most dangerous failures concentrated at clinical extremes-nonurgent presentations (35%) and emergency conditions (48%). Among gold-standard emergencie…
In a 960-response vignette test, ChatGPT Health undertriaged 52% of gold-standard emergencies, directing diabetic ketoacidosis and impending respiratory failure to 24-48 h evaluation instead of the emergency department, with failures concentrated at clinical extremes and triage shifting toward less urgent care when by-
Findings are based on clinician-authored vignettes rather than prospective real-world patient encounters, and analyses of race, sex and barriers to care were underpowered to exclude clinically meaningful differences, requiring prospective validation before consumer-scale deployment.
Evidence
- Peer-reviewedNature Medicine2026-02-23
How should this claim be treated?
Truvace Impact Record TRV-2026-0181, v1: “ChatGPT Health performance in a structured test of triage recommendations.” Truvace, 2026-07-13. /record/TRV-2026-0181 (accessed at citation time). sha256 4bf89b7554fae7b2…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-0181 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.