Truace
Subscribe

ChatGPT-5 versus nurse triage of yellow and green adult emergency department patients

Source article: Agreement and classification performance of ChatGPT-5-generated and nurse-led emergency department triage using an expert reference decision: A retrospective study

This study evaluated the agreement and classification performance of ChatGPT-5-generated triage compared with nurse triage using an expert reference decision in adult emergency department patients. In this retrospective cross-sectional study, 1564 adult patients with documented yellow or green triage classifications were included. Standardized triage-level patient information was entered into ChatGPT-5 using an identical prompt for all patients. Agreement was assessed using Cohen's κ, and classification performa…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1421· cite & verify
Agreement and classification performance of ChatGPT-5-generated and nurse-led emergency department triage using an expert reference decision: A retrospective study
Construction site tour of the Skilled Nursing Residence, in Edgartown, on the island of Martha's Vineyard, in Dukes County, Maryland on August 5, 2024 - 42 by USDAgov. Public domain
Trace impact readingContested
P 67The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 71The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Researchers compared ChatGPT-5 triage and nurse triage with an expert reference decision in a retrospective review of 1564 adult emergency department patients documented as yellow or green. The same prompt was used for every patient, and standardized triage-level information was entered into the model.

ChatGPT-5 agreed more closely with the experts (κ 0.670 vs. 0.467) and was more accurate and more sensitive. The result suggests AI could support triage in ambiguous lower-acuity cases. It was retrospective and used only yellow and green categories, so it does not show how the tool would perform live or for more urgent patients. The cost of its extra yellow assignments for green patients is also unclear.

Main points

  1. The study included 1564 adult patients with documented yellow or green triage classifications.
  2. Agreement with the expert reference was κ = 0.670 for ChatGPT-5 and 0.467 for nurses.
  3. Overall accuracy was 83.5% for ChatGPT-5 and 73.3% for nurse triage.
  4. Sensitivity for yellow classification was 82.0% for ChatGPT-5 versus 49.0% for nurses.
  5. ChatGPT-5 over-assigned yellow to expert-green patients more often than nurses did (15.0% vs. 2.3%).

The gain

For adult emergency patients classed yellow or green, ChatGPT-5 triage reached higher accuracy and sensitivity against an expert reference than nurse triage, so it undertriaged fewer yellow patients.

The problem

In expert-reference green patients, ChatGPT-5 triage assigned yellow more often than nurses did (15.0% vs. 2.3%), which suggests more over-triage of low-acuity patients.

The rundown

The design was retrospective and cross-sectional. Standardized triage-level patient information was entered into ChatGPT-5 with an identical prompt, and agreement was measured with Cohen's κ alongside sensitivity, specificity and accuracy.

The cohort had a mean age of 39.8 years and was 55.9% female. Experts classified half of the patients as yellow. ChatGPT-5 classified 48.5% as yellow, whereas nurses classified only 25.6% as yellow.

Nurses sent 51.0% of expert-reference yellow patients to green, compared with 18.0% for ChatGPT-5. The trade-off ran the other way for green patients, where ChatGPT-5 gave yellow more often. The authors conclude that ChatGPT-5 approximated the expert decision more closely only under the conditions tested.

What this doesn’t fix

The findings apply only to the conditions evaluated: retrospective data on adult patients with documented yellow or green classifications, so performance on other acuity levels or in live use is not established.

Sources

  1. Peer-reviewedInternational Emergency Nursing2026-10-12

The debate