Moderate evidence
More
ChatGPT and Gemini read ER heart tracings worse than doctors, 89-patient study finds
Study title: Performance of AI models in ECG interpretation and risk stratification for major adverse cardiac events in the emergency department
- Evidence type
- Controlled or validated comparison
- Sample
- n = 89
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-3038cite or check it
Study title: Performance of AI models in ECG interpretation and risk stratification for major adverse cardiac events in the emergency department
A prospective study of 89 emergency department patients with chest pain or palpitations compared ECG reads by ChatGPT, Gemini and emergency specialists against a cardiologist consensus. Emergency specialists agreed closely with cardiologists (κ = 0.689-0.874), while ChatGPT scored κ = 0.386 and Gemini κ = 0.224.
Read the study abstract
BackgroundArtificial intelligence (AI) may support electrocardiogram (ECG) interpretation and risk stratification in emergency departments, but its clinical utility remains uncertain.ObjectiveTo evaluate the performance of ChatGPT and Gemini in ECG interpretation and risk stratification for major adverse cardiac events (MACE).MethodsIn 89 prospective ED patients presenting with chest pain or palpitations, ECGs evaluated by AI models and emergency specialists were compared against cardiologist consensus. Cohen's…

The problem
In a prospective study of 89 emergency department patients with chest pain or palpitations, ChatGPT (κ = 0.386) and Gemini (κ = 0.224) agreed with cardiologists far less on ECG reads than emergency specialists did (κ = 0.689-0.874).
In brief
Both chatbots flagged most patients who had a major adverse cardiac event within 30 days. Of 27 patients (30.3%) with such events, ChatGPT had 96.3% sensitivity and 85.4% accuracy; Gemini had 92.6% sensitivity and 79.8% accuracy.
The authors say the findings are preliminary and do not support using either model to rule out cardiac risk. At most they suggest a supplementary decision-support role, pending multi-center validation.
Main points
- The study enrolled 89 prospective emergency department patients with chest pain or palpitations at a single center.
- Emergency specialists agreed with cardiologist consensus at κ = 0.689-0.874 on ECG interpretation.
- ChatGPT reached κ = 0.386 and Gemini κ = 0.224 against cardiologists.
- Gemini called ischemic features (ST elevation, ST depression, T-wave inversion) positive significantly more often than ChatGPT (p < 0.05).
- Thirty-day MACE occurred in 27 patients (30.3%); ChatGPT had 96.3% sensitivity and 85.4% accuracy, Gemini 92.6% and 79.8%.
- The difference in positive MACE risk calls between the models was not significant (McNemar's exact test, p = 0.55).
The full story
Researchers compared ECGs evaluated by ChatGPT, Gemini and emergency specialists against cardiologist consensus in 89 patients who presented to an emergency department with chest pain or palpitations. Cohen's kappa measured agreement, and diagnostic metrics were calculated for risk stratification of major adverse cardiac events.
Emergency specialists matched cardiologists well, while both chatbots lagged. McNemar testing found Gemini was significantly more likely than ChatGPT to call ischemic features such as ST elevation, ST depression and T-wave inversion.
For 30-day MACE, which occurred in 27 patients, ChatGPT was numerically ahead with 96.3% sensitivity and 85.4% accuracy versus 92.6% and 79.8% for Gemini. The proportion of positive risk calls did not differ significantly (p = 0.55), and the authors say equivalence cannot be inferred.
Sources
1 on file- Technology and Health Caredoi:10.1177/09287329261509123
Talk it over
ChatGPT reached κ = 0.386 and Gemini κ = 0.224 against cardiologists, yet both flagged most of the 27 patients with a major adverse cardiac event. How do you weigh those two results against each other?
ChatGPT and Gemini read ER heart tracings worse than doctors, 89-patient study findshttps://truvace.com/item/performance-of-ai-models-in-ecg-interpretation-and-risk-stratification-for-major
ChatGPT reached κ = 0.386 and Gemini κ = 0.224 against cardiologists, yet both flagged most of the 27 patients with a major adverse cardiac event. How do you weigh those two results against each other?
Answer in the threadThis was one emergency department with 89 patients with chest pain or palpitations. How might results differ in another hospital, and which patients would you want in a larger study?
Answer in the threadIf you were an emergency specialist working with these findings, how would you use ChatGPT or Gemini on a chest pain patient, if at all, given the authors' call for multi-center validation?
Answer in the thread
If you have more time
Find out what a multi-center validation of ECG reading by chatbots would need to include. What results would settle whether they could work as supplementary decision support?
Answer in the threadGemini called ischemic features positive significantly more often than ChatGPT. What might that mean for patients and staff in a busy emergency department?
Answer in the thread
ace
What readers say