ChatGPT-4.1 treatment decisions for complex coronary artery disease versus heart teams

Source article: Performance of large language models in complex coronary artery disease decision-making and potential improvement strategy in a real-world cohort

Objective: Previous studies have demonstrated that large language models (LLMs) can achieve performance comparable to multidisciplinary heart teams (HTs) in coronary revascularisation decision-making but limited by population bias. We aimed to assess LLMs’ performance in complex coronary artery disease (CAD) decision-making using a real-world cohort and to explore the potential improvement strategy. Methods: Patients with stable CAD and angiographically confirmed three-vessel or left main disease randomly select…

By the Truvace deskOctober 15, 20262 min read1 sourcePeer-reviewedTRV-2026-1470· cite & verify

Performance of large language models in complex coronary artery disease decision-making and potential improvement strategy in a real-world cohort — Professor Richard Harvey FRS by Duncan.Hull. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0

Trace impact readingContested

P 64

Both sides are scored from claims and sources, not community votes.

G 63

In brief

Researchers tested ChatGPT-4.1 on 564 patients with stable three-vessel or left main coronary artery disease drawn at random from a consecutive angiography registry. The model chose among five treatment categories using patient information and a prompt of the kind real-world patients commonly use, and its answers were compared with multidisciplinary heart-team decisions.

The results suggest that patients asking an AI chatbot simple questions about complex heart disease cannot rely on the answers, since agreement with specialists was under half. Better prompting narrowed the gap, but the authors still stress remaining safety risk, and the work does not show the model is ready for clinical use.

Main points

The cohort had 564 patients with stable coronary artery disease and angiographically confirmed three-vessel or left main disease.
Initial-prompt agreement between the LLM and heart teams was 47.0%.
The model's sensitivity for recommending further testing was 0.0%.
Agreement varied by SYNTAX score: 94.1% at 33 or above, 61.7% at 23 to 32, and 20.9% at 0 to 22.
Prompting strategies raised overall agreement to 79.3%.

The gain

Three predefined prompting strategies raised LLM agreement with heart-team revascularisation decisions from 47.0% to 79.3% in the same complex coronary artery disease cohort.

The problem

With a typical patient-style prompt, ChatGPT-4.1 agreed with heart-team decisions in only 47.0% of complex coronary artery disease cases and never identified cases needing further testing.

The rundown

The study was cross-sectional and built on a real-world registry, addressing earlier work that found LLMs comparable to heart teams but was limited by population bias. The primary outcome was the agreement rate between heart-team and LLM recommendations.

Performance depended heavily on case complexity as measured by the Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery score. The model did well in the highest-score group but poorly in the intermediate and lowest groups.

Three predefined prompting strategies were tested for their ability to raise agreement. The authors conclude that prompt engineering and subgroup analysis can partly improve performance, yet simple patient-style queries cannot independently generate reliable decisions.