Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1470
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-15T18:13:49.342943Z
status: published
lens: trace
sector: health
headline: Performance of large language models in complex coronary artery disease decision-making and potential improvement strategy in a real-world cohort
dek: Objective: Previous studies have demonstrated that large language models (LLMs) can achieve performance comparable to multidisciplinary heart teams (HTs) in coronary revascularisation decision-making but limited by population bias. We aimed to assess LLMs’ performance in complex coronary artery disease (CAD) decision-making using a real-world cohort and to explore the potential improvement strategy.
Methods: Patients with stable CAD and angiographically confirmed three-vessel or left main disease randomly select…
gain_title: Three predefined prompting strategies raised LLM agreement with heart-team revascularisation decisions from 47.0% to 79.3% in the same complex coronary artery disease cohort.
problem_title: With a typical patient-style prompt, ChatGPT-4.1 agreed with heart-team decisions in only 47.0% of complex coronary artery disease cases and never identified cases needing further testing.
trace_subject: ChatGPT-4.1 treatment decisions for complex coronary artery disease versus heart teams
gain_reading: Three predefined prompting strategies raised LLM agreement with heart-team revascularisation decisions from 47.0% to 79.3% in the same complex coronary artery disease cohort.
gain_evidence: After using prompting strategies, the agreement rate increased significantly (79.3%, p<0.001).
problem_reading: With a typical patient-style prompt, ChatGPT-4.1 agreed with heart-team decisions in only 47.0% of complex coronary artery disease cases and never identified cases needing further testing.
problem_evidence: LLM performed poorly in further testing (sensitivity=0.0%) decision.
quick_read: Researchers tested ChatGPT-4.1 on 564 patients with stable three-vessel or left main coronary artery disease drawn at random from a consecutive angiography registry. The model chose among five treatment categories using patient information and a prompt of the kind real-world patients commonly use, and its answers were compared with multidisciplinary heart-team decisions.

The results suggest that patients asking an AI chatbot simple questions about complex heart disease cannot rely on the answers, since agreement with specialists was under half. Better prompting narrowed the gap, but the authors still stress remaining safety risk, and the work does not show the model is ready for clinical use.
limitation: Even after prompting improvements, agreement with heart teams was not complete, and the authors emphasise that potential safety risk remains.
tag: Dual reading
key_points: The cohort had 564 patients with stable coronary artery disease and angiographically confirmed three-vessel or left main disease. | Initial-prompt agreement between the LLM and heart teams was 47.0%. | The model's sensitivity for recommending further testing was 0.0%. | Agreement varied by SYNTAX score: 94.1% at 33 or above, 61.7% at 23 to 32, and 20.9% at 0 to 22. | Prompting strategies raised overall agreement to 79.3%.
rundown: The study was cross-sectional and built on a real-world registry, addressing earlier work that found LLMs comparable to heart teams but was limited by population bias. The primary outcome was the agreement rate between heart-team and LLM recommendations.

Performance depended heavily on case complexity as measured by the Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery score. The model did well in the highest-score group but poorly in the intermediate and lowest groups.

Three predefined prompting strategies were tested for their ability to raise agreement. The authors conclude that prompt engineering and subgroup analysis can partly improve performance, yet simple patient-style queries cannot independently generate reliable decisions.
sources:
- peer_reviewed | BMJ Health &amp; Care Informatics | https://doi.org/10.1136/bmjhci-2026-102072 | 2026-10-13
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

ce8c377833b19356ee0b3ebf85626e1c2d1b5b1ed1774cb60bc2ce0ba470a517

previous

0000000000000000000000000000000000000000000000000000000000000000