Truace
Subscribe

An AI system acting as a panel of doctors diagnosed cases more accurately than generalist physicians

Study title: Sequential diagnosis with language models

Researchers built an interactive benchmark that mimics how clinicians work up a case step by step, starting from a brief presentation and choosing which information to gather. They then developed an AI system that acts like a panel of physicians, updating its differential diagnosis and choosing tests with cost in mind.

Read the study abstract

We explored the capabilities of frontier language models in sequential diagnosis, a diagnostic paradigm that captures the iterative nature of clinical reasoning. Given a brief clinical presentation, clinicians selectively acquired information to reach a diagnosis, enabling assessment of diagnostic accuracy and workup cost. We developed an interactive benchmark and an artificial intelligence (AI) system that simulates a panel of physicians, continually revising the differential diagnosis and selecting cost-effect…

By the Truvace desk1 min read1 sourcePeer-reviewedTRV-2026-2234· cite & verify
Peer-reviewed study: Science

In brief

The results suggest language models could support diagnosis that is both more accurate and cheaper, since the benchmark tracks workup cost alongside correctness. However, the work was done in a controlled benchmark rather than in live care, so it remains unclear how the system would perform with real patients and clinical teams.

Main points

  1. The study frames sequential diagnosis as an iterative process in which information is acquired selectively before a diagnosis is reached.
  2. The AI system simulates a panel of physicians and continually revises the differential diagnosis.
  3. It was scored on 83.2% accuracy for NEJM clinicopathological conference cases and 80.3% for MIMIC-IV-ED emergency department admissions.
  4. Generalist physicians scored 36.1% and 46.8% on the same two sets, respectively.

The gain

An AI system that simulates a panel of physicians and selects tests reached higher diagnostic accuracy than generalist physicians on two benchmark case sets.

The rundown

The benchmark was designed to assess both diagnostic accuracy and workup cost, because clinicians in the setup selectively request information rather than receiving a full case at once.

The AI system was built to select cost-effective tests while revising its differential diagnosis as new information came in, mirroring how clinical reasoning unfolds over time.

Two case sources were used: New England Journal of Medicine clinicopathological conference cases and emergency department admissions from the MIMIC-IV-ED dataset. On both, the system's accuracy exceeded that of generalist physicians.

Sources

  1. Peer-reviewedScience2026-10-22

The debate