Truace
Subscribe

Differential Diagnosis Performance of ChatGPT in Oral and Maxillofacial Pathology

Objective: To evaluate ChatGPT (GPT-5.1) for ranked differential diagnoses in oral and maxillofacial pathology, assess image effects on diagnostic performance and examine output stability across repeated runs. Methods: This retrospective diagnostic accuracy study included 160 biopsy-confirmed cases. Cases were evaluated using a standardised prompt under two conditions: text-only and text plus clinical photographs and/or radiographs. Each case underwent three independent runs. Performance was measured using Top-1…

By the Truvace desk1 min read1 sourcePeer-reviewedTRV-2026-2004· cite & verify
Differential Diagnosis Performance of ChatGPT in Oral and Maxillofacial Pathology
"22-0008-010 Camp Lejeune" by NavyMedicine is marked with Public Domain Mark 1.0. To view the terms, visit https://creativecommons.org/publicdomain/mark/1.0/.

In brief

Researchers tested ChatGPT (GPT-5.1) on 160 biopsy-confirmed oral and maxillofacial pathology cases, asking it for ranked differential diagnoses. Each case was run three times, once with text only and once with text plus clinical photographs or radiographs.

The results show a gap between suggesting the right answer somewhere in a list and committing to it as the top choice. That limits use in clinical settings, though the authors see room for hypothesis generation and education. How well images could help with more standardised inputs remains unclear.

Main points

  1. Top-1 accuracy was 44.0% with text only and 43.3% with text plus images.
  2. Top-10 accuracy reached 80.7% for text-only and 82.9% for text plus image inputs.
  3. No significant differences between input conditions were seen at any Top-K threshold.
  4. Top-1 results were stable across three runs, but broader Top-K thresholds varied in the text-only condition.

The problem

ChatGPT listed the biopsy-confirmed diagnosis first in only about 43 to 44 percent of oral and maxillofacial pathology cases, and adding clinical images did not significantly improve accuracy, so it is not reliable enough for clinical decision support.

The rundown

The study was a retrospective diagnostic accuracy analysis using 160 cases confirmed by biopsy. A standardised prompt was applied under two input conditions, and accuracy was scored at Top-1, Top-3, Top-5 and Top-10 thresholds.

Accuracy rose as the list widened, with the reference diagnosis appearing in about four of five cases at the Top-10 level. Adding photographs or radiographs did not produce a statistically significant gain at any threshold.

The authors conclude the findings support further investigation for hypothesis generation and educational use, not clinical decision support or replacement of expert judgement and histopathological confirmation.

Sources

  1. Peer-reviewedOral Diseases2026-10-19

The debate