Truace
Subscribe

LLM diagnosis and treatment planning for teeth needing initial root canal treatment

Source article: Comparing Diagnostic Ability of Two Off-the-Shelf Large Language Models with That of Endodontists for Teeth Requiring Initial Root Canal Treatment: A Multicenter Retrospective Investigation

Introduction: Despite growing interest in large language models (LLMs) for endodontic diagnosis, prior studies have used simulated cases, examination formats, or small selected series. This multicenter retrospective study compared the diagnostic accuracy of ChatGPT-5.2 and Gemini 3.0 with that of calibrated endodontists across 645 teeth requiring initial root canal treatment. Methods: Cases were evaluated across four endpoints: pulpal diagnosis, periapical diagnosis, treatment planning, and case difficulty. Stan…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1352· cite & verify
Comparing Diagnostic Ability of Two Off-the-Shelf Large Language Models with That of Endodontists for Teeth Requiring Initial Root Canal Treatment: A Multicenter Retrospective Investigation
"09-8018-23" by NavyMedicine is marked with Public Domain Mark 1.0. To view the terms, visit https://creativecommons.org/publicdomain/mark/1.0/.
Trace impact readingNegative state
P 72The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 67The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Researchers at multiple centers retrospectively tested ChatGPT-5.2 and Gemini 3.0 against calibrated endodontists on 645 teeth needing a first root canal. Each case, supplied as a standardized clinical summary with radiographs, was assessed once for pulpal diagnosis, periapical diagnosis, treatment plan and case difficulty.

The results suggest that these general-purpose tools are useful for narrow tasks but unreliable as a whole-case assessor, so any clinical use needs expert oversight. Findings on rarer categories are only suggestive because the subgroups were small, and the work is retrospective rather than a test in live care.

Main points

  1. Single-endpoint accuracy ranged from 74% to 88% across the two models.
  2. Composite accuracy over the three clinical endpoints was 61.1% for ChatGPT-5.2 and 59.4% for Gemini 3.0.
  3. Four-endpoint accuracy fell to 47.3% and 46.2% respectively.
  4. Only treatment planning differed between models after correction, favoring Gemini 3.0 (OR 1.457, adjusted p = .003).
  5. Macro F1 for treatment planning was 0.385 for ChatGPT-5.2 and 0.550 for Gemini 3.0.

The gain

On individual endpoints such as pulpal or periapical diagnosis, both LLMs reached 74% to 88% accuracy on real root canal cases, which supports a role as adjunctive tools.

The problem

When the endpoints were combined, both models made at least one clinically relevant error in about half of cases, and performance dropped in low-prevalence categories.

The rundown

The study addressed a gap in earlier work, which relied on simulated cases, examination-style questions or small selected series. Here the sample was 645 real teeth from several centers, and the models were compared with calibrated endodontists under a fixed single-pass protocol.

Statistical comparison used exact-match accuracy, Cohen's kappa, Gwet's AC1, macro-averaged F1, McNemar tests and generalized estimating equations. Macro F1 peaked for periapical diagnosis at 0.724 for ChatGPT-5.2 and 0.719 for Gemini 3.0.

Both models did worse in minority categories of treatment planning, which the authors call suggestive given small subgroup sizes. The authors conclude that LLMs can serve as adjunctive tools in endodontic diagnosis and planning, but not without expert supervision.

What this doesn’t fix

Findings on minority categories are described as suggestive because of small subgroup sizes, and the study is retrospective.

Sources

  1. Peer-reviewedJournal of Endodontics2026-10-13

The debate