Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Review of 136 dental X-ray AI studies finds heatmap explanations rarely checked for accuracy

Study title: Seeing the Gaps: Heatmaps, Metrics, and the Hidden Failures of Explainable AI in Dentomaxillofacial Radiology - A Systematic Review

Evidence type
Systematic review or meta-analysis
Sample
n = 136
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2447cite or check it

Study title: Seeing the Gaps: Heatmaps, Metrics, and the Hidden Failures of Explainable AI in Dentomaxillofacial Radiology - A Systematic Review

A systematic review of 136 studies on deep learning for dental radiograph classification examined the heatmaps researchers use to show what the AI is looking at. Grad-CAM was the most common technique, and the main reason for using heatmaps was to understand how the model makes decisions. Formal faithfulness testing was rarely performed, meaning few studies checked whether the heatmaps truly reflect the model's reasoning.

Read the study abstract

This systematic review aims to evaluate the techniques and reported patterns of heatmap-based visual explanations in deep learning (DL) systems for dental radiographic classification tasks. A literature search following PRISMA guidelines was conducted in PubMed, Scopus, IEEE Xplore Library, Oria.no and Google Scholar to retrieve studies using DL for dental classification tasks. Only articles with a visual explanation (heatmap) were selected. Risk of bias assessment was done with QUADAS-2, PROBAST-AI and CEBM. Th…

The problem

A systematic review of 136 dental radiograph AI studies found formal faithfulness testing of heatmap explanations was rarely done, so the heatmaps cannot be assumed to show how the model actually reasons.

In brief

Some heatmaps matched established radiographic criteria, while others showed label-dependent patterns that sometimes emphasised irrelevant areas. Cropping images to the region of interest changed the visual explanations. Domain experts found the heatmaps plausible when they matched clinical knowledge, but the authors stress that plausibility is not the same as technical faithfulness.

The authors conclude the explanations remain insufficiently validated and call for evaluation metrics designed for dentomaxillofacial radiology. They also note an implausible heatmap does not necessarily mean the model has failed.

Main points

  1. The review covered 136 studies using deep learning for dental radiographic classification that included heatmap explanations.
  2. Grad-CAM was the most commonly used heatmap technique.
  3. The most common reason for using visual explanations was to understand the model's decision-making process.
  4. Some heatmaps reflected established radiographic criteria; others showed label-dependent patterns that sometimes emphasised irrelevant areas.
  5. Cropping to the region of interest influenced the visual explanations.
  6. Formal faithfulness testing was rarely performed in the included studies.
  7. The authors call for domain- and task-specific evaluation metrics for explainable AI in dentomaxillofacial radiology.

The full story

Searches followed PRISMA guidelines across PubMed, Scopus, IEEE Xplore Library, Oria.no and Google Scholar. Only studies using deep learning for dental classification tasks that included a heatmap visual explanation were selected. Risk of bias was assessed with QUADAS-2, PROBAST-AI and CEBM.

The authors analysed the heatmap technique, the reasons for including it, the explanations provided and the patterns seen. Heatmaps looked clinically plausible to domain experts when aligned with domain knowledge, but the review says that does not establish they faithfully represent model reasoning.

The authors say an implausible explanation does not necessarily indicate model failure, and recommend domain- and task-specific metrics for judging explainable AI in this field. No funding was received and the authors declared no competing interests.

Talk it overFour questions for a class, a team or the dinner table

  1. Who carries this cost, and what would have to change to prevent it?

    The problem A systematic review of 136 dental radiograph AI studies found formal faithfulness testing of heatmap explanations was rarely done, so the heatmaps cannot be assumed to show how the model actually reasons.

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 136, from Dentomaxillofacial Radiology.

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The review evaluated studies that were not primarily designed to assess visual explanations, which limits how directly their heatmap findings can be judged.

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

The questions are the same on every record. The details under them are taken from this record as written.

Sources

1 on file
  1. Dentomaxillofacial Radiologydoi:10.1093/dmfr/twag074
    Peer-reviewed2026-11-07

The debate

More from Health