Truace
Subscribe

At one hospital, AI fracture flags were right 99.1% when confident but 59.1% when not

Study title: Diagnostic accuracy & clinical importance of AI confidence for extremity fracture detection: 2,508-patient retrospective cohort

Purpose To estimate diagnostic performance of a deep learning algorithm for extremity fracture detection on radiographs in patients aged ≥ 2 years using a refined reference standard. Secondary, to compare positive predictive value (PPV) by the algorithm's built-in confidence level (high vs. low) and diagnostic performance between adults (≥ 18 years) and children.

Read the study abstract

Purpose To estimate diagnostic performance of a deep learning algorithm for extremity fracture detection on radiographs in patients aged ≥ 2 years using a refined reference standard. Secondary, to compare positive predictive value (PPV) by the algorithm's built-in confidence level (high vs. low) and diagnostic performance between adults (≥ 18 years) and children. Methods This retrospective single-center study consecutively included patients with radiography of a suspected extremity fracture between January and D…

Truvace record1 min read1 sourceEvidence: Moderate · 63/100 — Observational study · n = 2,508 · 1 independent sourcePeer-reviewedTRV-2026-1176 · cite or check it
Diagnostic accuracy & clinical importance of AI confidence for extremity fracture detection: 2,508-patient retrospective cohort
Hospital Universitari Doctor Peset, València 04 by 19Tarrestnom65. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0

In brief

The index test was the algorithm output (bounding boxes with confidence). Sensitivity, specificity, PPV and negative predictive value (NPV) were calculated with 95% confidence intervals; differences were tested using the McNemar and a score test and patient-clustered logistic regression.

Main points

  1. Purpose To estimate diagnostic performance of a deep learning algorithm for extremity fracture detection on radiographs in patients aged ≥ 2 years using a refined reference standard.
  2. Secondary, to compare positive predictive value (PPV) by the algorithm's built-in confidence level (high vs. low) and diagnostic performance between adults (≥ 18 years) and children.
  3. Methods This retrospective single-center study consecutively included patients with radiography of a suspected extremity fracture between January and December 2024.

The gain

In a single-center review of 2,508 patients, the AI's high-confidence fracture detections on X-rays were correct 99.1% of the time, and the authors judged overall performance good.

The full story

Purpose To estimate diagnostic performance of a deep learning algorithm for extremity fracture detection on radiographs in patients aged ≥ 2 years using a refined reference standard. Secondary, to compare positive predictive value (PPV) by the algorithm's built-in confidence level (high vs. low) and diagnostic performance between adults (≥ 18 years) and children.

Methods This retrospective single-center study consecutively included patients with radiography of a suspected extremity fracture between January and December 2024. The index test was the algorithm output (bounding boxes with confidence).

Sources

  1. Peer-reviewedEmergency Radiology2026-09-22

The debate