68/100

Moderate evidence

Cite
More
EmailLinkedInX

AI matched experts on 94% of esophageal motility tests in a 984-patient study

Study title: A Clinically Weighted Transformer-Based Deep Learning Model for Automated FLIP Panometry v2.0 Interpretation

Evidence type
Controlled or validated comparison
Sample
n = 984
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: A Clinically Weighted Transformer-Based Deep Learning Model for Automated FLIP Panometry v2.0 Interpretation

Researchers built a transformer-based deep learning model to interpret FLIP Panometry v2.0, a test that assesses esophageal motility through contractile responses and esophagogastric junction opening. On a held-out test set it reproduced expert six-category classification in 83.5% of studies (95% CI 76.4-88.7%).

Read the study abstract

Functional lumen imaging probe (FLIP) Panometry evaluates esophageal motility by measuring contractile responses and esophagogastric junction opening, but interpretation requires expertise and varies between readers. We aimed to develop a transformer-based deep learning model for automated FLIP Panometry v2.0 interpretation, incorporating a clinically weighted cost matrix that penalizes clinically consequential errors. 984 adults who underwent FLIP Panometry and high-resolution manometry (HRM) were included. Exp…

A Clinically Weighted Transformer-Based Deep Learning Model for Automated FLIP Panometry v2.0 Interpretation
"374 MDG enhances operational readiness at Yokota" by Airman 1st Class David S. Calcote, U.S. Air Force, public domain.

The gain

In a single-center study of 984 adults, a transformer model matched expert readings of FLIP Panometry esophageal tests in 83.5% of six-category test cases and 94% of normal versus not-normal calls, with 97% sensitivity.

In brief

When results were grouped, binary accuracy for normal versus not normal was 94%, with 97% sensitivity and 89% specificity. In the three-class scheme (normal, abnormal, obstructive), accuracy was 89%, and obstructive patterns were detected with 93% sensitivity and 96% specificity.

The model was trained with a cost matrix that penalizes clinically consequential errors, and no normal studies were classified as obstructive or the reverse. The authors say the findings support a potential role for AI-assisted interpretation but that external validation is required before clinical use.

Main points

  1. The study included 984 adults who underwent both FLIP Panometry and high-resolution manometry.
  2. Expert FLIP Panometry v2.0 interpretation, assigned blinded to manometry results, served as the reference standard for training and evaluation.
  3. Six-category agreement with experts was 83.5% (95% CI 76.4-88.7%) on the held-out test set.
  4. Binary accuracy was 94%, with 97% sensitivity and 89% specificity.
  5. Three-class accuracy was 89%; obstructive pattern sensitivity was 93% and specificity 96%.
  6. No normal studies were classified as obstructive, or the reverse.
  7. Cost-sensitive training was associated with 15% lower aggregate misclassification cost than an unweighted baseline, with a wide confidence interval (-16% to 39%).

The full story

FLIP Panometry measures esophageal contractile responses and esophagogastric junction opening, but interpretation requires expertise and varies between readers. The team set out to automate reading of the newer v2.0 scheme using a transformer-based model.

The cohort comprised 984 adults who had both FLIP Panometry and high-resolution manometry (Chicago Classification v4.0). Manometry characterized the cohort and corroborated expert FLIP labels, and inconclusive or discordant manometry studies were excluded.

The model was evaluated on a held-out test set under three schemes: six categories, binary, and three classes. A weighted cost matrix penalized clinically consequential errors, and the weighted model was compared with an unweighted baseline.

Weighted training was associated with 15% lower aggregate misclassification cost, but the confidence interval (-16% to 39%) spans zero effect. Accuracy was numerically higher than prior convolutional models trained on different data.

Sources

1 on file
  1. Clinical Gastroenterology and Hepatologydoi:10.1016/j.cgh.2026.10.036
    Peer-reviewed2026-11-19

Talk it over

Three questions to start on, two more if you have time
The model matched expert readings in 83.5% of six-category cases and 94% of normal versus not-normal calls. Which of those two results would matter more to you if you were a patient waiting for a diagnosis?

Start with a question

AI matched experts on 94% of esophageal motility tests in a 984-patient studyhttps://truvace.com/item/a-clinically-weighted-transformer-based-deep-learning-model-for-automated-flip-p

  1. The model matched expert readings in 83.5% of six-category cases and 94% of normal versus not-normal calls. Which of those two results would matter more to you if you were a patient waiting for a diagnosis?

    Answer in the thread
  2. All 984 adults came from a single center, and inconclusive or discordant manometry studies were excluded. How could that selection change how the model performs on patients in another clinic?

    Answer in the thread
  3. If you were one of the experts who read the FLIP Panometry studies, what result from an outside hospital would make you trust the model enough to use it?

    Answer in the thread
If you have more time
  • Look up how much readers of FLIP Panometry disagree with each other. How would that change what counts as matching the experts?

    Answer in the thread
  • The cost matrix cut aggregate misclassification cost by 15%, with a confidence interval of -16% to 39%. What further test would show whether that weighting helps?

    Answer in the thread

What readers say