acemulticlass lesion classification in capsule endoscopy using vision transformer versus CNN architectures
Source article: Transformer-Based Deep Learning Framework for Automated Lesion Detection in Capsule Endoscopy: A Comparative Study With CNN Architectures
Abstract: Goals To compare a vision transformer with 2 convolutional neural network architectures for multiclass lesion classification in capsule endoscopy images. Background Manual review of capsule endoscopy is time-consuming and subject to interobserver variability. Deep learning can automate lesion recognition; however, most prior capsule endoscopy work evaluates a small number of classes, and systematic comparisons between transformer and convolutional architectures across many lesion categories are limited. Study Tw…
Contested: both sides are scored from claims and sources, not community votes.
"injection" by Yuya Tamai is licensed under CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/.
A comparative study merged SEE-AI and Kvasir-Capsule into a 21-class capsule endoscopy image dataset and fine-tuned a Vision Transformer, DenseNet121, and ResNet50. On an independent test set of 8,696 frames, the transformer achieved 92.2% accuracy and 0.99 AUC, substantially higher than the two CNN baselines under the reported experimental conditions.
The result matters because manual capsule endoscopy review is time-consuming and variable, and automated multiclass recognition could reduce workload, but the study's image-level split means performance may not transfer to new patients or procedures. The authors frame the findings as supporting further evaluation of transformers rather than as evidence ready for clinical deployment.
- Study merged SEE-AI and Kvasir-Capsule into ~58,000 frames across 21 lesion classes, resized to 224x224 and split into training (40,587), validation (8,696), and test (8,696).
- Vision Transformer, DenseNet121, and ResNet50 were fine-tuned with categorical cross-entropy and Adam with early stopping and evaluated on accuracy, macroaveraged precision/recall/F1, and AUC.
- DenseNet121 achieved 74.0% accuracy (F1-score 0.78; AUC 0.85) and ResNet50 achieved 38.0% accuracy (F1-score 0.40; AUC 0.55) on the same test set.
A pretrained Vision Transformer fine-tuned on a merged 21-class capsule endoscopy dataset achieved 92.2% accuracy and 0.99 AUC on an independent test set, outperforming DenseNet121 and ResNet50.
Because the dataset was split at the image level, correlated frames from the same examination may inflate performance, so results cannot be interpreted as patient-level generalization and require grouped reanalysis and external validation before clinical use.
The rundown
Researchers combined two public capsule endoscopy datasets into a 21-class set of about 58,000 frames and compared three pretrained models fine-tuned under identical conditions. The Vision Transformer reached 92.2% accuracy with macroaveraged 0.92 precision/recall/F1 and 0.99 AUC, while DenseNet121 and ResNet50 reached 74.0% and 38.0% accuracy respectively.
The authors explicitly note the evaluation was frame-level, not patient-level, and that correlation within examinations means the numbers are benchmark results on this public dataset rather than estimates of real-world generalization. They call for grouped reanalysis, external validation, and workflow studies before clinical implementation.
Performance estimates are based on an image-level split rather than patient or procedure-level split, so frames from the same examination may be correlated and results should not be interpreted as patient-level generalization or definitive architectural superiority without grouped reanalysis and external validation.
Sources
- Peer-reviewedJournal of Clinical Gastroenterology2026-08-14
How should this claim be treated?
The debate