Data leakage and governance in AI tuberculosis detection from chest radiographs

Source article: Building and Validating a Data-Governance Workflow for Artificial Intelligence-Based Tuberculosis Detection from Chest Radiographs, and Quantifying the Impact of Data Leakage

Studies applying artificial intelligence (AI) to tuberculosis (TB) detection from chest radiographs (CXR) focus on models and metrics, yet the quality, provenance and reproducibility of the data workflow decide the reliability of results; duplicate images and non-patient-level splitting inflate evaluation metrics. We propose a seven-step data-governance workflow: multi-source standardisation, provenance management, three-hash deduplication (MD5, pHash, dHash), patient-level splitting, invariant-check auditing, l…

By the Truvace deskOctober 19, 20262 min read1 sourcePeer-reviewedTRV-2026-1930· cite & verify

Trace impact readingContested

P 65

Both sides are scored from claims and sources, not community votes.

G 63

In brief

Researchers proposed a seven-step data-governance workflow for building tuberculosis chest radiograph datasets for AI. It covers standardising sources, tracking provenance, removing duplicates, splitting by patient, auditing, lung segmentation and freezing data versions. They applied it to 46,410 images from seven public sources and ran four leakage scenarios 20 times each.

The work matters because inflated evaluation scores can make diagnostic AI look more dependable than it is. The findings show leakage size varies with data structure, so it has to be measured rather than assumed. How far the results carry over to other datasets or to clinical use is not established by the abstract.

Main points

The workflow removed duplicates at file and content level using three hashes: MD5, pHash and dHash.
It standardised 16,554 TB Portals DICOM images and generated 34,554 lung masks without failure.
On the three-class multi-source task, AUC stayed near 0.997 in every scenario, consistent with source-class confounding.
On the drug-resistance probe, the inflation fell to 1.2 points after a deduplication gap found during revision was repaired.
In a holdout design, an internal gap of 1.1 points shrank to 0.4 points on a patient-disjoint holdout.

The gain

A seven-step data-governance workflow cleaned and sealed a 46,410-image chest radiograph corpus and made leakage effects verifiable and measurable for AI tuberculosis research.

The problem

Random image-level splitting inflated internal AUC on a single-source drug-resistance task by 1.7 percentage points, so leakage can overstate how reliable AI tuberculosis models look.

The rundown

The study compared four scenarios, with deduplication on or off and patient-level or random splits, repeating each 20 times with paired splits. Each scenario was compared with the clean scenario using per-repeat differences.

On the multi-source three-class corpus, paired leakage effects were statistically detectable but below 0.1 percentage points. The authors link the saturated AUC to source-class confounding, meaning the model may be distinguishing data sources rather than disease.

On the single-source drug-resistance task within TB Portals, the inflation was 1.7 points (95% CI 0.9 to 2.4; p = 0.001). The authors conclude that leakage magnitude depends on data structure and must be measured.

What this doesn’t fix

The saturated AUC near 0.997 across scenarios is consistent with source-class confounding, so the multi-source task could not show leakage effects well, and a deduplication gap was found only during revision.