70/100

Moderate evidence

GainPublished recordHealthGood Space

Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset

Truvace record
TRV-2026-3088raw data (JSON)
Version
v1Published
First published
Evidence type
Controlled or validated comparison
Sources
1 sourcePeer-reviewed
sha256
b6840a5de35a…check it

To evaluate whether large language model (LLM)-assisted label cleaning can identify label-report discordance in CT-RATE, a large-scale public chest CT dataset. After report-level deduplication, 24,446 unique radiology reports from 20,209 unique patients were identified. Twelve reports were excluded from the primary GPT-5.4 analysis because of persistent content-safety filtering or unparsable output, leaving 24,434 reports and 439,812 label instances across 18 abnormality categories. GPT-5.4-derived binary labels…

The gain

In a study of 24,434 chest CT reports, a three-model AI majority vote scored an F1 of 95.6% against two blinded radiologists on 250 reports, versus 90.8% for the dataset's original labels.

What this doesn’t fix

Because all labels were derived from report text, they reflect report alignment rather than image-level truth.

Sources

1 on file
  1. Journal of Medical Systemsdoi:10.1007/s10916-026-02487-x
    Peer-reviewed2026-11-20

Cite this record

v1

Truvace Impact Record TRV-2026-3088, v1: “Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset.” Truvace, 2026-11-21. /record/TRV-2026-3088 (accessed on the day you copy it). sha256 b6840a5de35ac9bb…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv1b6840a5de35a…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3088 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.