TRV-2026-3088Version 1 · Published

Saved · see the current record

Why this version exists

Certified into the record

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-3088
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-21T00:02:56.403054Z
status: published
lens: g_space
sector: health
headline: Large Language Model-Assisted Cleaning of Report-Derived Labels in a Large-Scale Chest CT Dataset
dek: To evaluate whether large language model (LLM)-assisted label cleaning can identify label-report discordance in CT-RATE, a large-scale public chest CT dataset. After report-level deduplication, 24,446 unique radiology reports from 20,209 unique patients were identified. Twelve reports were excluded from the primary GPT-5.4 analysis because of persistent content-safety filtering or unparsable output, leaving 24,434 reports and 439,812 label instances across 18 abnormality categories. GPT-5.4-derived binary labels…
gain_title: In a study of 24,434 chest CT reports, a three-model AI majority vote scored an F1 of 95.6% against two blinded radiologists on 250 reports, versus 90.8% for the dataset's original labels.
problem_title: (none)
trace_subject: (none)
gain_reading: In a study of 24,434 chest CT reports, a three-model AI majority vote scored an F1 of 95.6% against two blinded radiologists on 250 reports, versus 90.8% for the dataset's original labels.
gain_evidence: the majority vote achieved an unweighted label-macro-averaged F1 score of 95.6% and κ of 0.940, compared with 90.8% and 0.879 for the original labels
problem_reading: (none)
problem_evidence: (none)
quick_read: Researchers used GPT-5.4 to re-derive yes/no labels for 18 abnormality categories from 24,434 radiology reports in CT-RATE, a large public chest CT dataset, and compared them with the original labels. Overall agreement was 96.4%, with a Cohen's κ of 0.884, and radiologists adjudicated the disagreements.

In a prespecified confirmatory set of 250 reports annotated independently by two blinded radiologists, a three-model majority vote reached an F1 of 95.6% versus 90.8% for the original labels, a 4.8-point gain (95% CI, 3.6-6.2). Each single model also beat the original labels.

Lymphadenopathy showed the lowest agreement, so that category had the most label-report discordance. The labels reflect what the reports say, not what the images show, so they do not confirm image-level truth.
limitation: Because all labels were derived from report text, they reflect report alignment rather than image-level truth.
tag: Evidence-backed gain
key_points: The analysis covered 24,434 reports from 20,209 patients and 439,812 label instances across 18 abnormality categories; 12 reports were excluded for content-safety filtering or unparsable output. | GPT-5.4 labels agreed with the original CT-RATE labels 96.4% of the time (κ = 0.884), with lymphadenopathy showing the lowest agreement and κ. | On the 250-report confirmatory set, the majority vote scored an F1 of 95.6% and κ of 0.940, versus 90.8% and 0.879 for the original labels. | Pre-consensus agreement between the two blinded radiologists was κ = 0.936, and 1.02% of label instances flipped across three repeated majority-vote runs. | The refined labels and code are to be made public upon acceptance.
rundown: The team deduplicated the dataset to 24,446 unique reports, then generated binary labels for each of 18 abnormality categories using GPT-5.4 with structured outputs. Radiologists adjudicated the instances where the AI labels and the original labels disagreed.

A 100-report random development set was manually annotated to compare the original labels, individual model labels and three-model majority-vote labels against radiologist reference labels. The methods were then tested on a stratified confirmatory set of 250 reports annotated independently by two blinded radiologists.

The majority vote improved on the original labels by 4.8 F1 points, and the run-to-run flip rate of 1.02% suggests the output was largely stable. The authors say the approach may offer a scalable way to improve the quality of public imaging datasets.
sources:
- peer_reviewed | Journal of Medical Systems | https://doi.org/10.1007/s10916-026-02487-x | 2026-11-20
prev: 0000000000000000000000000000000000000000000000000000000000000000
SHA-256 hash
b6840a5de35ac9bb833df52994257e4cde88abbfdcd2e74ed7aa8c126fc6fc6a
Previous version’s hash
0000000000000000000000000000000000000000000000000000000000000000
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3088 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.