Truace
Subscribe

Metastasis Extraction From Non-Small Cell Lung Cancer Clinical Notes Using Large Language Model-Based Classification: Retrospective Comparative Study

Background: Identification of metastasis status in non-small cell lung cancer (NSCLC) is a critical part of understanding disease prognosis, treatment courses, trial eligibility, and population-level cancer surveillance. However, metastasis status is inconsistently recorded in structured cancer registry fields, since manual abstraction of clinical notes is often an error-prone process. This challenge highlights an opportunity for leveraging large language models (LLMs) to conduct high-scale metastasis extraction…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1636· cite & verify
Metastasis Extraction From Non-Small Cell Lung Cancer Clinical Notes Using Large Language Model-Based Classification: Retrospective Comparative Study
Metastatic follicular thyroid carcinoma - CT scan - Case 264 (8559838032) by Yale Rosen from USA. CC BY-SA 2.0 · https://creativecommons.org/licenses/by-sa/2.0

In brief

Researchers at the Winship Cancer Institute compared three language models on two tasks: detecting metastasis at any site and detecting brain or central nervous system metastasis in notes from non-small cell lung cancer patients. They tested a fine-tuned GatorTron encoder alongside zero-shot MedGemma-27B-text and Llama 3.1-70B, using one cohort with registry labels and another with manual annotations.

Metastasis status matters for prognosis, treatment and trial eligibility, yet registries often record it inconsistently, so automated extraction could fill gaps in cancer surveillance. The evidence is retrospective and comes from one institution, and the manually annotated cohort is very small. The recovery analysis was exploratory, so real-world reliability remains to be confirmed.

Main points

  1. MedGemma performed best on overall metastasis, with F1 of 0.80 in cohort 1 and 1.0 at patient level in cohort 2.
  2. Llama 3.1-70B performed best on brain/CNS metastasis, with F1 of 0.79 in cohort 1 and 0.93 at patient level in cohort 2.
  3. Site-specific classification by MedGemma reached F1 of 0.82 for liver and 0.79 for bone metastasis.
  4. Error analysis tied several errors to incomplete registry labels, ambiguous clinical language or missing documentation.

The gain

LLMs classified metastasis from clinical notes with fairly high accuracy and recovered plausible labels for patients with missing registry fields, which could complement cancer registries.

The rundown

Cohort 1 covered 579 patients and 24,887 notes across 69 note types from 2023 to 2025, using registry-linked metastasis fields as the reference standard. Cohort 2 covered 22 patients and 644 radiology notes from 2010 to 2021, drawn from two completed randomized trials, with dual-annotated labels.

A separate group of 675 patients with missing registry labels was used for an exploratory missingness-recovery analysis, validated against annotations of a random subsample. The best-performing model was also applied to site-specific metastasis classification.

The authors reported that zero-shot generative models handled nuanced documentation and context-dependent brain/CNS metastasis better. The fine-tuned encoder had strong performance, with F1 of 0.72 for overall metastasis in cohort 1.

Sources

  1. Peer-reviewedJMIR Cancer2026-10-16

The debate