AI system helps staff find cancer immunotherapy side effects in patient notes faster

Study title: An agentic AI system enhances clinical detection of immune-related adverse events in a multi-phase validation study

Researchers built an agentic large language model system that reads clinical notes and pulls out whether an immune-related adverse event is present, when it happened, how severe it was, what caused it and how certain the note is. It covers six event types seen in patients on immune checkpoint inhibitors, where manual chart review is slow and inconsistent.

Read the study abstract

Immune-related adverse events (irAEs) affect up to 40% of patients receiving immune checkpoint inhibitors, yet identification requires laborious, inconsistent manual chart review. Here we show that agentic artificial intelligence coupled with human verification could enhance efficiency, performance and consistency for irAE assessment. We developed and evaluated an agentic large language model system to extract the presence, temporality, severity grade, attribution and certainty of six irAE types from clinical no…

By the Truvace deskOctober 30, 20261 min read1 sourceEvidence: Moderate · 70/100 — Randomized controlled trial · 1 independent sourcePeer-reviewedTRV-2026-2318· cite & verify

An agentic AI system enhances clinical detection of immune-related adverse events in a multi-phase validation study — Cancer Research UK, Harrison Square - geograph.org.uk - 1801736 by Robbie. CC BY-SA 2.0 · https://creativecommons.org/licenses/by-sa/2.0

In brief

The work matters because these side effects are common and finding them by hand is laborious. The system performed best at spotting current events and less well at grading severity, and its accuracy dropped somewhat in live silent deployment. The results support AI-assisted review with a human checking the output rather than fully automated review.

Main points

Retrospectively, the system reached a macro-averaged F1 of 0.92 for detecting current events but only 0.66 for multi-class severity grading.
Self-consistency improved F1 by 0.14, and the best configuration cost approximately US$0.02 per note.
In 3 months of prospective silent deployment, binary F1 was 0.72-0.79.
In a randomized crossover study of trial staff, AI assistance cut annotation time by 2.71 minutes per item.
Interannotator agreement rose from a Krippendorff's α of 0.22-0.51 to 0.82-0.85 with assistance.

The gain

With AI assistance, clinical trial staff spent 2.71 fewer minutes per annotation, matched reference answers more often, and agreed with each other far more consistently when reviewing charts for immunotherapy side effects.

The rundown

Immune-related adverse events affect up to 40% of patients receiving immune checkpoint inhibitors. Identifying them currently depends on manual chart review that is laborious and varies between reviewers.

The team developed and tested the agentic system in several phases: a retrospective evaluation, a prospective silent deployment over 3 months, and a randomized crossover study with clinical trial staff. The system extracts presence, temporality, severity grade, attribution and certainty of six event types.

In the crossover study, assistance improved complete-match accuracy (odds ratio 1.45; 95% CI 1.01-2.08; P = 0.045) and sharply raised agreement between annotators, indicating more consistent assessments when humans verify AI output.