Moderate evidence
More
Study of 85 patients finds AI matches experts on tumor size but not on growth calls
Study title: Assessment of Vestibular Schwannoma Growth After Radiosurgery: Variability in Clinical Decision Thresholds and the Role of Artificial Intelligence
- Evidence type
- Observational study
- Sample
- n = 85
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2898cite or check it
Study title: Assessment of Vestibular Schwannoma Growth After Radiosurgery: Variability in Clinical Decision Thresholds and the Role of Artificial Intelligence
A retrospective review of 85 patients with vestibular schwannoma treated with stereotactic radiosurgery between 2001 and 2023 compared AI tumor segmentations with manual ones on 170 MRI examinations. AI agreed with experts on binary growth classification in 83.5% of cases (Cohen κ 0.47).
Read the study abstract
To quantify expert interobserver variability in longitudinal vestibular schwannoma growth assessment after radiosurgery and evaluate AI segmentation performance relative to human variability. Retrospective review of patients with sporadic unilateral VS treated with stereotactic radiosurgery (2001-2023). Manual and AI segmentations were compared across the cohort. A growth-enriched subset of 24 tumors underwent additional expert assessment. Agreement was evaluated using intraclass correlation coefficients (ICC),…

The problem
In a retrospective study of 85 radiosurgery patients (170 MRI exams), AI segmentation reached only 83.5% binary agreement with experts on tumor growth (Cohen κ 0.47), with limits of agreement from -53.33 to 66.60 percentage points.
In brief
The AI matched experts well on absolute tumor volumes (ICC 0.99 at baseline, 0.97 at follow-up) and its agreement with humans was comparable to agreement among the experts themselves. Agreement dropped for percent volume change (ICC 0.83) and for the 20% growth threshold that guides clinical decisions.
Experts also disagreed with each other on growth: Fleiss κ for ≥20% growth was 0.52, against ICC values of 0.95 to 0.98 for volumes. The authors conclude that AI should be benchmarked against this human variability when growth thresholds are applied.
Main points
- The cohort had 170 MRI examinations from 85 patients (45.9% female, median age 66 years, median follow-up 53 months).
- Human interobserver agreement was excellent for baseline volume (ICC 0.95), follow-up volume (ICC 0.98) and percent volume change (ICC 0.89).
- Human agreement on ≥20% growth classification fell to a Fleiss κ of 0.52.
- AI-expert agreement was ICC 0.99 for baseline volume and 0.97 for follow-up volume, but 0.83 for percent change; median Dice coefficient was 0.85 (IQR 0.77-0.90).
- Bland-Altman analysis showed a mean AI bias of +6.64 percentage points on percent volume change.
The full story
Researchers reviewed patients with sporadic unilateral vestibular schwannoma treated with stereotactic radiosurgery from 2001 to 2023. They compared manual and AI segmentations across the cohort, and a growth-enriched subset of 24 tumors received extra expert assessment; 11 of those 24 met the strict-majority growth definition.
Agreement was measured with intraclass correlation coefficients, Fleiss and Cohen κ, Bland-Altman analysis and Dice similarity coefficients. Volume measurements agreed closely between AI and experts, while derived growth measures varied more.
The Bland-Altman limits of agreement for percent change ran from -53.33 to 66.60 percentage points. Because clinical decisions rest on a 20% growth threshold, this spread matters more than the strong volume agreement.
Sources
1 on file- Journal of Imaging Informatics in Medicinedoi:10.1007/s10278-026-02516-8
Talk it over
AI matched experts on tumor volume (ICC 0.99 at baseline) but reached only 83.5% binary agreement on growth. How would you weigh the strong volume match against the weaker growth calls?
Study of 85 patients finds AI matches experts on tumor size but not on growth callshttps://truvace.com/item/assessment-of-vestibular-schwannoma-growth-after-radiosurgery-variability-in-cli
AI matched experts on tumor volume (ICC 0.99 at baseline) but reached only 83.5% binary agreement on growth. How would you weigh the strong volume match against the weaker growth calls?
Answer in the threadExperts disagreed with each other on growth too, with a Fleiss κ of 0.52. How should AI be judged when the human standard it is compared with is itself uneven?
Answer in the threadIf you were a patient treated with stereotactic radiosurgery, and a 20% growth threshold guided your next step, who would you want making the growth call, and why?
Answer in the thread
If you have more time
Growth was assessed by multiple experts only on a growth-enriched subset of 24 tumors. What would a larger review with several experts on every tumor need to include?
Answer in the threadThe limits of agreement for percent change ran from -53.33 to 66.60 percentage points. What spread would you accept before an AI measurement guided a treatment decision, and who should set it?
Answer in the thread
ace
What readers say