Truace
Subscribe

Can General-purpose Multimodal AI Reliably Measure Cobb Angles? A Head-to-head Diagnostic Agreement Study of GPT-5.1 and Gemini 3.1 Pro in Adolescent Idiopathic Scoliosis

General-purpose multimodal artificial intelligence (AI) models can interpret radiographs through natural-language prompts, but their quantitative agreement and anatomical traceability in adolescent idiopathic scoliosis remain uncertain. We compared GPT-5.1 and Gemini 3.1 Pro with a joint expert-consensus reference on 125 anonymized standing coronal whole-spine radiographs. Each model evaluated each image once in an independent chat session using the same standardized prompt; only the first response was retained.…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1361· cite & verify
Can General-purpose Multimodal AI Reliably Measure Cobb Angles? A Head-to-head Diagnostic Agreement Study of GPT-5.1 and Gemini 3.1 Pro in Adolescent Idiopathic Scoliosis
Hospital Universitari Doctor Peset, València 09 by 19Tarrestnom65. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0

In brief

Researchers gave 125 anonymized standing whole-spine radiographs to GPT-5.1 and Gemini 3.1 Pro, each prompted once in a separate chat session, and compared their Cobb angle measurements with a joint expert-consensus reference. Gemini 3.1 Pro agreed with the reference more closely than GPT-5.1 on every main measure reported.

The result matters because general-purpose chat models can read images from plain-language prompts, which could tempt clinical use in adolescent idiopathic scoliosis. Even the stronger model fell short of what autonomous measurement would need, and the study could not establish absolute accuracy or whether the models give repeatable answers.

Main points

  1. Gemini 3.1 Pro had an ICC of 0.842 versus 0.587 for GPT-5.1.
  2. Mean absolute error was 5.06° for Gemini versus 9.62° for GPT-5.1, and Gemini's mean bias was 3.46° versus 8.68°.
  3. Gemini placed 61.6% of measurements within 5° of the reference, versus 24.8% for GPT-5.1.
  4. Exact upper end-vertebra agreement was 14.4% versus 8.8%, and exact lower end-vertebra agreement was under 20% for both models.

The problem

Neither GPT-5.1 nor Gemini 3.1 Pro agreed closely enough with expert consensus on case-level Cobb angles and end-vertebra identification to measure scoliosis curves autonomously.

The rundown

The study used 125 anonymized standing coronal whole-spine radiographs. Each model evaluated each image once, using the same standardized prompt in an independent session, and only the first response was kept. Outcomes included intraclass correlation, mean absolute error, Bland-Altman analysis, error-threshold performance, categorical agreement, and end-vertebra concordance.

Categorical agreement favored Gemini 3.1 Pro: 80.0% versus 52.8% for angle-derived severity categories and 78.4% versus 52.8% for prompt-constrained management categories. Both differences were reported as paired P < 0.001.

The authors concluded that Gemini performed better under the tested single-session protocol but that neither model supported autonomous Cobb angle measurement, and that expert verification remains necessary.

Sources

  1. Peer-reviewedJournal of Imaging Informatics in Medicine2026-10-13

The debate