Truace
Subscribe

AI chatbots score highly on plastic surgery anatomy questions but a few make serious errors

Study title: Large Language Models and Applied Surgical Anatomy: Evaluation of Anatomical Reasoning in Expert-Derived Plastic Surgery Scenarios

Researchers tested six chatbots, ChatGPT, Gemini, Claude, DeepSeek, Copilot and Manus, on 25 applied surgical anatomy scenarios written by an anatomy professor and two plastic surgeons. Answers were graded on accuracy, clinical reasoning, completeness and safety, up to 10 points per question.

Read the study abstract

Large language models (LLMs) are increasingly being integrated into medical education, research, and clinical decision-support applications; however, their ability to apply anatomical knowledge in clinically relevant surgical scenarios remains insufficiently studied. This study evaluated and compared the performance of six contemporary LLMs (ChatGPT, Gemini, Claude, DeepSeek, Copilot, and Manus) using 25 expert-derived applied surgical anatomy scenarios relevant to plastic surgery. Scenarios were developed by an…

By the Truvace desk1 min read1 sourceEvidence: Emerging · 44/100 — Benchmark or lab evaluation · n = 25 · 1 independent sourcePeer-reviewedTRV-2026-2289· cite & verify
Large Language Models and Applied Surgical Anatomy: Evaluation of Anatomical Reasoning in Expert-Derived Plastic Surgery Scenarios
Facial Plastic Surgeon Using Surgical Loupes and Headlight During Procedure by BalikianFacialPlasticSurgery. CC BY 4.0 · https://creativecommons.org/licenses/by/4.0
Trace impact readingNegative state
P 52The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 47The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

The results suggest current chatbots handle this kind of anatomy reasoning well, which may matter as they enter medical education and decision support. Yet the rare critical errors, all in one vascular physiology scenario, show that high averages can hide clinically relevant mistakes, and the benchmark cannot show how the tools perform in real surgical practice.

Main points

  1. Models differed significantly in overall performance (Friedman χ2 = 43.51, p < 0.001).
  2. Scores ranged from ChatGPT 250/250 and Gemini 248/250 to DeepSeek 241, Copilot 240, Claude 235 and Manus 219.
  3. Critical errors were infrequent and confined to one reconstructive vascular physiology scenario.
  4. ChatGPT, Gemini and Copilot made no critical errors; DeepSeek, Claude and Manus made one each.

The gain

On 25 expert-written plastic surgery anatomy scenarios, ChatGPT scored 250/250 and Gemini 248/250, with all six chatbots tested scoring at least 219/250.

The problem

Despite high scores, DeepSeek, Claude and Manus each made one critical error in a reconstructive vascular physiology scenario, showing anatomical misconceptions that could matter for surgical decisions.

The rundown

The scenarios were developed by a panel of one associate professor of anatomy and two plastic surgeons, and each model received a standardized prompt. A structured rubric scored anatomical accuracy, clinical reasoning, completeness and safety, and critical errors were assessed separately.

Although all models performed excellently overall, the authors say future evaluations should look beyond overall accuracy to clinically meaningful error patterns that could influence surgical decision-making.

What this doesn’t fix

Ceiling effects limited discrimination among top models, so the results reflect performance on this specific benchmark rather than a general ranking of anatomical reasoning ability.

Sources

  1. Peer-reviewedClinical Anatomy2026-10-26

The debate