Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1428
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-15T10:41:17.297481Z
status: published
lens: trace
sector: education
headline: Benchmarking Large Language Models (LLMs) Against Medical Students Using Psychometrically Evaluated Authentic Clinically Oriented Anatomy Multiple-Choice Questions (MCQs) at the Apply and Analyze Levels of Bloom's Taxonomy
dek: Large language models (LLMs) are increasingly used by medical students as study aids, yet evidence regarding their performance on higher-order, clinically oriented anatomy questions relative to that of students-particularly at the Apply and Analyze levels of Bloom's Taxonomy-remains limited. This study aimed to compare the accuracy of five LLMs (ChatGPT-5, Gemini 2.5 Pro, Claude, Microsoft Copilot, and DeepSeek R1) with that of medical students on authentic, psychometrically evaluated anatomy multiple-choice que…
gain_title: All five tested LLMs answered every one of 20 Apply- and Analyze-level anatomy questions correctly, while students averaged about two-thirds accuracy on the same items.
problem_title: When LLMs rated each other's anatomy answers, they scored them as good as or better than human observers did, never lower, which undermines the reliability of LLM-based peer assessment.
trace_subject: LLM performance and evaluation on clinically oriented anatomy multiple-choice questions for medical education
gain_reading: All five tested LLMs answered every one of 20 Apply- and Analyze-level anatomy questions correctly, while students averaged about two-thirds accuracy on the same items.
gain_evidence: All five LLMs achieved perfect accuracy (100%, 20/20), significantly outperforming students, whose overall accuracy was 67.95% (p < 0.001).
problem_reading: When LLMs rated each other's anatomy answers, they scored them as good as or better than human observers did, never lower, which undermines the reliability of LLM-based peer assessment.
problem_evidence: a leniency bias among peer LLM evaluators raises concerns about the reliability of LLM-based peer assessment
quick_read: Researchers at Bursa Uludag University Faculty of Medicine gave 20 clinically oriented anatomy multiple-choice questions to five LLMs: ChatGPT-5, Gemini 2.5 Pro, Claude, Microsoft Copilot and DeepSeek R1. Each model answered every question correctly, while students from the 2025-2026 academic year averaged 67.95%. Student accuracy dropped from Apply-level to Analyze-level questions, but the LLMs stayed at ceiling.

The results suggest current LLMs could serve as supplementary anatomy study tools, since they were unaffected by difficulty features that separated student performance. Yet the models' high ratings of each other's output raise doubts about using LLMs to grade LLMs. Because the test was small and every model scored perfectly, the study could not distinguish the models by accuracy.
limitation: The study used only 20 questions from a single institution, each given once to the models, and perfect LLM scores left zero variance, so the models could not be statistically compared.
tag: Dual reading
key_points: Students scored 74.79% on Apply-level items and 61.13% on Analyze-level items, a significant decline (p = 0.02). | Analyze-level questions discriminated between students more strongly than Apply-level ones (0.446 vs. 0.248; p < 0.001). | On rubric scoring, Gemini 2.5 Pro and Claude scored 10.0/10 for anatomical accuracy and reasoning, while Microsoft Copilot scored lowest at 8.0/10 per human observers. | The 20 questions were selected from a pool of 397 and each was presented once to every model using a standardized prompt.
rundown: The study was a retrospective, cross-sectional analysis. Ten Apply-level and ten Analyze-level questions were drawn from a 397-question pool and analyzed alongside anonymized, aggregated student examination records. Accuracy was compared with Fisher's exact test, and item difficulty and discrimination were compared with independent-samples t-tests.

Response quality was scored using a six-domain anatomy rubric by two human observers and by peer LLMs in a cross-evaluation design. All models scored maximally on correct answer selection, Bloom's taxonomy appropriateness, scientific consistency and absence of hallucination. Differences appeared only in anatomical accuracy and anatomical reasoning.

Because LLM accuracy had zero variance, inter-model statistical comparisons were not applicable. The authors conclude the findings support LLMs as supplementary tools while stressing the continued need for expert oversight.
sources:
- peer_reviewed | Clinical Anatomy | https://doi.org/10.1002/ca.70240 | 2026-10-01
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

930044f903ecb4bc4b266afe0558e6090980f49af81c471ec0b8d837337c24ed

previous

0000000000000000000000000000000000000000000000000000000000000000