The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2866
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T18:02:43.817881Z
status: published
lens: trace
sector: health
headline: An Analysis of Large Language Model Performance on Hematology Key Feature Questions
dek: Large language models (LLMs) demonstrate strong performance on multiple-choice questions but their ability to independently generate clinical decisions without pre-defined answer choices remains less established. We therefore evaluated LLM performance on post-graduate key feature questions (KFQs), which are designed to assess clinical decision-making at critical points. Thirteen LLMs were assessed in June 2026 using 30 novel hematology KFQs developed and reviewed by two hematologists with training in medical edu…
gain_title: In a test of 13 AI models on 30 new postgraduate hematology key feature questions, exact accuracy ranged from 73.3% to 96.7%, with Gemini 3 Flash answering 29 of 30 correctly.
problem_title: In the same 30-question hematology test of 13 AI models, 71 of 390 responses were wrong, and 49.3% of those errors were knowledge errors and 45.1% were decision errors.
trace_subject: AI model performance on hematology key feature questions
gain_reading: In a test of 13 AI models on 30 new postgraduate hematology key feature questions, exact accuracy ranged from 73.3% to 96.7%, with Gemini 3 Flash answering 29 of 30 correctly.
gain_evidence: exact accuracy ranged from 73.3% to 96.7% | Gemini 3 Flash achieved the highest observed accuracy (29/30; 96.7%
problem_reading: In the same 30-question hematology test of 13 AI models, 71 of 390 responses were wrong, and 49.3% of those errors were knowledge errors and 45.1% were decision errors.
problem_evidence: Among 71 incorrect responses, 35 (49.3%) were classified as knowledge errors, 32 (45.1%) as decision errors | Knowledge and decision errors remained common among incorrect responses
quick_read: Researchers tested 13 large language models in June 2026 on 30 new hematology key feature questions, which assess clinical decision-making at critical points without offering answer choices. Across 390 responses, exact accuracy ranged from 73.3% to 96.7%.

Gemini 3 Flash scored highest at 29 of 30 (96.7%). Gemini 3 Flash Force Thinking and OpenEvidence each got 27 of 30 (90.0%), while DeepSeek V3 scored lowest at 22 of 30 (73.3%). Differences between models were not statistically significant.

Of 71 wrong answers, 35 were knowledge errors, 32 were decision errors and four were comprehension errors. The authors say this supports checking factual accuracy and clinical judgment alongside overall scores.
limitation: The question bank was limited to 30 questions, which the authors say precludes conclusions about equivalent performance between models.
tag: Dual reading
key_points: Thirteen LLMs were assessed in June 2026 using 30 novel hematology key feature questions written and reviewed by two hematologists trained in medical education. | Each model assessed every case independently, with context reset between cases, producing 390 responses. | Exact accuracy ranged from 73.3% to 96.7%; Gemini 3 Flash led with 29 of 30 correct (95% CI, 82.8-99.9%). | Gemini 3 Flash Force Thinking and OpenEvidence both scored 27 of 30 (90.0%); DeepSeek V3 scored 22 of 30 (73.3%). | Model differences were not significant for exact correctness (Cochran's Q = 20.00; p = 0.067) or partial credit (Friedman's χ² = 17.95; p = 0.117). | Among 71 incorrect responses, 49.3% were knowledge errors, 45.1% decision errors and 5.6% comprehension errors.
rundown: The study targets a gap: LLMs perform well on multiple-choice questions, but their ability to reach clinical decisions without pre-defined answer choices is less established. Key feature questions were used because they test decisions at critical points in a case.

Responses were scored for exact correctness and partial credit and compared using mixed-effects regression, Cochran's Q and Friedman's test. None showed statistically significant differences between the models.

Incorrect answers were sorted into knowledge, decision and comprehension errors. Knowledge and decision errors were close to evenly split, while comprehension errors were rare. The authors call for research on whether learners can spot model errors and whether LLM help improves hematology decision-making.
sources:
- peer_reviewed | Journal of Cancer Education | https://doi.org/10.1007/s13187-026-03025-6 | 2026-11-12
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

c601aac7ca66b723c911533af421fa8d45da957afab3a6528f76ae75f4f94b1c

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000