aceSemantic Clinical Knowledge Augmentation Improves Medical Question Answering Across Commercial and Open Large Language Models
Objective To evaluate whether semantic clinical knowledge augmentation through Semantic Clinical Artificial Intelligence (SCAI) improves medical question answering across commercial and open large language models, and to examine how a reasoning distilled model performs on the same benchmark. Materials and methods We evaluated Google Gemini, Microsoft Copilot, Meta Llama 3 70B, and DeepSeek-R1-Distill-Llama-70B on official text only United States Medical Licensing Exam (USMLE) sample questions from Step 1 (n = 87…
Pairing language models with the SCAI semantic knowledge pipeline raised accuracy on 313 USMLE sample questions for Gemini, Llama 3 70B, and DeepSeek-R1-Distill, with statistically significant gains on all three exam steps.
The evaluation used only official text-only USMLE sample questions, and the Copilot improvement was not statistically significant, so results may not extend to all models or to clinical practice.
Evidence
- Peer-reviewedIEEE Journal of Biomedical and Health Informatics2026-10-09
How should this claim be treated?
Truvace Impact Record TRV-2026-1676, v1: “Semantic Clinical Knowledge Augmentation Improves Medical Question Answering Across Commercial and Open Large Language Models.” Truvace, 2026-10-17. /record/TRV-2026-1676 (accessed at citation time). sha256 6eef8ae4cba3fba0…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1676 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.