aceAccuracy of Large Language Model Chatbots in Retrieving MRI Safety Conditions for Medical Implants
Background Retrieving MRI safety conditions for implanted devices is increasingly complex and time-consuming. Purpose To evaluate the accuracy of internet-enabled large language model (LLM) chatbots in retrieving implant-specific MRI safety conditions, using manufacturer documentation as the reference standard. Study type Prospective comparative evaluation in a constructed device cohort, followed by a retrospective clinical pilot. Population Fifty purposively selected device configurations and 20 patients (10 wi…
ChatGPT correctly retrieved about 94% of implant MRI safety parameters in a constructed cohort and 98% in a small clinical pilot, which could ease a time-consuming lookup task.
Even the best-performing chatbot made errors, and Gemini and Claude were accurate on only about 78-79% of parameters, so outputs need checking against manufacturer documentation before clinical use.
Devices were purposively selected, the clinical pilot was retrospective and involved only 20 patients and a newer ChatGPT version, and errors persisted even with the best chatbot.
Evidence
- Peer-reviewedJournal of Magnetic Resonance Imaging2026-10-21
How should this claim be treated?
Truvace Impact Record TRV-2026-2226, v1: “Accuracy of Large Language Model Chatbots in Retrieving MRI Safety Conditions for Medical Implants.” Truvace, 2026-10-24. /record/TRV-2026-2226 (accessed at citation time). sha256 d35f5a3311e1c3ed…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2226 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.