aceMedical Literature-Specific Artificial Intelligence: Analyzing the performance of OpenEvidence, Heidi Evidence, Consensus, and PubMed.ai in the context of Lacrimal Drainage Disorders
Purpose This study aimed to report the performance of medical literature-specific artificial intelligence (MLS-AI) deep learning tools in the context of lacrimal drainage disorders. Methods Prompt engineering was performed using 22 previously validated questions and statements to include common and rare aspects of lacrimal drainage disorders. Prompts were presented at least twice to the latest versions of OpenEvidence, Heidi Evidence, Consensus, and PubMed.ai [Accessed May 14-26, 2026]. The prompts included gene…
OpenEvidence answered 20 of 22 lacrimal drainage disorder prompts correctly and none incorrectly, suggesting some literature-specific AI tools can give largely reliable specialty answers.
PubMed.ai gave factually incorrect answers to 5 of 22 prompts and no response to 2, and Heidi hallucinated twice, showing some tools produce unreliable clinical information.
The authors graded the responses themselves on only 22 prompts in one ophthalmic subtopic, and they state that all responses currently need human oversight and verification before being accepted as true.
Evidence
- Peer-reviewedOphthalmic Plastic & Reconstructive Surgery2026-10-08
How should this claim be treated?
Truvace Impact Record TRV-2026-1740, v1: “Medical Literature-Specific Artificial Intelligence: Analyzing the performance of OpenEvidence, Heidi Evidence, Consensus, and PubMed.ai in the context of Lacrimal Drainage Disorders.” Truvace, 2026-10-17. /record/TRV-2026-1740 (accessed at citation time). sha256 2485e0dbb271e355…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1740 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.