Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1740
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-17T22:01:58.101836Z
status: published
lens: trace
sector: health
headline: Medical Literature-Specific Artificial Intelligence: Analyzing the performance of OpenEvidence, Heidi Evidence, Consensus, and PubMed.ai in the context of Lacrimal Drainage Disorders
dek: Purpose This study aimed to report the performance of medical literature-specific artificial intelligence (MLS-AI) deep learning tools in the context of lacrimal drainage disorders. Methods Prompt engineering was performed using 22 previously validated questions and statements to include common and rare aspects of lacrimal drainage disorders. Prompts were presented at least twice to the latest versions of OpenEvidence, Heidi Evidence, Consensus, and PubMed.ai [Accessed May 14-26, 2026]. The prompts included gene…
gain_title: OpenEvidence answered 20 of 22 lacrimal drainage disorder prompts correctly and none incorrectly, suggesting some literature-specific AI tools can give largely reliable specialty answers.
problem_title: PubMed.ai gave factually incorrect answers to 5 of 22 prompts and no response to 2, and Heidi hallucinated twice, showing some tools produce unreliable clinical information.
trace_subject: Medical literature-specific AI tools answering lacrimal drainage disorder questions
gain_reading: OpenEvidence answered 20 of 22 lacrimal drainage disorder prompts correctly and none incorrectly, suggesting some literature-specific AI tools can give largely reliable specialty answers.
gain_evidence: OpenEvidence's responses were graded as correct in 90.9% (20/22), and partially correct in 9.1% (2/22), with no factually incorrect responses.
problem_reading: PubMed.ai gave factually incorrect answers to 5 of 22 prompts and no response to 2, and Heidi hallucinated twice, showing some tools produce unreliable clinical information.
problem_evidence: 22.7% (5/22) factually incorrect answers, and 9% (2/22) with no response | Heidi hallucinated twice
quick_read: Researchers tested four medical literature-specific AI tools, OpenEvidence, Heidi Evidence, Consensus and PubMed.ai, using 22 validated prompts on lacrimal drainage disorders, including general, rare and controversial topics. The authors graded each response as correct, partially correct or factually incorrect.

The results show that tools marketed for the same purpose can differ greatly in dependability, which matters for clinicians who may consult them for specialty questions. The study covers one narrow ophthalmic topic, and the authors conclude that all responses currently need human oversight and verification.
limitation: The authors graded the responses themselves on only 22 prompts in one ophthalmic subtopic, and they state that all responses currently need human oversight and verification before being accepted as true.
tag: Dual reading
key_points: Twenty-two previously validated questions and statements covered common and rare aspects of lacrimal drainage disorders. | OpenEvidence was correct on 90.9% of prompts, versus 63.6% for Heidi and 54.5% for Consensus. | PubMed.ai performed worst, with 22.7% factually incorrect answers and two prompts unanswered. | The authors say the tools could complement one another depending on context and prompt type.
rundown: Prompts were presented at least twice to the latest versions of each tool, accessed May 14-26, 2026. They ranged from managing congenital nasolacrimal duct obstructions to terminology such as idiopathic canalicular inflammatory disease, and included controversial topics such as silicone intubation and mitomycin-C in dacryocystorhinostomy.

Responses were assessed for quality, organization, clarity, evidence-based content, updated knowledge, specificity, speed and factual inaccuracies. Heidi was graded 63.6% correct, 31.8% partially correct and 4.6% incorrect, while Consensus was 54.5% correct, 40.9% partially correct and 4.6% incorrect.

The authors rank OpenEvidence slightly ahead of Heidi, with Consensus a close third and PubMed.ai far behind. They say the tools need further training to capture more indexed mainstream journal publications.
sources:
- peer_reviewed | Ophthalmic Plastic & Reconstructive Surgery | https://doi.org/10.1097/iop.0000000000003302 | 2026-10-08
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

2485e0dbb271e355ad09fb0b0f26d880cd4698f76ba8a929e4b3d5cfd5ca6a58

previous

0000000000000000000000000000000000000000000000000000000000000000