aceLarge Language Model-Generated Electronic Consultations: A Blinded Comparative Study of OpenEvidence and Gastroenterologist Electronic Consultations
Objective: To evaluate whether a context-specific large language model, OpenEvidence, can generate gastroenterology/hepatology eConsult responses for metabolic dysfunction-associated steatotic liver disease (MASLD), while characterizing the frequency and potential severity of errors. Patients and methods: In this retrospective cohort study at a tertiary academic medical center, 125 specialist electronic consultations (eConsults) for MASLD (from January 2020 to December 2024) were paired with OpenEvidence-generat…
In blinded ratings of 125 paired MASLD eConsults, responses generated by the OpenEvidence language model scored higher on quality and on appropriate information amount than specialist-written responses.
Among the 125 OpenEvidence-generated MASLD eConsults, 12 contained minor errors and 1 contained a potentially major error, so clinically relevant mistakes still occurred.
The study was a retrospective, single-center comparison of perceived quality for one condition, and the AI outputs still included errors, one potentially major.
Evidence
- Peer-reviewedMayo Clinic Proceedings: Digital Health2026-09-18
How should this claim be treated?
Truvace Impact Record TRV-2026-2035, v1: “Large Language Model-Generated Electronic Consultations: A Blinded Comparative Study of OpenEvidence and Gastroenterologist Electronic Consultations.” Truvace, 2026-10-20. /record/TRV-2026-2035 (accessed at citation time). sha256 d6fbdb06d0a5dda6…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2035 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.