aceCognition, oversight, and the safety case for documentation AI
Large language models (LLMs) can now draft hospital discharge summaries. Reported hallucination rates vary by measurement context, from about 1%-2% in structured summarisation to over 60% under provocation testing. The common argument that LLMs outperform a tired resident treats a high-stakes clinical and legal document as a text-quality problem. We argue that a potentially underappreciated concern is the weakening of clinician oversight when reviewing fluent machine-generated text, and that the joint clinician-…
When clinicians review fluent machine-generated discharge summaries, their oversight may weaken, and hallucination rates range from about 1%-2% in structured summarisation to over 60% under provocation testing.
The authors describe the oversight weakening as a potentially underappreciated concern, and the excerpt gives no empirical evidence that the proposed safeguards work.
Evidence
- Peer-reviewedJournal of Hospital Medicine2026-10-15
How should this claim be treated?
Truvace Impact Record TRV-2026-1487, v1: “Cognition, oversight, and the safety case for documentation AI.” Truvace, 2026-10-15. /record/TRV-2026-1487 (accessed at citation time). sha256 e4d163aad6a7ebb6…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1487 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.