aceOveruse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios
Background: The Choosing Wisely initiative made recommendations where screening, testing, or treatment may be overused, with caveats around appropriate use. Clinicians are increasingly using large language models (LLMs). Objective: To evaluate whether commercial LLMs appropriately call for restraint or action from scenarios based on Choosing Wisely recommendations. Design: Benchmark evaluation. Fifteen Choosing Wisely recommendations were used to develop 225 scenarios (per recommendation, 10 in which restraint w…
Across 225 benchmark scenarios, six commercial LLMs rarely withheld appropriate care, with a false restraint rate of 0.6% and none for 11 of 15 recommendations.
The same LLMs endorsed unnecessary action in 5.4% of responses overall, concentrated in thyroid ultrasound for an incidental subcentimeter nodule (42.8%) and routine electrocardiography in low-risk adults (28.3%).
The study used single prompts without follow-up, was a benchmark rather than real-world use, and findings may not extend to consumer-facing use.
Evidence
- Peer-reviewedAnnals of Internal Medicine2026-10-20
How should this claim be treated?
Truvace Impact Record TRV-2026-1976, v1: “Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios.” Truvace, 2026-10-20. /record/TRV-2026-1976 (accessed at citation time). sha256 0b31529cedb5888c…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1976 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.