Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1976
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-20T06:54:02.954910Z
status: published
lens: trace
sector: health
headline: Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios
dek: Background: The Choosing Wisely initiative made recommendations where screening, testing, or treatment may be overused, with caveats around appropriate use. Clinicians are increasingly using large language models (LLMs).
Objective: To evaluate whether commercial LLMs appropriately call for restraint or action from scenarios based on Choosing Wisely recommendations.
Design: Benchmark evaluation. Fifteen Choosing Wisely recommendations were used to develop 225 scenarios (per recommendation, 10 in which restraint w…
gain_title: Across 225 benchmark scenarios, six commercial LLMs rarely withheld appropriate care, with a false restraint rate of 0.6% and none for 11 of 15 recommendations.
problem_title: The same LLMs endorsed unnecessary action in 5.4% of responses overall, concentrated in thyroid ultrasound for an incidental subcentimeter nodule (42.8%) and routine electrocardiography in low-risk adults (28.3%).
trace_subject: LLM recommendations for restraint versus action in Choosing Wisely clinical scenarios
gain_reading: Across 225 benchmark scenarios, six commercial LLMs rarely withheld appropriate care, with a false restraint rate of 0.6% and none for 11 of 15 recommendations.
gain_evidence: False restraint was rare (0.6% [CI, 0.1% to 1.2%]), was absent for 11 of 15 recommendations
problem_reading: The same LLMs endorsed unnecessary action in 5.4% of responses overall, concentrated in thyroid ultrasound for an incidental subcentimeter nodule (42.8%) and routine electrocardiography in low-risk adults (28.3%).
problem_evidence: thyroid ultrasound for an incidental subcentimeter nodule (overuse recommended in 42.8% of LLM responses) | routine electrocardiography in low-risk adults (overuse recommended in 28.3% of LLM responses)
quick_read: Researchers built 225 clinical scenarios from 15 Choosing Wisely recommendations, 10 where restraint was appropriate and 5 where action was appropriate. They tested six commercial models from OpenAI, Anthropic, and Google, collecting 4050 responses in February and March 2026 through production interfaces.

The results suggest these models are generally cautious about withholding care, but they can reinforce low-value testing in specific situations. Because the work was a benchmark using single prompts, it does not show how clinicians or patients would be affected in practice.
limitation: The study used single prompts without follow-up, was a benchmark rather than real-world use, and findings may not extend to consumer-facing use.
tag: Dual reading
key_points: Overall LLM-recommended overuse was 5.4% (95% CI, 0.6% to 12.2%), and no overuse was recommended for 8 of 15 recommendations. | Two recommendations accounted for most overuse: thyroid ultrasound for an incidental subcentimeter nodule and routine electrocardiography in low-risk adults. | False restraint was 0.6% overall and highest for prostate-specific antigen screening at 3.3%. | The remaining 5 recommendations with overuse had rates of 0.6% to 3.3% of responses.
rundown: The design used 15 Choosing Wisely recommendations, each with 10 restraint scenarios and 5 action scenarios. Each of six models ran every scenario three times, using a prompt written from the clinician perspective.

Co-primary outcomes were overuse, meaning endorsing action when restraint was appropriate, and false restraint, meaning withholding an appropriate action. Responses were classified by an LLM judge with physician adjudication.

The authors concluded that the models rarely withheld recommended treatment when it was appropriate and that overuse was concentrated in two recommendations. The study had no human participants and reported no primary funding source.
sources:
- peer_reviewed | Annals of Internal Medicine | https://doi.org/10.7326/annals-26-01667 | 2026-10-20
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

0b31529cedb5888cf3ec27424c01adc8caf378346e0453a6b3b7eee3af0cd429

previous

0000000000000000000000000000000000000000000000000000000000000000