Moderate evidence
More
Doctor-persona prompts swung AI clinical decisions by 9.5 points in a 20-model test
Study title: Personas shift clinical action thresholds in large language models
- Evidence type
- Benchmark or lab evaluation
- Sample
- n = 5,000
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2937cite or check it
Study title: Personas shift clinical action thresholds in large language models
Researchers tested whether the decision style written into a system prompt changes what clinical AI recommends when the patient facts do not change. They gave 20 open-weight language models nine physician personas and measured 5,000,000 yes/no decisions. Without a persona, models said 'Yes' to 42.8% of decisions; with personas, the rate ranged from 36.9% to 46.4%.
Read the study abstract
Clinical large language model (LLM) deployment is shifting from feasibility to liability, while current guidance largely treats model behavior as a control problem. We tested whether decision-style system prompts shift clinical action thresholds when clinical facts are held constant, and whether these shifts are consistent across settings and models. We defined nine physician personas by crossing three ethical orientations (duty-based, care-based, utilitarian) with three cognitive styles (intuitive, integrative,…
The problem
In a test of 20 open-weight language models on 5,000,000 decisions, doctor-persona prompts moved the share of 'Yes' answers from 36.9% to 46.4% (a 9.5-point swing) with the clinical facts held constant.
In brief
The result matters because hospitals and vendors often treat the system prompt as a tone or communication setting. The study found it behaves more like a policy setting that moves thresholds for treatment, autonomy and other decisions, which bears on liability as clinical AI deployment grows.
Models differed in how easily they were swayed, from 4.9 to 16.1 points, and neither medical fine-tuning nor larger size gave consistent protection. The work used simulated and retrospective text, so it does not show effects on real patients.
Main points
- The team defined nine physician personas by crossing three ethical orientations (duty-based, care-based, utilitarian) with three cognitive styles (intuitive, integrative, analytic).
- Twenty open-weight language models were tested on 2500 simulated emergency department vignettes and 2500 MIMIC-IV-Note discharge summaries.
- Each text got five yes/no decisions on safety, autonomy, treatment, resource use and follow-up, and each condition was repeated ten times, for 5,000,000 decisions.
- Under baseline prompting, models answered 'Yes' to 42.8% of decisions; persona prompts moved that rate between 36.9% and 46.4%.
- Effects were largest for autonomy and treatment decisions.
- Results held across both text sets, with 85.7% directional agreement and a correlation of 0.82 for effect sizes.
- Susceptibility to personas ranged from 4.9 to 16.1 points by model, with no consistent protection from medical fine-tuning or model size.
The full story
The study was designed to hold clinical evidence fixed while changing only the decision-style instructions. Nine personas combined an ethical orientation with a cognitive style, and 20 open-weight models answered five binary questions about each case, covering safety, autonomy, treatment, resource use and follow-up.
Cases came from two sources: 2500 simulated emergency department vignettes and 2500 discharge summaries from the MIMIC-IV-Note dataset. Repeating each condition ten times produced 5,000,000 decisions in total.
Baseline prompting yielded 42.8% affirmative answers. Persona prompts produced a range from 36.9% to 46.4%, a 9.5-point swing. The shifts were biggest for autonomy and treatment items and agreed in direction across the two corpora in 85.7% of comparisons.
Models varied widely, with susceptibility between 4.9 and 16.1 points, and neither medical fine-tuning nor size reliably reduced it. The authors conclude that prompting should be treated as a first-class deployment configuration.
Sources
1 on file- npj Health Systemsdoi:10.1038/s44401-026-00121-z
Talk it over
Doctor-persona prompts moved the share of 'Yes' answers from 36.9% to 46.4% with the clinical facts held constant. How would you weigh that swing against the idea that a system prompt is just a tone setting?
Doctor-persona prompts swung AI clinical decisions by 9.5 points in a 20-model testhttps://truvace.com/item/personas-shift-clinical-action-thresholds-in-large-language-models
Doctor-persona prompts moved the share of 'Yes' answers from 36.9% to 46.4% with the clinical facts held constant. How would you weigh that swing against the idea that a system prompt is just a tone setting?
Answer in the threadThe 5,000,000 decisions came from 2500 simulated emergency department vignettes and 2500 MIMIC-IV-Note discharge summaries. How far can text like that tell us what these models would do with real patients?
Answer in the threadIf you were a hospital choosing a system prompt for one of these models, which of the nine personas or which safeguard would you want in place before using it, and why?
Answer in the thread
If you have more time
Models differed in susceptibility from 4.9 to 16.1 points. What would you want to look up about how a particular model was tested before trusting its prompt settings?
Answer in the threadEffects were largest for autonomy and treatment decisions. How might that play out in a clinic or hospital you know, and who should check the prompt settings?
Answer in the thread
ace
What readers say