Moderate evidence
More
Confidence badges on AI answers swayed 149 people, for better and worse, a study finds
Study title: Displayed confidence amplifies reliance on AI verdicts: Benefits when correct, costs when wrong
- Evidence type
- Controlled or validated comparison
- Sample
- n = 149
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-3045cite or check it
Study title: Displayed confidence amplifies reliance on AI verdicts: Benefits when correct, costs when wrong
A within-subject experiment with 149 participants tested how a confidence label on an AI assistant's answers changes people's decisions. Participants judged 16 general knowledge statements, then judged them again after seeing a simulated assistant's true/false verdicts marked with a high- or low-confidence badge.
Read the study abstract
Large language models (LLMs) produce fluent text that is often only partially correct; therefore, users must decide what to accept. How certainty is displayed may shape these decisions, independent of whether the underlying claim is correct. In a fully within-subject experiment (N = 149), participants judged 16 general knowledge statements in a fixed order before and after viewing a simulated assistant's true/false verdicts accompanied by a high- or low-confidence badge. Verdict accuracy was fixed at 50%, and ac…
58Gain side
How the two sides scoreClose call
58Problem side
The gain
In an experiment with 149 participants, a high-confidence badge on a simulated AI assistant's verdicts raised final accuracy by 4.2 percentage points when the verdict was correct.
The problem
In an experiment with 149 participants, a high-confidence badge on a simulated AI assistant's verdicts lowered final accuracy by 6.7 percentage points when the verdict was wrong, though the badge carried no information about correctness.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
In brief
The high-confidence badge made people follow the verdict more, whether it was right or wrong. Final accuracy rose by 4.2 percentage points when the verdict was correct and fell by 6.7 points when it was wrong.
Verdict accuracy was fixed at 50% and the badge was assigned independently of correctness, so it told participants nothing about whether an answer was right. The authors call for testing calibrated, claim-level confidence displays and verification pathways.
Main points
- The study used a fully within-subject design with 149 participants.
- Each participant judged 16 general knowledge statements in a fixed order before and after seeing the assistant's verdicts.
- Verdicts were true/false and carried a high- or low-confidence badge.
- Verdict accuracy was fixed at 50%, and accuracy and confidence displays were assigned orthogonally.
- High confidence raised final accuracy by 4.2 points (95% CI 0.4 to 8.0) when the verdict was correct.
- High confidence lowered final accuracy by 6.7 points (95% CI -12.2 to -1.2) when the verdict was wrong.
- The badge raised relative reliance on the AI and lowered relative reliance on participants' own judgment.
The full story
Large language models produce fluent text that is often only partly correct, so users must decide what to accept. The researchers asked whether the way certainty is displayed shapes those decisions regardless of whether the claim is true.
Participants answered 16 general knowledge statements, saw a simulated assistant's verdicts with a high- or low-confidence badge, and then judged again. Because accuracy was fixed at 50% and assigned independently of the badge, any effect of the badge could not reflect real information about correctness.
The high-confidence badge increased adherence to verdicts in both cases. Splitting the effect into two pathways showed the badge raised relative AI reliance and lowered relative self-reliance. The authors conclude that displayed confidence influences decisions even without diagnostic value, and propose testing calibrated claim-level confidence displays and verification pathways.
What this doesn’t fix
The assistant was simulated, verdict accuracy was fixed at 50%, and the task was general knowledge statements, so results may not carry over to real LLM use.
Sources
1 on file- Acta Psychologicadoi:10.1016/j.actpsy.2026.108130
Talk it over
A high-confidence badge raised final accuracy by 4.2 percentage points when the verdict was correct and cut it by 6.7 points when wrong. How would you weigh those two effects against each other?
Confidence badges on AI answers swayed 149 people, for better and worse, a study findshttps://truvace.com/item/displayed-confidence-amplifies-reliance-on-ai-verdicts-benefits-when-correct-cos
A high-confidence badge raised final accuracy by 4.2 percentage points when the verdict was correct and cut it by 6.7 points when wrong. How would you weigh those two effects against each other?
Answer in the threadIn the experiment, verdict accuracy was fixed at 50% and the badge was assigned independently of correctness. What does that design let us say about the badge, and how might real AI tools differ?
Answer in the threadIf you were one of the 149 participants judging the 16 general knowledge statements, how would you decide whether to follow a verdict marked high-confidence?
Answer in the thread
If you have more time
The authors suggest testing calibrated, claim-level confidence displays and verification pathways. What would you want such a display to show, and what result would tell you it works?
Answer in the threadThink of a tool you know that shows a confidence score or rating. How could you check whether that number tracks how often the tool is right?
Answer in the thread
ace
What readers say