Emerging evidence
More
ChatGPT-5 matched dental antibiotic guidelines in 10 of 17 cases, but not for weak immunity
Study title: Guideline Concordance of ChatGPT-Generated Responses on Antibiotic Prophylaxis in Endodontics
- Evidence type
- Benchmark or lab evaluation
- Sample
- n = 17
- Sources
- 1 sourcePeer-reviewed
- Published
- 1 min read
- Truvace record
- TRV-2026-1200cite or check it
Study title: Guideline Concordance of ChatGPT-Generated Responses on Antibiotic Prophylaxis in Endodontics
A descriptive study from September 2026 evaluated ChatGPT-5 on 17 endodontic antibiotic prophylaxis scenarios based on AHA, AAE, and ADA guidelines. An experienced endodontist rated each response, finding full agreement in 10 cases, partial agreement in 4, and disagreement in 3, with stronger performance on infective endocarditis prophylaxis and antibiotic selection, dosage, and timing.
Read the study abstract
Antibiotic prophylaxis decision-making in endodontics represents a core competency in dental education, requiring the integration of systemic risk assessment, guideline interpretation, and antimicrobial stewardship principles. Although artificial intelligence (AI) tools such as ChatGPT are increasingly accessed by students and clinicians for clinical guidance, their educational reliability in teaching evidence-based antibiotic prophylaxis remains unclear. This descriptive study evaluated the accuracy and guideli…

48Gain side
How the two sides scoreClose call
51Problem side
The gain
ChatGPT-5 fully agreed with dental guidelines in 10 of 17 antibiotic prophylaxis scenarios, as rated by one endodontist, doing best on infective endocarditis and on antibiotic choice, dose and timing.
The problem
ChatGPT-5 expanded prophylaxis beyond guidelines in 4 of 17 scenarios and disagreed in 3 involving immunocompromised patients; none was judged unsafe, but the expansion risks teaching inappropriate prescribing.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
In brief
The mixed results matter because students and clinicians increasingly consult AI for clinical guidance, and over-expansion of prophylaxis in medically complex patients could teach inappropriate prescribing and undermine antimicrobial stewardship, even though no response was deemed clinically unsafe. Uncertainty remains about generalizability beyond 17 scenarios and single-rater assessment.
Main points
- Study tested 17 clinically relevant questions grounded in AHA, AAE, and ADA guidelines submitted to ChatGPT-5.
- Responses were scored by an experienced endodontist on clinical correctness, guideline concordance, and educational risk using a five-point Likert scale.
- Full agreement occurred in 58.8% of scenarios, partial agreement in 23.5% due to expanded indications, disagreement in 17.6% involving immunocompromised conditions.
- No response was deemed clinically unsafe, but expanded recommendations were flagged as educational risk for inappropriate prescribing.
The full story
Researchers submitted 17 guideline-grounded endodontic questions to ChatGPT-5 and had an experienced endodontist rate responses for correctness, concordance, and educational risk, supplemented by qualitative analysis of indication expansion and reasoning depth.
Results showed 10 scenarios with full agreement, 4 with partial agreement linked to expanded indications, and 3 with disagreement centered on HIV with low CD4 counts and chemotherapy-related neutropenia, with no response judged clinically unsafe but with noted risk for teaching inappropriate use.
What this doesn’t fix
Findings are limited to 17 predefined scenarios assessed by a single experienced endodontist and do not support conclusions about educational effectiveness or curricular implementation.
Sources
1 on file- Journal of Dental Educationdoi:10.1002/jdd.70343
Talk it over
ChatGPT-5 fully agreed with the guidelines in 10 of 17 scenarios but expanded prophylaxis in 4, and none was judged unsafe. How would you weigh the 10 full matches against the risk of teaching inappropriate prescribing?
ChatGPT-5 matched dental antibiotic guidelines in 10 of 17 cases, but not for weak immunityhttps://truvace.com/item/guideline-concordance-of-chatgpt-generated-responses-on-antibiotic-prophylaxis-i
ChatGPT-5 fully agreed with the guidelines in 10 of 17 scenarios but expanded prophylaxis in 4, and none was judged unsafe. How would you weigh the 10 full matches against the risk of teaching inappropriate prescribing?
Answer in the threadOne endodontist alone rated all 17 scenarios, and the 17 were set in advance from AHA, AAE and ADA guidelines. How might a second endodontist's ratings change the 10 full agreements, and what would settle it?
Answer in the threadThe 3 disagreements involved HIV with low CD4 counts and chemotherapy-related neutropenia. If you were a dental student who got one of those answers, what would you do before acting on it?
Answer in the thread
If you have more time
Look up what the AHA, AAE and ADA guidelines say about immunocompromised patients. Where might a chatbot answer reasonably differ from them, and who should decide?
Answer in the threadWhat would a follow-up study need, such as more raters or more scenarios, to show whether ChatGPT-5 expanding prophylaxis is a pattern or a quirk of these 17 questions?
Answer in the thread
ace
What readers say