44/100

Emerging evidence

Cite
More
EmailLinkedInX

ChatGPT-5 matched dental antibiotic guidelines in 10 of 17 cases, but not for weak immunity

Study title: Guideline Concordance of ChatGPT-Generated Responses on Antibiotic Prophylaxis in Endodontics

Evidence type
Benchmark or lab evaluation
Sample
n = 17
Sources
1 sourcePeer-reviewed
Published
1 min read

Study title: Guideline Concordance of ChatGPT-Generated Responses on Antibiotic Prophylaxis in Endodontics

A descriptive study from September 2026 evaluated ChatGPT-5 on 17 endodontic antibiotic prophylaxis scenarios based on AHA, AAE, and ADA guidelines. An experienced endodontist rated each response, finding full agreement in 10 cases, partial agreement in 4, and disagreement in 3, with stronger performance on infective endocarditis prophylaxis and antibiotic selection, dosage, and timing.

Read the study abstract

Antibiotic prophylaxis decision-making in endodontics represents a core competency in dental education, requiring the integration of systemic risk assessment, guideline interpretation, and antimicrobial stewardship principles. Although artificial intelligence (AI) tools such as ChatGPT are increasingly accessed by students and clinicians for clinical guidance, their educational reliability in teaching evidence-based antibiotic prophylaxis remains unclear. This descriptive study evaluated the accuracy and guideli…

Guideline Concordance of ChatGPT-Generated Responses on Antibiotic Prophylaxis in Endodontics
Root canal on a patient by Ruth Holcomb, U.S. Air Force. Public domain · Wikimedia Commons

48Gain side

How the two sides scoreClose call

51Problem side

The gain

ChatGPT-5 fully agreed with dental guidelines in 10 of 17 antibiotic prophylaxis scenarios, as rated by one endodontist, doing best on infective endocarditis and on antibiotic choice, dose and timing.

The problem

ChatGPT-5 expanded prophylaxis beyond guidelines in 4 of 17 scenarios and disagreed in 3 involving immunocompromised patients; none was judged unsafe, but the expansion risks teaching inappropriate prescribing.

A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.

What the scores mean

G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.

P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.

In brief

The mixed results matter because students and clinicians increasingly consult AI for clinical guidance, and over-expansion of prophylaxis in medically complex patients could teach inappropriate prescribing and undermine antimicrobial stewardship, even though no response was deemed clinically unsafe. Uncertainty remains about generalizability beyond 17 scenarios and single-rater assessment.

Main points

  1. Study tested 17 clinically relevant questions grounded in AHA, AAE, and ADA guidelines submitted to ChatGPT-5.
  2. Responses were scored by an experienced endodontist on clinical correctness, guideline concordance, and educational risk using a five-point Likert scale.
  3. Full agreement occurred in 58.8% of scenarios, partial agreement in 23.5% due to expanded indications, disagreement in 17.6% involving immunocompromised conditions.
  4. No response was deemed clinically unsafe, but expanded recommendations were flagged as educational risk for inappropriate prescribing.

The full story

Researchers submitted 17 guideline-grounded endodontic questions to ChatGPT-5 and had an experienced endodontist rate responses for correctness, concordance, and educational risk, supplemented by qualitative analysis of indication expansion and reasoning depth.

Results showed 10 scenarios with full agreement, 4 with partial agreement linked to expanded indications, and 3 with disagreement centered on HIV with low CD4 counts and chemotherapy-related neutropenia, with no response judged clinically unsafe but with noted risk for teaching inappropriate use.

What this doesn’t fix

Findings are limited to 17 predefined scenarios assessed by a single experienced endodontist and do not support conclusions about educational effectiveness or curricular implementation.

Sources

1 on file
  1. Journal of Dental Educationdoi:10.1002/jdd.70343
    Peer-reviewed2026-09-23

Talk it over

Three questions to start on, two more if you have time
ChatGPT-5 fully agreed with the guidelines in 10 of 17 scenarios but expanded prophylaxis in 4, and none was judged unsafe. How would you weigh the 10 full matches against the risk of teaching inappropriate prescribing?

Start with a question

ChatGPT-5 matched dental antibiotic guidelines in 10 of 17 cases, but not for weak immunityhttps://truvace.com/item/guideline-concordance-of-chatgpt-generated-responses-on-antibiotic-prophylaxis-i

  1. ChatGPT-5 fully agreed with the guidelines in 10 of 17 scenarios but expanded prophylaxis in 4, and none was judged unsafe. How would you weigh the 10 full matches against the risk of teaching inappropriate prescribing?

    Answer in the thread
  2. One endodontist alone rated all 17 scenarios, and the 17 were set in advance from AHA, AAE and ADA guidelines. How might a second endodontist's ratings change the 10 full agreements, and what would settle it?

    Answer in the thread
  3. The 3 disagreements involved HIV with low CD4 counts and chemotherapy-related neutropenia. If you were a dental student who got one of those answers, what would you do before acting on it?

    Answer in the thread
If you have more time
  • Look up what the AHA, AAE and ADA guidelines say about immunocompromised patients. Where might a chatbot answer reasonably differ from them, and who should decide?

    Answer in the thread
  • What would a follow-up study need, such as more raters or more scenarios, to show whether ChatGPT-5 expanding prophylaxis is a pattern or a quirk of these 17 questions?

    Answer in the thread

What readers say