+ HEALTH The accurate identification of children with refractory Mycoplasma pneumoniae pneumonia (RMPP) remains challe…± LABOR The advent of automation, artificial intelligence and disruptive technologies has affected the role of the Hu…+ ENTERTAINMENT This paper provides a systematic exploration of Artificial Intelligence (AI) for music generation within a cl…− POLICY Generative AI (GenAI) has shifted AI capabilities from discriminative prediction to creative interaction, off…± CLIMATE Automation and sustainability are closely intertwined in the evolution of Industry 4.0 and 5.0, where artific…+ ENTERTAINMENT This article aims to investigate the factors that affect behavioural intention (BI) and user behaviour (UB) a…+ CLIMATE With accelerating climate change and rising global energy consumption, the application of artificial intellig…− HEALTH Biases in medical artificial intelligence (AI) arise and compound throughout the AI lifecycle. These biases c…
TruaceTracing the truth around AISunday, July 19, 2026
Subscribe
Health·The Trace·Automated dual reading·Published 2026-07-19

LLM chatbots answering periodontal patient questions for patient education

Source article: Comparison of artificial intelligence-based chatbots and expert periodontists in responding to patient questions: a multi-dimensional analysis

Large Language Model (LLM) -based chatbots are increasingly used in patient information processes. The aim of this study was to compare the performance of ChatGPT (GPT-5.1), Gemini (2.5 Flash), and Claude (Sonnet 4.5) with expert periodontologists in responding to periodontal questions. Responses were evaluated in terms of scientific accuracy, completeness, conciseness & focus, empathy, and clarity, and differences among groups were investigated. The question pool was developed de novo based on clinical experien…

TRV-2026-0265Peer-reviewedPermanent record — cite & verify
Trace impact reading

Contested: both sides are scored from claims and sources, not community votes.

P 67The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.G 69The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.
Comparison of artificial intelligence-based chatbots and expert periodontists in responding to patient questions: a multi-dimensional analysis

"Hus Findery chatbots" by Antti Törrönen is licensed under CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/.

The quick read

A July 2026 peer-reviewed study compared ChatGPT GPT-5.1, Gemini 2.5 Flash, and Claude Sonnet 4.5 against expert periodontologists on 20 periodontal patient questions. Nine blinded periodontologists rated anonymized answers for scientific accuracy, completeness, conciseness & focus, empathy, and clarity.

The results matter because patients increasingly use chatbots for dental information. Comparable accuracy with higher completeness and empathy suggests a supportive role for AI in patient education, but lower conciseness in some models and the stated need for physician supervision highlight risks of information overload and unstructured delivery that could impair patient comprehension.

Main points
  • Compared ChatGPT (GPT-5.1), Gemini (2.5 Flash), and Claude (Sonnet 4.5) to three experienced periodontologists on 20 open-ended periodontal questions.
  • Nine independent periodontologists blindly rated anonymized responses on 5-point Likert scales for accuracy, completeness, conciseness & focus, empathy, and clarity.
  • No significant difference in scientific accuracy (p = 0.425); chatbots scored higher on completeness (p < 0.0001) and empathy (p < 0.0001).
  • Gemini scored lower on conciseness & focus, and clarity differed only between ChatGPT and Gemini, with experts using fewer and more focused expressions.
Gain

LLM chatbots answered periodontal patient questions with scientific accuracy comparable to expert periodontologists while scoring higher on completeness and empathy.

Problem

Some chatbot models showed lower conciseness & focus and clarity, producing longer less-focused answers that may make it harder for patients to maintain focus and perceive information in a structured manner.

The rundown

Researchers created 20 open-ended periodontal questions validated by Lawshe's method, collected expert answers from three periodontologists, and submitted the same standardized prompt to three LLMs, recording first responses.

All responses were anonymized and scored by nine independent periodontologists using Friedman test with Bonferroni correction; inter-rater agreement was within a good range across all domains.

What this doesn’t fix

Findings based on 20 de novo questions and first responses to a standardized prompt, with model differences affecting conciseness and clarity, requiring physician oversight before clinical use.

Sources

Reader signal

How should this claim be treated?

The debate