52/100

Moderate evidence

Cite
More
EmailLinkedInX

AI chatbot answers on tooth whitening and veneers too hard to read, study of 126 replies finds

Study title: Quality, safety, and readability of large language model responses to patient-derived questions on tooth whitening, resin infiltration, composite bonding, and porcelain veneers

Evidence type
Benchmark or lab evaluation
Sample
n = 126
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Quality, safety, and readability of large language model responses to patient-derived questions on tooth whitening, resin infiltration, composite bonding, and porcelain veneers

A study in The Saudi Dental Journal put 42 patient-derived questions about tooth whitening, resin infiltration, composite bonding and porcelain veneers to ChatGPT, Gemini and Meta AI. Two evaluators scored all 126 answers for quality, safety and readability. None of the three platforms met the recommended Grade 8 reading level, with Flesch-Kincaid grade levels between 10.2 and 16.5.

Read the study abstract

Patients are increasingly consulting large language model (LLM) platforms for information regarding aesthetic dental procedures prior to clinical appointments. This study evaluated the quality, safety, and readability of responses produced by ChatGPT, Gemini, and Meta AI in relation to patient-derived questions. Forty-two patient-derived questions regarding tooth whitening, resin infiltration, composite bonding, and porcelain veneers were presented to each platform. Two calibrated evaluators independently assess…

The problem

In a study scoring 126 chatbot answers to 42 patient questions on whitening, bonding and veneers, none of ChatGPT, Gemini or Meta AI met the Grade 8 reading target (Flesch-Kincaid 10.2 to 16.5) or all benchmarks for complete, safe answers.

In brief

ChatGPT scored lowest on quality. Average Global Quality Scores were 3.60 for ChatGPT, 3.98 for Gemini and 4.15 for Meta AI, and ChatGPT trailed both rivals on the rubric total and the Modified DISCERN tool. Gemini and Meta AI differed only on actionability, where Meta AI scored higher.

The authors conclude that no platform consistently delivered complete, safe and accessible answers, and they call for professional oversight when patients use chatbots for guidance before a dental consultation. The study scored the text of answers rather than following patients, so it does not show how people acted on the advice.

Main points

  1. Forty-two patient-derived questions covered tooth whitening, resin infiltration, composite bonding and porcelain veneers, each put to ChatGPT, Gemini and Meta AI, giving 126 responses.
  2. Two calibrated evaluators scored every response with a five-domain quality rubric, a five-item Modified DISCERN tool and the Global Quality Score.
  3. Average Global Quality Scores were 3.60 for ChatGPT, 3.98 for Gemini and 4.15 for Meta AI.
  4. ChatGPT scored significantly lower than Gemini and Meta AI on rubric total, Modified DISCERN and Global Quality Score (all adjusted p < 0.005).
  5. Accuracy and patient-centeredness did not differ significantly among the platforms (p = 0.292 and p = 0.071).
  6. No platform reached the recommended Grade 8 reading level; Flesch-Kincaid grade levels ranged from 10.2 to 16.5.

The full story

Readability and comprehension were measured with the Flesch Reading Ease Score and the Flesch-Kincaid Grade Level. Every platform exceeded the Grade 8 target the authors cite as recommended for patient information, so the answers were harder to read than that benchmark.

Statistical comparisons used the Friedman test for repeated measures, followed by paired Wilcoxon signed-rank tests with Bonferroni correction. Rubric total, Modified DISCERN total and Global Quality Score all differed significantly across platforms (p < 0.001 for each).

Meta AI and Gemini outperformed ChatGPT on most quality dimensions relevant to aesthetic dental patient information. Gemini and Meta AI did not differ significantly except on actionability, where Meta AI scored higher.

Sources

1 on file
  1. The Saudi Dental Journaldoi:10.1007/s44445-026-00265-6
    Peer-reviewed2026-11-16

Talk it over

Three questions to start on, two more if you have time
None of ChatGPT, Gemini or Meta AI met the Grade 8 reading target, with Flesch-Kincaid grade levels from 10.2 to 16.5. How would you weigh hard-to-read answers against answers that were accurate?

Start with a question

AI chatbot answers on tooth whitening and veneers too hard to read, study of 126 replies findshttps://truvace.com/item/quality-safety-and-readability-of-large-language-model-responses-to-patient-deri

  1. None of ChatGPT, Gemini or Meta AI met the Grade 8 reading target, with Flesch-Kincaid grade levels from 10.2 to 16.5. How would you weigh hard-to-read answers against answers that were accurate?

    Answer in the thread
  2. Two evaluators scored all 126 answers, and their first-round agreement was only slight (kappa 0.011 to 0.191) before they reached consensus. How much weight would you put on the final scores?

    Answer in the thread
  3. If you were a patient asking about porcelain veneers before a dental consultation, what would you do with an answer written at a grade level of 16.5?

    Answer in the thread
If you have more time
  • Look up how the Flesch-Kincaid grade level is worked out. Which features of a chatbot answer on whitening or bonding might push it above Grade 8?

    Answer in the thread
  • The authors call for professional oversight when patients use chatbots before a dental consultation. Who should provide it, and what would it look like in a clinic you know?

    Answer in the thread

What readers say