Truace
Subscribe

AI chatbots answer newborn hearing screening questions well, but parents may find them hard to read

Study title: Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent-Centered Questions on Newborn Hearing Screening: A Cross-Sectional Comparative Study

Researchers put 38 parent-centred questions about newborn hearing screening to three chatbots: ChatGPT-5.1, Gemini 2.5 Flash and DeepSeek-V3. Responses were rated on accuracy, comprehensiveness, clarity and credibility, plus a quality tool and readability formulas. Overall quality was rated good, but the answers were hard to read, with Flesch Reading Ease scores of 51.5-59.3.

Read the study abstract

Aims and objectives: Parents increasingly use large language models (LLMs) to obtain health-related information; however, the quality, readability, and reliability of AI-generated patient education remain uncertain. This study evaluated the performance of three LLM-based chatbots in providing parent-centred information regarding newborn hearing screening. Methods: Thirty-eight parent-centred questions regarding newborn hearing screening were submitted to ChatGPT-5.1, Gemini 2.5 Flash, and DeepSeek-V3. Responses…

By the Truvace desk2 min read1 sourceEvidence: Emerging · 48/100 — Benchmark or lab evaluation · n = 38 · 1 independent sourcePeer-reviewedTRV-2026-2374· cite & verify
Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent-Centered Questions on Newborn Hearing Screening: A Cross-Sectional Comparative Study
Saint Kitts and Nevis moves toward national hearing program with support from U S Air Force audiologists (9541749) by U.S. Air Force photo by Tech. Sgt. Jessica Smith McMahan. Public domain
Trace impact readingContested
P 48The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 52The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

The models differed. DeepSeek-V3 beat both rivals on accuracy and comprehensiveness but scored lower on clarity, while Gemini 2.5 Flash had higher reading-ease scores. Because parents use chatbots to look up health information, readability and variability across tools affect how useful the answers are.

The authors conclude the chatbots may serve as complementary information resources for parents, and say further research is needed before routine clinical use.

Main points

  1. The study submitted 38 parent-centred newborn hearing screening questions to ChatGPT-5.1, Gemini 2.5 Flash and DeepSeek-V3.
  2. Responses were scored on a five-point Likert scale for accuracy, comprehensiveness, clarity and credibility.
  3. DeepSeek-V3 outperformed both other chatbots on accuracy and comprehensiveness (p < 0.05) but had lower clarity scores (p < 0.05).
  4. Flesch Reading Ease scores of 51.5-59.3 indicated difficult readability across the chatbots.
  5. Flesch-Kincaid Grade Level scores of 8.4-8.9 corresponded to an eighth-grade reading level.
  6. Gemini 2.5 Flash had higher reading-ease scores (p = 0.001).
  7. Modified EQIP quality scores did not differ significantly between models (p = 0.148) and indicated good quality.

The gain

Three chatbots answered 38 parent questions on newborn hearing screening at acceptable quality, with mEQIP scores indicating good quality and DeepSeek-V3 leading on accuracy and comprehensiveness.

The problem

Chatbot answers on newborn hearing screening stayed relatively hard for parents to read, with Flesch Reading Ease scores of 51.5-59.3 indicating difficult readability and quality varying across models.

The rundown

The cross-sectional comparison tested three large language model chatbots on 38 questions that parents might ask about newborn hearing screening. Each answer was rated for accuracy, comprehensiveness, clarity and credibility, and scored with the modified EQIP tool for quality.

Readability was measured with two formulas. Flesch Reading Ease scores of 51.5-59.3 signalled difficult text, while Flesch-Kincaid Grade Level scores of 8.4-8.9 matched roughly an eighth-grade reading level. Gemini 2.5 Flash scored higher on reading ease (p = 0.001).

On content, DeepSeek-V3 scored higher than ChatGPT-5.1 and Gemini 2.5 Flash for accuracy and comprehensiveness but lower for clarity. Quality scores on the modified EQIP tool did not differ significantly among the models and indicated good quality.

The authors judge chatbots generally acceptable and potentially complementary resources for parents, with substantial variability across models, and call for further research before routine clinical use.

What this doesn’t fix

The authors say substantial variability was observed across models and that further research is needed before routine clinical use.

Sources

  1. Peer-reviewedJournal of Evaluation in Clinical Practice2026-12-01

The debate