52/100

Moderate evidence

Cite
More
EmailLinkedInX

Chatbots gave patients uneven answers on prenatal DNA screening, a three-chatbot analysis finds

Study title: What Patients are Reading About Prenatal Cell-Free DNA: An Analysis of Leading Publicly Available AI-Chatbots

Evidence type
Benchmark or lab evaluation
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: What Patients are Reading About Prenatal Cell-Free DNA: An Analysis of Leading Publicly Available AI-Chatbots

Researchers analysed what three publicly available AI chatbots say about prenatal cell-free DNA (cfDNA) screening, a blood test used in pregnancy. Their content analysis, published in Prenatal Diagnosis, found that answers varied substantially across platforms and across simulated user profiles, and included misinformation, omissions and incomplete information.

Read the study abstract

Objective: To characterize the quality, consistency, and framing of information about prenatal cell-free DNA (cfDNA) screening generated by publicly available chatbots, including variation across simulated user age profiles. Methods: We conducted a qualitative content analysis of outputs from three publicly available chatbots. To reduce carryover effects and personalization, prompts were entered using a VPN, private browsing, and cleared browser cookies. Outputs were generated using distinct simulated user age p…

What Patients are Reading About Prenatal Cell-Free DNA: An Analysis of Leading Publicly Available AI-Chatbots
"Telemedicine Consult" by Intel Free Press, CC BY-SA 2.0.

The problem

In a qualitative analysis of three public chatbots, researchers found misinformation, frequent omission of cfDNA screening limitations, wording that sometimes confused screening with diagnosis, and few or no citations to high-quality sources.

In brief

Common problems were inconsistent explanations of what the screening is for and how far it reaches, and frequent omission of the test's limitations and what follows from a result. Some wording blurred the line between screening and diagnostic testing, and readability ranged from a 9th to a 14th grade level.

The authors say this matters because patients increasingly use such tools outside clinical settings. Answers also shifted in tone, complexity and emphasis depending on the simulated user's age, which the authors read as implicit tailoring of how information is presented.

Main points

  1. The study was a qualitative content analysis of outputs from three publicly available chatbots on prenatal cfDNA screening.
  2. Prompts were entered through a VPN, private browsing and cleared cookies to reduce carryover effects and personalization.
  3. Outputs were generated for distinct simulated user age profiles.
  4. Chatbot answers differed substantially between platforms and profiles, with misinformation, omissions and incomplete information identified.
  5. Test limitations and downstream implications were frequently left out.
  6. Readability ranged from a 9th to a 14th grade level, and some terminology conflated screening with diagnostic testing.
  7. Citations to high-quality sources were limited or absent.
  8. Age profiles were associated with differences in tone, complexity and emphasis.

The full story

The team examined informational completeness, how benefits and limitations were framed, terminology, source transparency and age-related differences in responses from three publicly available chatbots. To limit personalization, they used a VPN, private browsing and cleared browser cookies for each prompt.

They found inconsistent explanations of the purpose and scope of cfDNA screening, frequent omission of limitations and downstream implications, and readability at a 9th to 14th grade level. Terminology sometimes conflated screening with diagnostic testing, and citations to high-quality sources were limited or absent.

Responses to different simulated age profiles differed in tone, complexity and emphasis, which the authors suggest reflects implicit tailoring. They conclude that chatbots may provide inaccurate or inconsistent cfDNA information and that accuracy, transparency, discussion of limitations and source citation need to improve.

Sources

1 on file
  1. Prenatal Diagnosisdoi:10.1002/pd.70291
    Peer-reviewed2026-11-14

Talk it over

Three questions to start on, two more if you have time
Researchers found chatbot answers on cfDNA screening that sometimes blurred screening with diagnosis and often left out limitations. Which of those problems would matter most to a person reading the answer alone, and why?

Start with a question

Chatbots gave patients uneven answers on prenatal DNA screening, a three-chatbot analysis findshttps://truvace.com/item/what-patients-are-reading-about-prenatal-cell-free-dna-an-analysis-of-leading-pu

  1. Researchers found chatbot answers on cfDNA screening that sometimes blurred screening with diagnosis and often left out limitations. Which of those problems would matter most to a person reading the answer alone, and why?

    Answer in the thread
  2. The researchers used simulated age profiles, a VPN and cleared cookies, and no real patients. How far can that setup tell us what pregnant people actually see when they ask these chatbots?

    Answer in the thread
  3. Readability ranged from a 9th to a 14th grade level, and tone shifted by age profile. Who might be missed by answers pitched that way?

    Answer in the thread
If you have more time
  • If you put the same cfDNA screening question to the three chatbots, what would you look for in the answers, and which sources would you compare them with?

    Answer in the thread
  • Who should be responsible for making sure chatbot answers about cfDNA screening name its limitations and cite high-quality sources: the chatbot makers, health agencies or clinicians?

    Answer in the thread

What readers say