52/100

Moderate evidence

Cite
More
EmailLinkedInX

Chatbot advice on prostate biopsy was hard to read, urologists found

Study title: Evaluating Artificial Intelligence Chatbots for Patient Education on Prostate Biopsy in the Prostate Cancer Diagnostic Pathway

Evidence type
Benchmark or lab evaluation
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Evaluating Artificial Intelligence Chatbots for Patient Education on Prostate Biopsy in the Prostate Cancer Diagnostic Pathway

Five urologists rated the prostate biopsy information produced by ChatGPT, Gemini, Copilot and Perplexity. Each free chatbot got one standardized English-language question on March 18, 2026, and only its first response was scored. ChatGPT had the highest median DISCERN quality score, followed by Gemini, Perplexity and Copilot.

Read the study abstract

Prostate biopsy is an important step in the diagnosis of suspected prostate cancer. Many patients look for online information before the procedure, and artificial intelligence chatbots are becoming part of this process. This study evaluated the quality, readability, actionability, transparency, and guideline concordance of prostate biopsy information generated by ChatGPT, Gemini, Copilot, and Perplexity. A standardized English-language question was submitted once to the free version of each platform on March 18,…

The problem

In a test of ChatGPT, Gemini, Copilot and Perplexity, five urologists found every chatbot's prostate biopsy answer was written at an advanced reading level, and guideline concordance differed between platforms.

In brief

Understandability was highest for ChatGPT and Gemini, while actionability was similar across all four platforms. Perplexity scored highest on the Web Resource Rating, which assesses transparency. All responses read at advanced levels, which can make them harder for many patients to follow.

Guideline concordance with European Association of Urology recommendations was moderate to good overall but varied by platform. The authors conclude chatbots may offer useful introductory information, while clinician-guided counseling remains necessary to individualize preparation, explain risks and reinforce post-biopsy warning signs.

Main points

  1. The study compared four free chatbots: ChatGPT, Gemini, Copilot and Perplexity.
  2. Each chatbot received one standardized English-language question on March 18, 2026, in a new or temporary session, and only the first response was analyzed.
  3. Five independent urologists scored the answers with DISCERN, the Patient Education Materials Assessment Tool for Printable Materials, the Web Resource Rating scale and a European Association of Urology guideline concordance assessment.
  4. ChatGPT had the highest median DISCERN score, followed by Gemini, Perplexity and Copilot.
  5. Understandability was highest for ChatGPT and Gemini, and actionability was similar across platforms.
  6. Perplexity had the highest Web Resource Rating score.
  7. Coleman-Liau Index values showed advanced reading levels across all responses.

The full story

The study tested how well popular chatbots explain prostate biopsy, a key step in diagnosing suspected prostate cancer, to patients who often look for information online beforehand. It used a single standardized English question per platform, submitted to free versions in separate new or temporary sessions.

Five urologists rated the first response from each chatbot. Inter-rater agreement was substantial for all summary ratings. Scores covered quality (DISCERN), understandability and actionability, transparency (Web Resource Rating) and concordance with European Association of Urology guidance, and readability was measured with the Coleman-Liau Index.

ChatGPT led on quality, ChatGPT and Gemini led on understandability, and Perplexity led on transparency. Actionability was similar across the four. All responses were written at advanced reading levels, and guideline concordance was moderate to good overall with differences between platforms.

Sources

1 on file
  1. Journal of Cancer Educationdoi:10.1007/s13187-026-03014-9
    Peer-reviewed2026-11-16

Talk it over

Three questions to start on, two more if you have time
Five urologists scored one answer per chatbot, and only the first response was analyzed. What would you want to see from repeated questions before trusting the ranking of ChatGPT, Gemini, Perplexity and Copilot?

Start with a question

Chatbot advice on prostate biopsy was hard to read, urologists foundhttps://truvace.com/item/evaluating-artificial-intelligence-chatbots-for-patient-education-on-prostate-bi

  1. Five urologists scored one answer per chatbot, and only the first response was analyzed. What would you want to see from repeated questions before trusting the ranking of ChatGPT, Gemini, Perplexity and Copilot?

    Answer in the thread
  2. ChatGPT led on quality, Perplexity led on transparency, and every answer read at an advanced level. Which of these matters most to a patient facing a prostate biopsy, and why?

    Answer in the thread
  3. The authors say clinician-guided counseling remains necessary to explain risks and post-biopsy warning signs. Who should decide how chatbot information fits into that counseling?

    Answer in the thread
If you have more time
  • Find the European Association of Urology guidance on prostate biopsy. Which parts would you check a chatbot answer against first?

    Answer in the thread
  • The question was asked in English on one date, March 18, 2026. Which patients might get different answers, and how could a study find out?

    Answer in the thread

What readers say