Truace
Subscribe

AI assistants for cancer pain management: A comparative evaluation of ChatGPT and Gemini responses in terms of response quality and readability

BackgroundCancer pain is common and distressing in oncology patients, who often seek management information via online and AI-based tools.ObjectiveThis study aimed to evaluate the quality and readability of responses provided by ChatGPT-4 and Gemini-2 to frequently asked questions regarding cancer painMethodsOn April 15, 2025, responses were collected from each artificial intelligence (AI) model using a set of frequently asked questions about cancer pain. These questions were selected based on expert input from…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1599· cite & verify
AI assistants for cancer pain management: A comparative evaluation of ChatGPT and Gemini responses in terms of response quality and readability
"McArdle Laboratory for Cancer Research" by wackybadger is licensed under CC BY-SA 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/2.0/.

In brief

Researchers put ten frequently asked questions about cancer pain, chosen with input from oncology and pain management specialists, to ChatGPT-4 and Gemini-2 on April 15, 2025. Eleven independent experts rated the answers on accuracy, completeness, relevance and clinical usefulness, and readability was scored with the Flesch-Kincaid Grade Level.

The work matters because cancer patients often look for pain information from online and AI-based tools. The study suggests these assistants could help with patient education, but the abstract reports only statistical differences on some questions and reading-level variation, so it does not show how patients' pain outcomes change in practice.

Main points

  1. Ten cancer pain questions were put to both chatbots, and eleven experts rated the responses on a four-point Likert scale.
  2. Significant differences between the models appeared on questions 2, 3, 5, 7, 8 and 9.
  3. No significant difference appeared on questions 1, 4, 6 and 10, so quality was similar on those.
  4. ChatGPT-4 produced a more consistent range of reading levels, while Gemini-2 varied more and may be more sensitive to prompt type.

The gain

AI models answering cancer pain questions may support patient education and potentially aid clinical decision-making in pain management, with similar answer quality on some questions.

The rundown

The study used ChatGPT-4 and Gemini-2 and collected responses on a single date, April 15, 2025. Questions were selected from expert input in oncology and pain management, and ratings covered accuracy, completeness, relevance and clinical usefulness.

Reported statistics for the questions with differences ranged from p = 0.003 to p = 0.011. The abstract does not state which model scored higher on those questions.

On readability, ChatGPT-4 stayed within a more consistent range of reading levels. Gemini-2 showed wider variation, which the authors link to greater sensitivity to different kinds of prompts.

Sources

  1. Peer-reviewedWORK: A Journal of Prevention, Assessment & Rehabilitation2026-10-09

The debate