Truace
Subscribe
52/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

All 6 AI chatbots gave some unsafe COPD advice, a 55-question test finds

Study title: Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study

Evidence type
Benchmark or lab evaluation
Sample
n = 330
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2452cite or check it

Study title: Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study

A comparison of six public-facing chatbots answering 55 questions on chronic obstructive pulmonary disease (COPD) found unsafe responses in every model. In the first round of 330 responses, ChatGPT, Gemini, Copilot and DeepSeek each gave 4 unsafe answers (7.3%), Qwen gave 5 (9.1%) and Doubao gave 6 (10.9%).

Read the study abstract

Background: Large language models (LLMs) are increasingly used for health information, but chronic obstructive pulmonary disease (COPD)-related questions often involve safety-sensitive decisions, including treatment escalation, oxygen therapy, and exacerbation management. Because generative outputs may vary across repeated prompts, response stability is also relevant to comparative evaluation. Objective: This study aims to compare 6 public-facing LLM chatbots in answering COPD-related public health questions acr…

The problem

In a test of 55 COPD questions, every one of six public chatbots produced unsafe answers, from 7.3% of responses for ChatGPT, Gemini, Copilot and DeepSeek to 10.9% for Doubao.

In brief

Five blinded raters scored the answers against a GOLD 2026-centered reference standard. The difference in safety between models was not statistically significant, and the authors say that should not be read as evidence the models are equally safe. Accuracy, empathy, information quality, source transparency and readability did differ significantly across models.

A second round run several months later showed mostly stable results, but 9 safety classifications changed: 5 went from safe to unsafe and 4 from unsafe to safe. The authors conclude the chatbots may suit supplementary COPD education, with clinician oversight for individual treatment decisions.

Main points

  1. Researchers built a 55-question English COPD question bank from an initial pool of 108 questions, using web retrieval, Google Trends, PubMed-informed review, patient education resources and expert screening.
  2. Six public chatbots were tested: ChatGPT, Gemini, Copilot, DeepSeek, Qwen and Doubao, producing 330 first-round responses.
  3. Unsafe answers by model: 4 each (7.3%) for ChatGPT, Gemini, Copilot and DeepSeek, 5 (9.1%) for Qwen and 6 (10.9%) for Doubao.
  4. The overall safety comparison was not significant (Cochran Q=1.909, df=5; P=.86), which the authors say does not show equivalent safety.
  5. Across 330 matched question-model pairs, safety classifications agreed in 97.3% of pairs (κ=0.821, 95% CI 0.543-0.979).
  6. Nine safety classifications changed between rounds: 5 from safe to unsafe and 4 from unsafe to safe.
  7. Stability of accuracy and empathy ratings was high (weighted κ 0.872 and 0.856), as was DISCERN (ICC 0.954) and adapted EQIP (ICC 0.925).

The full story

The questions covered safety-sensitive COPD decisions such as treatment escalation, oxygen therapy and exacerbation management. Five blinded raters scored manually against a GOLD 2026-centered reference standard, and six readability indices were calculated for each answer.

The first round was the primary analysis. A second independent round, added after first-round data collection and run several months later, served as a matched retest sensitivity analysis using the same 55 questions.

Stability was measured with exact agreement and Cohen κ for safety, quadratic-weighted κ for ordinal ratings and intraclass correlation coefficients for continuous scores, with bootstrap confidence intervals. Most readability indices were also highly stable overall, though some models varied more than others.

Talk it overFour questions for a class, a team or the dinner table

  1. Who carries this cost, and what would have to change to prevent it?

    The problem In a test of 55 COPD questions, every one of six public chatbots produced unsafe answers, from 7.3% of responses for ChatGPT, Gemini, Copilot and DeepSeek to 10.9% for Doubao.

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a benchmark or lab test with a sample of 330, from JMIR Medical Informatics.

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The study used a standardized single-turn English question set, and the first-round safety test neither established a between-model difference nor demonstrated equivalence.

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

The questions are the same on every record. The details under them are taken from this record as written.

Sources

1 on file
  1. JMIR Medical Informaticsdoi:10.2196/104836
    Peer-reviewed2026-11-09

The debate

More from Health