52/100

Moderate evidence

Published recordHealthProblem Space (a documented problem)

Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study

Truvace record
TRV-2026-2452raw data (JSON)
Version
v1Published
First published
Evidence type
Benchmark or lab evaluation
Sources
1 sourcePeer-reviewed
sha256
417bbf122271…check it

Background: Large language models (LLMs) are increasingly used for health information, but chronic obstructive pulmonary disease (COPD)-related questions often involve safety-sensitive decisions, including treatment escalation, oxygen therapy, and exacerbation management. Because generative outputs may vary across repeated prompts, response stability is also relevant to comparative evaluation. Objective: This study aims to compare 6 public-facing LLM chatbots in answering COPD-related public health questions acr…

The problem

In a test of 55 COPD questions, every one of six public chatbots produced unsafe answers, from 7.3% of responses for ChatGPT, Gemini, Copilot and DeepSeek to 10.9% for Doubao.

What this doesn’t fix

The study used a standardized single-turn English question set, and the first-round safety test neither established a between-model difference nor demonstrated equivalence.

Sources

1 on file
  1. JMIR Medical Informaticsdoi:10.2196/104836
    Peer-reviewed2026-11-09

Cite this record

v1

Truvace Impact Record TRV-2026-2452, v1: “Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study.” Truvace, 2026-11-10. /record/TRV-2026-2452 (accessed on the day you copy it). sha256 417bbf1222716c7a…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv1417bbf122271…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2452 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.