Moderate evidence
Published recordHealthProblem Space (a documented problem)
More
Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study
- Truvace record
- TRV-2026-2452raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Benchmark or lab evaluation
- Sources
- 1 sourcePeer-reviewed
- sha256
- 417bbf122271…check it
Background: Large language models (LLMs) are increasingly used for health information, but chronic obstructive pulmonary disease (COPD)-related questions often involve safety-sensitive decisions, including treatment escalation, oxygen therapy, and exacerbation management. Because generative outputs may vary across repeated prompts, response stability is also relevant to comparative evaluation. Objective: This study aims to compare 6 public-facing LLM chatbots in answering COPD-related public health questions acr…
The problem
In a test of 55 COPD questions, every one of six public chatbots produced unsafe answers, from 7.3% of responses for ChatGPT, Gemini, Copilot and DeepSeek to 10.9% for Doubao.
What this doesn’t fix
The study used a standardized single-turn English question set, and the first-round safety test neither established a between-model difference nor demonstrated equivalence.
Sources
1 on file- JMIR Medical Informaticsdoi:10.2196/104836
Cite this record
v1Truvace Impact Record TRV-2026-2452, v1: “Safety, Accuracy, Empathy, Information Quality, and Readability of Chronic Obstructive Pulmonary Disease Health Advice Generated by 6 Large Language Models: Cross-Sectional Comparative Evaluation Study.” Truvace, 2026-11-10. /record/TRV-2026-2452 (accessed on the day you copy it). sha256 417bbf1222716c7a…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2452 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace