Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1911
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-19T18:12:32.097457Z
status: published
lens: trace
sector: health
headline: Evaluating COPD care assistance: a comparative study of ChatGPT-3.5, ChatGPT-4.0, and ERNIE bot performance
dek: Background: Large language model (LLM) chatbots are increasingly used to provide health information, yet their reliability in addressing chronic obstructive pulmonary disease (COPD) care questions in the Chinese context remains unclear. To evaluate and compare the performance of three publicly available LLM chatbots - ChatGPT-3.5, ChatGPT-4.0 (OpenAI), and ERNIE Bot (Baidu) - in answering chronic obstructive pulmonary disease (COPD) care questions in China.
Methods: We performed a cross-sectional comparative eva…
gain_title: Most chatbot answers to Chinese-language COPD care questions were rated accurate by respiratory physicians, suggesting a role in basic patient education.
problem_title: Responses from the same chatbots lacked comprehensiveness and concrete, patient-executable steps, so they are unsuitable as standalone clinical advice.
trace_subject: LLM chatbot answers to COPD patient education and self-management questions in Chinese
gain_reading: Most chatbot answers to Chinese-language COPD care questions were rated accurate by respiratory physicians, suggesting a role in basic patient education.
gain_evidence: LLM chatbots may support basic COPD patient education in Chinese settings
problem_reading: Responses from the same chatbots lacked comprehensiveness and concrete, patient-executable steps, so they are unsuitable as standalone clinical advice.
problem_evidence: PEMAT actionability was limited, indicating that few responses included concrete, patient-executable steps.
quick_read: Researchers tested ChatGPT-3.5, ChatGPT-4.0 and Baidu's ERNIE Bot on 40 COPD care questions asked in Chinese. Three consultant respiratory physicians rated the answers. Most were judged accurate, and ERNIE Bot scored higher on accuracy than both ChatGPT versions in this Chinese-language setting.

The results suggest chatbots could assist with basic patient education, but accuracy alone does not make advice useful. Answers were thin and gave few concrete steps patients could follow. The authors call for safeguards and clinician oversight, and the study cannot show how patients would actually use or be affected by these answers.
limitation: The evaluation was a cross-sectional test of 40 questions in Chinese, and the ERNIE Bot advantage was described only for this Chinese-language evaluation, so results may not generalise to other languages or real patient use.
tag: Dual reading
key_points: The study used 40 COPD care questions across 7 prespecified domains tied to patient education and self-management. | Each question was submitted in Chinese in a new chat session to limit context carryover. | Three consultant-level respiratory physicians independently rated accuracy on a 1 to 3 scale. | ERNIE Bot had higher total accuracy scores and a higher share of good ratings than both ChatGPT versions. | Comprehensiveness for good responses averaged about 1.4 to 2.0 out of 5, and PEMAT actionability was limited.
rundown: This was a cross-sectional comparative evaluation of three publicly available chatbots: ChatGPT-3.5 and ChatGPT-4.0 from OpenAI and ERNIE Bot from Baidu. It was motivated by uncertainty over how reliable chatbots are on COPD care questions in the Chinese context.

Accuracy was summed across three raters on a 3 to 9 scale, with a majority-vote category derived. Responses rated good were also scored for comprehensiveness on a five-point scale, and information quality was assessed with a modified DISCERN framework and the PEMAT understandability and actionability domains.

The authors conclude the tools should not serve as standalone clinical advice and need appropriate safeguards and clinician oversight.
sources:
- peer_reviewed | Journal of Global Health | https://doi.org/10.7189/jogh.16.04297 | 2026-10-19
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

763c90be4545b8a4ff1a3bea64b7b332854ebc13c87745330c06b2a1e88ae84b

previous

0000000000000000000000000000000000000000000000000000000000000000