Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1397
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-15T01:10:18.127026Z
status: published
lens: trace
sector: health
headline: Clinical performance and practical limitations of large language models in respiratory medicine
dek: This Journal club discusses three recent studies evaluating large language models (LLMs) in respiratory medicine and allergy. In COPD, ChatGPT-3.5 was rated higher than Bing Chat by respiratory specialists for accuracy, completeness, clarity and safety, although clinically important omissions remained. In asthma, 13 out of 14 models exceeded 85% accuracy on multiple-choice questions and five exceeded 95%, but performance was lower and more variable in objective structured clinical examinations. In allergen immun…
gain_title: In asthma testing, 13 of 14 language models exceeded 85% accuracy on multiple-choice questions, and five exceeded 95%, showing strong performance on structured knowledge tasks.
problem_title: Performance dropped and varied in clinical examination formats, and COPD answers had clinically important omissions, so the models are not ready for unsupervised patient counselling or autonomous clinical use.
trace_subject: Large language model performance in respiratory medicine and allergy
gain_reading: In asthma testing, 13 of 14 language models exceeded 85% accuracy on multiple-choice questions, and five exceeded 95%, showing strong performance on structured knowledge tasks.
gain_evidence: 13 out of 14 models exceeded 85% accuracy on multiple-choice questions and five exceeded 95%
problem_reading: Performance dropped and varied in clinical examination formats, and COPD answers had clinically important omissions, so the models are not ready for unsupervised patient counselling or autonomous clinical use.
problem_evidence: clinically important omissions remained | performance was lower and more variable in objective structured clinical examinations | not unsupervised patient counselling or autonomous clinical deployment
quick_read: A journal club reviewed three studies that tested large language models on respiratory and allergy tasks. Specialists rated ChatGPT-3.5 above Bing Chat for COPD answers. Many models did very well on asthma multiple-choice questions, and ChatGPT-4 got a fair quality score for allergen immunotherapy information.

The pattern matters because it separates where these tools help from where they could mislead. Structured knowledge tasks suit them, but gaps in completeness, transparency and readability remain. The source supports supervised educational use only, and it leaves open how the models would behave with real patients or in autonomous deployment.
limitation: The source is a journal club discussion of three studies, and it says weaknesses persist in completeness, transparency, readability and clinically nuanced application, so findings do not support unsupervised or autonomous clinical use.
tag: Dual reading
key_points: Respiratory specialists rated ChatGPT-3.5 higher than Bing Chat on accuracy, completeness, clarity and safety in COPD, though important omissions persisted. | In asthma, 13 of 14 models exceeded 85% on multiple-choice questions and five exceeded 95%. | Asthma performance was lower and more variable in objective structured clinical examinations. | ChatGPT-4 on allergen immunotherapy had fair overall DISCERN quality but insufficient JAMA benchmark reliability, and its answers were often hard to read.
rundown: The COPD study compared two chatbots, with respiratory specialists rating ChatGPT-3.5 above Bing Chat across accuracy, completeness, clarity and safety. Even the better-rated tool left out clinically important information.

The asthma work tested 14 models. Most scored very high on multiple-choice questions, but results fell and varied more when the format moved to objective structured clinical examinations, which are closer to clinical application.

For allergen immunotherapy, ChatGPT-4 reached fair overall quality on the DISCERN instrument but did not meet the JAMA benchmark for reliability. Its responses were often difficult to read, which limits usefulness for patient information.

The authors conclude that current evidence supports supervised educational use in respiratory medicine and allergy. It does not support unsupervised patient counselling or autonomous clinical deployment.
sources:
- peer_reviewed | Breathe | https://doi.org/10.1183/20734735.0125-2026 | 2026-10-13
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

499778558052f3146632dffaf40dc1a7b7c61bd4d1f5520fddc5c80912bb6531

previous

0000000000000000000000000000000000000000000000000000000000000000