aceClinical performance and practical limitations of large language models in respiratory medicine
This Journal club discusses three recent studies evaluating large language models (LLMs) in respiratory medicine and allergy. In COPD, ChatGPT-3.5 was rated higher than Bing Chat by respiratory specialists for accuracy, completeness, clarity and safety, although clinically important omissions remained. In asthma, 13 out of 14 models exceeded 85% accuracy on multiple-choice questions and five exceeded 95%, but performance was lower and more variable in objective structured clinical examinations. In allergen immun…
In asthma testing, 13 of 14 language models exceeded 85% accuracy on multiple-choice questions, and five exceeded 95%, showing strong performance on structured knowledge tasks.
Performance dropped and varied in clinical examination formats, and COPD answers had clinically important omissions, so the models are not ready for unsupervised patient counselling or autonomous clinical use.
The source is a journal club discussion of three studies, and it says weaknesses persist in completeness, transparency, readability and clinically nuanced application, so findings do not support unsupervised or autonomous clinical use.
Evidence
- Peer-reviewedBreathe2026-10-13
How should this claim be treated?
Truvace Impact Record TRV-2026-1397, v1: “Clinical performance and practical limitations of large language models in respiratory medicine.” Truvace, 2026-10-15. /record/TRV-2026-1397 (accessed at citation time). sha256 499778558052f314…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1397 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.