Moderate evidence
Published recordHealthProblem Space
More
Impact of Prompt Formulation and Case Complexity on Large Language Model Performance in Otolaryngology
- Truvace record
- TRV-2026-2971raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Observational study
- Sources
- 1 sourcePeer-reviewed
- sha256
- 8e9183d281b1…check it
Objective: To assess how prompt formulation and clinical complexity influence the performance of large language models (LLMs) in real otolaryngology cases. Study design: Comparative observational study. Setting: University hospital study. Methods: Clinical information from 80 consecutive outpatients prospectively recruited at two otolaryngology centers and from 40 published case reports was submitted to ChatGPT-5, Claude-Sonnet-4.5, and Gemini-2.5-Pro using two prompt strategies: a physician-oriented prompt with…
The problem
In a comparative study of 120 ear, nose and throat cases, ChatGPT-5, Claude-Sonnet-4.5 and Gemini-2.5-Pro reached the primary diagnosis in 28.7% of complex cases versus 54.6% of moderate ones, and lay-language patient prompts were less accurate than physician prompts.
What this doesn’t fix
Case complexity was graded only in the published case reports, and inter-rater reliability was checked only on a subset of complex, practitioner-prompt cases.
Sources
1 on file- Otolaryngology–Head and Neck Surgerydoi:10.1002/ohn.70524
Cite this record
v1Truvace Impact Record TRV-2026-2971, v1: “Impact of Prompt Formulation and Case Complexity on Large Language Model Performance in Otolaryngology.” Truvace, 2026-11-16. /record/TRV-2026-2971 (accessed on the day you copy it). sha256 8e9183d281b1e5a0…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2971 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace