Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1566
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-16T10:41:53.654209Z
status: published
lens: p_space
sector: health
headline: Accuracy, completeness and hallucinations in large language model generated statements and references for common rhinologic disorders
dek: Large language models (LLMs) are increasingly adopted for medical documentation, clinical decision-making, and scientific writing. Although these applications demonstrated impressive capabilities, concerns persist regarding their accuracy, fabricated references, and ethical implications (1-3). The Massive Multitask Language Understanding (MMLU) test was introduced as a comprehensive benchmark to evaluate how well LLMs perform across a diverse range of tasks (4). This prospective comparative two part study evalua…
gain_title: (none)
problem_title: Concerns persist that LLMs used for medical documentation and clinical decisions may give inaccurate content and fabricated references, prompting a test on four common nose and sinus disorders.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: Concerns persist that LLMs used for medical documentation and clinical decisions may give inaccurate content and fabricated references, prompting a test on four common nose and sinus disorders.
problem_evidence: concerns persist regarding their accuracy, fabricated references, and ethical implications
quick_read: A prospective comparative two-part study asked international rhinology society members to rate ChatGPT-3.5 summaries of 100 words on epistaxis, allergic rhinitis, odontogenic sinusitis and chronic rhinosinusitis. The summaries covered background, investigations and management.

The second part checked the references produced by newer models for the same prompts. This matters because LLMs are being adopted in clinical and scientific work, yet the supplied text reports no findings, so how accurate or complete the outputs were remains unknown.
limitation: 
tag: Evidence-backed problem
key_points: The study had two parts: summary quality and reference quality. | The topics were epistaxis, allergic rhinitis, odontogenic sinusitis and chronic rhinosinusitis. | Reviewers were international rhinology society members, and otolaryngologists used standardized scales. | Reference tests covered CGPT-5 thinking and non-thinking, Claude Sonnet 4.5 non-thinking and Gemini 2.5.
rundown: The introduction says LLMs are increasingly adopted for medical documentation, clinical decision-making and scientific writing. It says that despite impressive capabilities, worries remain about accuracy, fabricated references and ethics.

The summary task used ChatGPT-3.5 to generate 100-word statements for each of four disorders. Experts assessed accuracy and completeness of those statements.

A separate evaluation examined references from contemporary model versions given the same prompts. Otolaryngologists rated overall reference quality with standardized scales. The text also mentions the MMLU benchmark as a broad test of LLM performance across tasks.
sources:
- peer_reviewed | Rhinology Journal | https://doi.org/10.4193/rhin26.460 | 2026-10-14
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

620f7febd9e50df5d4fe3ff4a7f0e5add130c1717c14d6125a583aff91df12c5

previous

0000000000000000000000000000000000000000000000000000000000000000