Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-2120
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-21T18:14:43.411945Z
status: published
lens: trace
sector: health
headline: Evaluator-Masked Assessment of 4 Consumer-Facing Large Language Models on Expert Anterior Segment Ophthalmology Questions
dek: Objective: To compare expert-rated quality and safety of responses from four contemporary large language models (LLMs)-ChatGPT, Gemini, Claude, and Grok-on expert-level anterior segment ophthalmology questions.
Design: Cross-sectional, masked, randomized-display-order, multievaluator comparative study.
Participants: Eleven fellowship-trained cornea and anterior segment subspecialists.
Methods: Forty-five expert-level questions spanning six anterior segment domains were administered to each LLM under zero-shot co…
gain_title: Eleven masked cornea specialists rated Claude and Grok answers to anterior segment questions as higher quality than ChatGPT and Gemini on accuracy, completeness and clarity.
problem_title: A larger share of questions drew at least one severe overall-safety rating for Claude and Grok than for ChatGPT and Gemini, even though mean safety scores did not differ significantly.
trace_subject: Expert-rated quality and safety of consumer LLM answers to anterior segment ophthalmology questions
gain_reading: Eleven masked cornea specialists rated Claude and Grok answers to anterior segment questions as higher quality than ChatGPT and Gemini on accuracy, completeness and clarity.
gain_evidence: Claude and Grok were each rated more favorably than ChatGPT and Gemini
problem_reading: A larger share of questions drew at least one severe overall-safety rating for Claude and Grok than for ChatGPT and Gemini, even though mean safety scores did not differ significantly.
problem_evidence: more questions elicited ≥ 1 severe overall-safety rating for Claude (35.6%) and Grok (33.3%) than for ChatGPT (6.7%) or Gemini (11.1%)
quick_read: Eleven fellowship-trained cornea and anterior segment specialists rated answers from ChatGPT, Gemini, Claude and Grok to 45 expert-level questions. Responses were masked to model identity and shown in random order, and each was scored on accuracy, completeness, clarity, safety and potential for harm.

The results show that the models which looked best on expert-rated quality also showed a heavier tail of severe safety concerns. That pattern could matter for anyone relying on chatbot answers for eye care, but the authors describe it as hypothesis-generating, and it reflects expert preference at one time point rather than verified clinical correctness.
limitation: The findings reflect expert preference rather than validated clinical correctness, were taken at a single time point without a reference answer key, and the severe-safety signal was a post hoc, hypothesis-generating result.
tag: Dual reading
key_points: The study covered 45 questions across six anterior segment domains, each given to four LLMs and rated by 11 subspecialists, for 9900 complete ratings. | Quality differences were significant on all three quality metrics, with Claude and Grok rated above ChatGPT and Gemini. | No statistically significant between-model difference was found in mean safety scores. | Severe-safety flags appeared for 35.6% of questions with Claude and 33.3% with Grok, versus 6.7% for ChatGPT and 11.1% for Gemini, and the pattern persisted when two evaluators had to agree.
rundown: Questions were given under zero-shot conditions in a single prespecified session on January 19, 2026, with memory, browsing and retrieval disabled. No specialist-authored reference answer key was used, so all metrics reflect expert perception.

Evaluators scored five 5-point Likert metrics. The analyses included Friedman tests with Kendall's W, Bonferroni-adjusted pairwise Wilcoxon tests, length-adjusted mixed models and a masking-integrity diagnostic.

The severe-safety comparison was a post hoc analysis (Cochran Q = 23.4, P < 0.001). Inter-rater agreement on individual ratings was low (Krippendorff's α 0.021-0.168), while the 11-evaluator mean showed moderate reliability (ICC(2,k) 0.58-0.69 for quality metrics).
sources:
- peer_reviewed | Ophthalmology Science | https://doi.org/10.1016/j.xops.2026.101398 | 2026-09-04
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

fe1c54d5225f2861c6fe3a617ba7e787f0b76685dc73d992a12fc8b7be8f0a26

previous

0000000000000000000000000000000000000000000000000000000000000000