Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1515
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-16T01:09:22.368316Z
status: published
lens: trace
sector: health
headline: Evaluating the Reliability, Accuracy, and Readability of ChatGPT-3.5 and GPT-4 in Providing Patient Education Related to Glenohumeral Joint Osteoarthritis
dek: Objectives Chatbots have been increasingly recognized as modern tools that provide patients with reliable health-related information. This study aimed to evaluate and compare ChatGPT-3.5 and GPT-4's ability to answer glenohumeral osteoarthritis-related questions. Methods Fifteen questions were derived from the 2020 AAOS Clinical Practice Guidelines for the Surgical Management of Glenohumeral Joint Osteoarthritis. Questions were categorized into three groups: risk factors, implant/intraoperative considerations, a…
gain_title: Both chatbots gave adequately well-informed answers to patient-style questions about glenohumeral osteoarthritis, as judged by four fellowship-trained shoulder surgeons.
problem_title: Both models scored lowest on evidence-based content, and GPT-3.5 answers were written at a harder reading level than GPT-4's.
trace_subject: Chatbot answers to patient questions about glenohumeral osteoarthritis
gain_reading: Both chatbots gave adequately well-informed answers to patient-style questions about glenohumeral osteoarthritis, as judged by four fellowship-trained shoulder surgeons.
gain_evidence: Both ChatGPT-3.5 and GPT-4 performed adequately in providing well-informed medical responses to patient queries about glenohumeral osteoarthritis.
problem_reading: Both models scored lowest on evidence-based content, and GPT-3.5 answers were written at a harder reading level than GPT-4's.
problem_evidence: Both models scored lowest on "Evidence-based" prompts. | indicating a more difficult reading level in GPT-3.5's responses
quick_read: Researchers wrote fifteen questions based on the 2020 AAOS guidelines for surgical management of glenohumeral osteoarthritis and put them to ChatGPT-3.5 and GPT-4. Four fellowship-trained shoulder and elbow surgeons then rated every response from 1 to 5 on five criteria, and readability was measured on the Flesch-Kincaid scale.

The findings matter because patients increasingly consult chatbots for health information, and this study suggests the answers are generally adequate but not strongly grounded in evidence. The abstract points to better literature-based content and more accessible wording as needs for future versions; it does not report how patients themselves would use or understand the answers.
limitation: The study used only fifteen guideline-derived questions on one condition, and the abstract reports no statistically significant differences between prompt themes, which limits how far the results generalize.
tag: Dual reading
key_points: Fifteen questions were drawn from the 2020 AAOS Clinical Practice Guidelines and grouped into risk factors, implant/intraoperative considerations, and pain/functional outcomes. | ChatGPT-3.5 averaged 19.7/25, with risk-factor prompts scoring highest. | GPT-4 averaged 18.7/25, with functional-outcome prompts scoring highest. | Neither model showed statistically significant differences between prompt themes. | Flesch-Kincaid scores were 18.3 for GPT-3.5 and 15.4 for GPT-4, so GPT-3.5 was harder to read.
rundown: Each response was rated by four surgeons for relevance, accuracy, clarity, completeness, and evidence-based support, giving a maximum of 25 points per question. Data were analyzed descriptively and statistically to compare the two models.

Strengths differed by model: clarity was the highest-scoring category for GPT-3.5, while relevance was highest for GPT-4. Evidence-based support was the lowest-scoring category for both.

The authors conclude that future chatbot versions should draw on systematic and reliable literature reviews and present content in an accessible, readable way.
sources:
- peer_reviewed | Rhode Island medical journal (2013) | https://pubmed.ncbi.nlm.nih.gov/42789808/ | 2026-10-01
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

54734056ecc395b97eea1b06257d229f1717e8f054468586aad461a20942a23b

previous

0000000000000000000000000000000000000000000000000000000000000000