Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1830
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-18T18:12:42.949089Z
status: published
lens: trace
sector: science
headline: A focused PubMed artificial intelligence search program creates better literature reviews than general purpose artificial intelligence and is comparable to trainees in shoulder and elbow surgery
dek: Background: As the use of artificial intelligence (AI) expands into orthopedic surgery, the potential to use AI for review and summation of literature leaves researchers and clinicians questioning the quality of AI-driven literature reviews. This study aims to compare AI-generated literature reviews to those done by orthopedic surgery trainees.
Methods: ChatGPT-4o and The-Literature, a directed AI platform, were asked to produce a literature review on 5 shoulder and elbow-related research topics. The primary out…
gain_title: A directed AI search platform produced shoulder and elbow literature reviews rated comparably to orthopedic trainees and better than a general-purpose chatbot.
problem_title: A general-purpose large language model produced weaker reviews, with most of its cited references fabricated.
trace_subject: AI-generated literature reviews on shoulder and elbow surgery topics
gain_reading: A directed AI search platform produced shoulder and elbow literature reviews rated comparably to orthopedic trainees and better than a general-purpose chatbot.
gain_evidence: None of the trainee reviews were significantly better than The-Literature. | The-Literature's reviews were significantly better than ChatGPT-4 (P = .0412).
problem_reading: A general-purpose large language model produced weaker reviews, with most of its cited references fabricated.
problem_evidence: The majority (58.8%) of ChatGPT-4's references were fabricated.
quick_read: Researchers asked ChatGPT-4o and a directed AI platform called The-Literature to write literature reviews on five shoulder and elbow topics. Their output was compared with reviews by a medical student, a resident, and a fellow. Two fellowship-trained shoulder and elbow surgeons graded the blinded reviews on a 1-5 scale.

The results suggest that how an AI tool is built and scoped may matter for reliability in clinical research tasks. A restricted search-based program performed at trainee level, while an open chatbot did poorly because it invented sources. The study covers only five topics and two expert graders, so it is unclear how the findings extend to other fields or newer models.
limitation: The comparison rests on five topics, a small number of human reviewers, and two graders, and tested ChatGPT-4 rather than other models, which limits generalization.
tag: Dual reading
key_points: Average overall ratings were 2.90 for the medical student, 3.90 for the resident, 3.50 for the fellow, 2.30 for ChatGPT-4, and 3.40 for The-Literature. | The resident and fellow reviews were significantly better than ChatGPT-4, but the medical student review was not. | The-Literature was rated significantly better than ChatGPT-4 and no trainee review significantly outperformed it. | 58.8% of ChatGPT-4's references were fabricated, which the authors identify as the main source of its poor utility.
rundown: The study design used five shoulder and elbow research topics. Outputs from ChatGPT-4, The-Literature, a fourth-year medical student, a third-year orthopedic resident, and a sports medicine fellow were blinded before grading.

Graders scored evidence and analysis, conclusion, references, and overall quality. Overall ratings had notable variability, with a standard deviation of 1.16 for ChatGPT-4 compared with 0.53 for the fellow.

The authors conclude that a limited, directed AI search program was comparable to orthopedic surgery trainees and outperformed an open large language model, which had poor overall utility.
sources:
- peer_reviewed | JSES International | https://doi.org/10.1016/j.jseint.2026.101790 | 2026-09-04
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

04cb82f48f171c602c2d44aa6ebc262c0e911098a48459295ab1c8c41fd99faa

previous

0000000000000000000000000000000000000000000000000000000000000000