Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1447
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-15T10:43:24.176648Z
status: published
lens: p_space
sector: science
headline: Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study
dek: Objectives Increasing manuscript volume has strained peer review, prompting interest in artificial intelligence (AI) as a supportive tool. We evaluated DeepSeek R1 as a structured peer reviewer of neurosurgical case-based manuscripts and compared its scoring patterns with those of an experienced editor and student reviewers. Design Exploratory, cross-sectional comparative study using a convenience sample of consecutively submitted manuscripts. Setting Single skull-base neurosurgical journal during a 6-month subm…
gain_title: (none)
problem_title: In a pilot of 11 neurosurgical manuscripts, DeepSeek R1 gave systematically higher subjective ratings than human reviewers, notably for Novelty, which limits its reliability as an unsupervised peer reviewer.
trace_subject: (none)
gain_reading: (none)
gain_evidence: (none)
problem_reading: In a pilot of 11 neurosurgical manuscripts, DeepSeek R1 gave systematically higher subjective ratings than human reviewers, notably for Novelty, which limits its reliability as an unsupervised peer reviewer.
problem_evidence: systematically inflated subjective ratings, particularly Novelty | DeepSeek scored significantly higher than the Experienced Reviewer for Study Objective ( p = 0.015) and higher than both human groups for Novelty
quick_read: A skull-base neurosurgical journal tested DeepSeek R1 as a structured peer reviewer. Over a six-month window in 2025, the model, the journal's Editor-in-Chief and 154 first-year medical students each rated 11 de-identified, unpublished case-based manuscripts on a 3-point scale across five domains.

The study matters because rising manuscript volume strains peer review, and AI is being considered as support. The pattern of inflated subjective scores suggests the tool cannot replace expert judgment. The small sample and exploratory design leave open how the findings generalize to other journals or models.
limitation: The study is exploratory and pilot-scale, using a convenience sample of 11 manuscripts from a single journal and one experienced reviewer, so conclusions about wider use are tentative.
tag: Evidence-backed problem
key_points: Eleven manuscripts were assessed by three reviewer groups: 154 first-year medical students, the Editor-in-Chief, and DeepSeek R1. | Reviewers scored Study Objective, Novelty, Study Design, Thoroughness and Clarity on a 3-point scale. | Significant between-group differences appeared in four of five domains; Study Design did not differ (p = 0.483). | DeepSeek rated Novelty higher than both human groups and Study Objective higher than the experienced editor. | The authors propose a human-in-the-loop model combining AI efficiency with expert judgment.
rundown: The design was an exploratory, cross-sectional comparison using a convenience sample of consecutively submitted manuscripts at a single skull-base neurosurgical journal between January and June 2025. Each student was randomly assigned one manuscript after a course in biostatistics, epidemiology and critical appraisal.

Kruskal-Wallis style statistics showed group differences for Study Objective (H = 8.31, p = 0.02), Novelty (H = 9.74, p = 0.008), Thoroughness (H = 10.49, p = 0.005) and Clarity (H = 8.13, p = 0.02). Study Design scoring was statistically similar across groups.

The authors describe DeepSeek as consistent in structured domains but prone to inflating subjective ones, and suggest a human-in-the-loop approach to peer review.
sources:
- peer_reviewed | Journal of Neurological Surgery Reports | https://doi.org/10.1055/a-2964-9336 | 2026-10-09
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

ace0f521e95b1ec4211d16aac1db690d811b633caf71b2caad14581933d53336

previous

0000000000000000000000000000000000000000000000000000000000000000