TRV-2026-1274Version 2 · Revised

Saved · see the current record

Why this version exists

Editorial pass: sharper headline; gain/problem restated as concrete outcomes; refiled trace to g_space (gain outcome, problem absent).

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-1274
version: 2
kind: revised
reason: Editorial pass: sharper headline; gain/problem restated as concrete outcomes; refiled trace to g_space (gain outcome, problem absent).
timestamp: 2026-11-04T09:55:38.254487Z
status: published
lens: g_space
sector: science
headline: Bias and Reliability of AI-Based Peer Review: A Comparative Study of ChatGPT and Claude Evaluating Scientific Abstracts
dek: Background The use of artificial intelligence (AI) models as reviewers of scientific content raises concerns about potential biases related to author identity and about the reproducibility of their evaluations. We assessed whether AI-based reviewers exhibit gender or geographic bias and evaluated the reproducibility of their scoring of scientific abstracts. Methods We randomly selected 10 general internal medicine journals indexed in the Journal Citation Reports (impact factor ≥ 1.5). For each journal, five orig…
gain_title: Across 800 evaluations of 50 medical abstracts under four fictional author identities, ChatGPT and Claude showed no consistent gender or geographic bias and scored with high reproducibility.
problem_title: (none)
trace_subject: (none)
gain_reading: Across 800 evaluations of 50 medical abstracts under four fictional author identities, ChatGPT and Claude showed no consistent gender or geographic bias and scored with high reproducibility.
gain_evidence: no consistent evidence of gender or geographic bias
problem_reading: (none)
problem_evidence: (none)
quick_read: In April 2026, researchers tested ChatGPT and Claude as peer reviewers by having each model score 50 general internal medicine abstracts twice under four fictional author identities, producing 800 evaluations of quality, novelty, and acceptance on a 0-10 scale.

By the October 2026 publication date, the study reported no consistent gender or geographic bias and high reproducibility, but noted that narrow score distributions may have inflated agreement and that comparison to human peer review validity was still needed.
limitation: High agreement may be inflated by narrow scoring, and validity compared to human peer review remains untested.
tag: Evidence-backed gain
key_points: 50 original research abstracts from 10 general internal medicine journals were each assigned to four fictional identities: African female, African male, American female, American male. | Each abstract was evaluated twice by ChatGPT and Claude on quality, novelty, and acceptance on a 0-10 scale, yielding 800 evaluations in April 2026. | ChatGPT quality and novelty scores were identical across identities with median 7 [1] and 5 [2], and Claude scores were identical across identities for all three dimensions.
rundown: The authors retrieved five original research abstracts from each of 10 general internal medicine journals indexed in Journal Citation Reports with impact factor ≥ 1.5, and created four fictional author identities per abstract. Bias was tested with multivariable ordinal logistic regression, and reproducibility with percent agreement and Fleiss' kappa.

Results showed median quality 7 [1], novelty 5 [2], and acceptance 6-7 [1-2] with minimal variation across identities, and multivariable analyses showed no overall association except for acceptance for Claude at the global level with no individual comparisons statistically significant.
sources:
- peer_reviewed | Journal of General Internal Medicine | https://doi.org/10.1007/s11606-026-10806-8 | 2026-10-02
prev: bc6be99179c00fc13fdf7420c91dbb6ac835ea110f3e8742dbf618ecb6a3ca59
SHA-256 hash
ba629b82503547937e59034833137d27b299cbf59bff0728c8e4b54b080556d6
Previous version’s hash
bc6be99179c00fc13fdf7420c91dbb6ac835ea110f3e8742dbf618ecb6a3ca59
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-1274 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.