The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2755
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T17:19:41.910032Z
status: published
lens: g_space
sector: science
headline: AI-accelerated meta-analysis in psychology: Large language models code study properties with high accuracy
dek: Theoretically driven meta-analyses often involve testing whether study characteristics moderate an effect in a pattern that supports one of multiple competing theoretical accounts. To conduct such analyses, researchers must manually extract, i.e., code, these characteristics from sometimes hundreds of studies. Ideally, studies should be coded by two researchers independently to prevent errors and biases from compromising the dataset's integrity. The laborious nature of this task, however, means that meta-analyst…
gain_title: In a test on 60 articles drawn from three published psychology meta-analyses, GPT-5 matched the original authors' study codes 93% of the time and Gemini 2.5 Pro 92% of the time, with Cohen's kappa of .75 to .96.
problem_title: (none)
trace_subject: (none)
gain_reading: In a test on 60 articles drawn from three published psychology meta-analyses, GPT-5 matched the original authors' study codes 93% of the time and Gemini 2.5 Pro 92% of the time, with Cohen's kappa of .75 to .96.
gain_evidence: GPT-5’s overall accuracy was 93% (SD = 7, min = 70%, max = 100%) while Gemini 2.5 Pro achieved 92% | Both models replicated the study codes from three recently published meta-analyses in psychology with high accuracy (>92% overall)
problem_reading: (none)
problem_evidence: (none)
quick_read: A psychologist at the University of Münster tested whether two AI models could act as a second coder for meta-analyses, the step in which researchers pull study characteristics out of papers. GPT-5 matched the original authors' codes 93% of the time and Gemini 2.5 Pro 92% of the time, on moderators that could be coded from the articles themselves.

The test covered three Psychological Bulletin meta-analyses, on memory judgments, attitudes toward older men and women, and social class and prosociality. For each, 20 randomly chosen studies from different sources were coded. The models worked from a coding manual and the article text, and returned structured data rather than chat replies.

Meta-analysts often double-code only a share of their studies; the three original teams double-coded 20%, 27% and all of theirs. The author argues AI double-coding of every study could make datasets more reliable. Results come from only three meta-analyses and two models, and the author calls for larger studies.
limitation: The test covered only three meta-analyses and two models, and agreement was measured against authors' codes that were often not double-coded and so may contain errors; the author says larger-scale studies are needed.
tag: Evidence-backed gain
key_points: The author selected three meta-analyses from Psychological Bulletin Volume 161 after screening 18 articles; 20 randomly chosen studies from each, one per source, made up the coding dataset. | GPT-5 averaged 93% agreement with the authors' codes on Ingendahl et al.; excluding two continuous moderators raised this to 98%, versus 95% for Gemini 2.5 Pro. | On Shakeri and North, Gemini 2.5 Pro reached 89% agreement and GPT-5 90%; 62% of the 29 disagreements with the authors' codes were shared by both models. | Both models scored 93% on Wu et al., and agreement between the two models was 94%. | Cohen's kappa was almost perfect for Ingendahl et al. (.90 for Gemini, .96 for GPT-5) and Wu et al. (.86 and .84), and substantial for Shakeri and North (.75 and .78). | A second GPT-5 run about three months later agreed with the first 96% of the time (minimum 85%). | An earlier study found older models coded study properties with 69% accuracy overall, versus the 92% or higher found here.
rundown: Study coding in psychology often requires judgment calls, such as whether a measure captured prosocial behavior or only intention. The author notes that a simulation study found miscategorizing just 2% of moderator codes can substantially affect discovery rates.

Part of the disagreement came from missing information. For Ingendahl et al., some codes had come from raw data or from emailing the source authors, so the models could not recover them. For Shakeri and North, an initial round exposed ambiguities in the coding manual, so the author amended it and generated new codes from a second random sample.

The author reports that some disagreements reflected ambiguity rather than model error, for example the models coded an election as a non-work context where the original authors coded it as work. Agreement was lowest for Shakeri and North, who double-coded the fewest studies.

The study used GPT-5 and Gemini 2.5 Pro through programming packages rather than chat windows, with Gemini's temperature set to zero. The author says this avoids manual transfer errors, and shared the data and analysis scripts on the Open Science Framework.
sources:
- peer_reviewed | Behavior Research Methods | https://doi.org/10.3758/s13428-026-03020-1 | 2026-04-27
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

9bb88c02312b88c0cbf42a06ed70558b0eba603cefdad576008baf680e586836

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000