48/100

Emerging evidence

GainPublished recordScienceGood Space

AI-accelerated meta-analysis in psychology: Large language models code study properties with high accuracy

Truvace record
TRV-2026-2755raw data (JSON)
Version
v1Published
First published
Evidence type
Benchmark or lab evaluation
Sources
1 sourcePeer-reviewed
sha256
9bb88c02312b…check it

Theoretically driven meta-analyses often involve testing whether study characteristics moderate an effect in a pattern that supports one of multiple competing theoretical accounts. To conduct such analyses, researchers must manually extract, i.e., code, these characteristics from sometimes hundreds of studies. Ideally, studies should be coded by two researchers independently to prevent errors and biases from compromising the dataset's integrity. The laborious nature of this task, however, means that meta-analyst…

The gain

In a test on 60 articles drawn from three published psychology meta-analyses, GPT-5 matched the original authors' study codes 93% of the time and Gemini 2.5 Pro 92% of the time, with Cohen's kappa of .75 to .96.

What this doesn’t fix

The test covered only three meta-analyses and two models, and agreement was measured against authors' codes that were often not double-coded and so may contain errors; the author says larger-scale studies are needed.

Sources

1 on file
  1. Behavior Research Methodsdoi:10.3758/s13428-026-03020-1
    Peer-reviewed2026-04-27

Cite this record

v1

Truvace Impact Record TRV-2026-2755, v1: “AI-accelerated meta-analysis in psychology: Large language models code study properties with high accuracy.” Truvace, 2026-11-12. /record/TRV-2026-2755 (accessed on the day you copy it). sha256 9bb88c02312b88c0…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv19bb88c02312b…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2755 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.