Emerging evidence
Published recordScienceGood Space
More
AI-accelerated meta-analysis in psychology: Large language models code study properties with high accuracy
- Truvace record
- TRV-2026-2755raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Benchmark or lab evaluation
- Sources
- 1 sourcePeer-reviewed
- sha256
- 9bb88c02312b…check it
Theoretically driven meta-analyses often involve testing whether study characteristics moderate an effect in a pattern that supports one of multiple competing theoretical accounts. To conduct such analyses, researchers must manually extract, i.e., code, these characteristics from sometimes hundreds of studies. Ideally, studies should be coded by two researchers independently to prevent errors and biases from compromising the dataset's integrity. The laborious nature of this task, however, means that meta-analyst…
The gain
In a test on 60 articles drawn from three published psychology meta-analyses, GPT-5 matched the original authors' study codes 93% of the time and Gemini 2.5 Pro 92% of the time, with Cohen's kappa of .75 to .96.
What this doesn’t fix
The test covered only three meta-analyses and two models, and agreement was measured against authors' codes that were often not double-coded and so may contain errors; the author says larger-scale studies are needed.
Sources
1 on file- Behavior Research Methodsdoi:10.3758/s13428-026-03020-1
Cite this record
v1Truvace Impact Record TRV-2026-2755, v1: “AI-accelerated meta-analysis in psychology: Large language models code study properties with high accuracy.” Truvace, 2026-11-12. /record/TRV-2026-2755 (accessed on the day you copy it). sha256 9bb88c02312b88c0…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2755 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace