aceModerate evidence
Published recordScienceGood Space
More
Evaluating large language models for title/abstract screening: a systematic review and meta-analysis & development of new tool
- Truvace record
- TRV-2026-2729raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- 856d55088855…check it
Background: Screening titles and abstracts for systematic reviews (SRs) is a labor-intensive and error-prone process. With the advent of large language models (LLMs), there is potential to automate and improve the efficiency of this process. This study aims to develop an LLM-based screening tool and compare its performance with existing LLM models in terms of sensitivity, specificity, and precision. Methods: A SR was conducted in August 2024, searching the MEDLINE, Embase, and Scopus databases for studies report…
The gain
A systematic review and meta-analysis of 14 LLM-based models for screening titles and abstracts found pooled sensitivity and specificity approaching 90%, with low overall risk of bias.
What this doesn’t fix
The authors' own tool had lower specificity (81%) and precision (14%) than its sensitivity, a trade-off they say requires careful consideration.
Sources
1 on file- Journal of Medical Artificial Intelligencedoi:10.21037/jmai-24-408
Cite this record
v1Truvace Impact Record TRV-2026-2729, v1: “Evaluating large language models for title/abstract screening: a systematic review and meta-analysis & development of new tool.” Truvace, 2026-11-12. /record/TRV-2026-2729 (accessed on the day you copy it). sha256 856d55088855668d…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2729 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.