aceModerate evidence
Published recordScienceThe Trace
More
Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews
- Truvace record
- TRV-2026-2818raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Observational study
- Sources
- 1 sourcePeer-reviewed
- sha256
- 1e224ae94108…check it
BACKGROUND: We evaluated the benefits and risks of using the Abstrackr machine learning (ML) tool to semi-automate title-abstract screening and explored whether Abstrackr's predictions varied by review or study-level characteristics. METHODS: For a convenience sample of 16 reviews for which adequate data were available to address our objectives (11 systematic reviews and 5 rapid reviews), we screened a 200-record training set in Abstrackr and downloaded the relevance (relevant or irrelevant) of the remaining rec…
58Gain side
How the two sides scoreGain scores higher
53Problem side
The gain
In a retrospective simulation on 16 evidence reviews, using Abstrackr's predictions with liberal-accelerated screening saved a median 26 hours of title-abstract screening per review (35% of records) compared with dual independent screening.
The problem
In the same 16-review simulation of Abstrackr, the tool wrongly excluded 0 to 3 studies per review (0 to 14% of those included), and 13% of 802 included studies were predicted irrelevant, which changed one meta-analysis from significant to non-significant.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
What this doesn’t fix
The 16 reviews were a convenience sample, the protocol was not published in advance, only one tool and one screening approach were tested, and only eight meta-analyses in one review were affected by missed studies, so the authors caution against generalising.
Sources
1 on file- Systematic Reviewsdoi:10.1186/s13643-020-01528-x
Cite this record
v1Truvace Impact Record TRV-2026-2818, v1: “Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews.” Truvace, 2026-11-12. /record/TRV-2026-2818 (accessed on the day you copy it). sha256 1e224ae94108cfdc…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2818 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.