The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2729
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T17:15:48.362911Z
status: published
lens: g_space
sector: science
headline: Evaluating large language models for title/abstract screening: a systematic review and meta-analysis & development of new tool
dek: Background: Screening titles and abstracts for systematic reviews (SRs) is a labor-intensive and error-prone process. With the advent of large language models (LLMs), there is potential to automate and improve the efficiency of this process. This study aims to develop an LLM-based screening tool and compare its performance with existing LLM models in terms of sensitivity, specificity, and precision. Methods: A SR was conducted in August 2024, searching the MEDLINE, Embase, and Scopus databases for studies report…
gain_title: A systematic review and meta-analysis of 14 LLM-based models for screening titles and abstracts found pooled sensitivity and specificity approaching 90%, with low overall risk of bias.
problem_title: (none)
trace_subject: (none)
gain_reading: A systematic review and meta-analysis of 14 LLM-based models for screening titles and abstracts found pooled sensitivity and specificity approaching 90%, with low overall risk of bias.
gain_evidence: The SROC analysis indicated overall excellent performance with sensitivity and specificity approaching 90%, with overall low risk of bias.
problem_reading: (none)
problem_evidence: (none)
quick_read: A systematic review searched MEDLINE, Embase and Scopus in August 2024 for studies reporting how well large language models screen titles and abstracts for systematic reviews. It identified 14 LLM-based models, and pooled results showed sensitivity and specificity approaching 90%.

The authors also built their own screening tool on GPT-4o-mini and tested it on existing systematic reviews. It caught every relevant record (100% sensitivity) but had lower specificity (81%) and precision of 14%, and processed each entry in 1.7 seconds at minimal cost.

Screening is a labour-intensive, error-prone step in producing systematic reviews, so faster automated screening could cut the workload for research teams. The authors say the trade-off in specificity needs careful consideration before such tools are used.
limitation: The authors' own tool had lower specificity (81%) and precision (14%) than its sensitivity, a trade-off they say requires careful consideration.
tag: Evidence-backed gain
key_points: The review searched MEDLINE, Embase and Scopus in August 2024 and followed PRISMA guidelines. | It identified 14 LLM-based screening models whose performance varied. | Pooled sensitivity and specificity approached 90%, with overall low risk of bias on the QUADAS-2 tool. | The authors' GPT-4o-mini tool reached 100% sensitivity, 81% specificity and 14% precision. | The tool processed entries at 1.7 seconds each with minimal cost.
rundown: Studies were included only if they reported enough data to build a confusion matrix; those requiring human intervention were excluded. Performance metrics were pooled with forest plots and summary receiver operating characteristic curves, and risk of bias was assessed with QUADAS-2.

The authors' own tool, built on GPT-4o-mini and tested against existing systematic reviews, missed no relevant records. Its precision of 14% and specificity of 81% mean many irrelevant records would still be passed on for human review.

The authors conclude that LLM-based models are a promising, cost-effective alternative to manual screening. They say integrating them could improve efficiency while maintaining reliability in academic research.
sources:
- peer_reviewed | Journal of Medical Artificial Intelligence | https://doi.org/10.21037/jmai-24-408 | 2025-05-12
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

856d55088855668dcec1f7ccaa253412a06a4d2ee4220a75d50b3e7a1e9fa9e8

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000