aceEvaluating prompting strategies and large language models in systematic literature review screening: Relevance and task-stage classification
This study quantifies how prompting strategies interact with large language models (LLMs) to automate the screening stage of systematic literature reviews (SLRs). We evaluate six LLMs (GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku, and Llama-4-Maverick) under five prompt types (zero-shot, few-shot, chain-of-thought [CoT], CoT-few-shot, and self-reflection) across relevance classification and six Level-2 tasks, using accuracy, precision, recall, and F1. Results show pronounced model-pr…
Some model-prompt pairings screen abstracts effectively at low cost, with zero-shot maximizing recall and CoT-few-shot giving stronger precision and F1, while GPT-4o-mini stays cheap.
Self-reflection prompting performed poorly because it was over-inclusive and unstable across models, so results are uneven and only validated on one corpus.
Results come from a single SLR-automation corpus, and the authors state further validation is needed before generalising to broader systematic review screening tasks.
Evidence
- Peer-reviewedResearch Synthesis Methods2026-10-19
How should this claim be treated?
Truvace Impact Record TRV-2026-1935, v1: “Evaluating prompting strategies and large language models in systematic literature review screening: Relevance and task-stage classification.” Truvace, 2026-10-19. /record/TRV-2026-1935 (accessed at citation time). sha256 52f97486c0948cc9…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1935 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.