Canonical text (the exact bytes fingerprinted)

TRUVACE RECORD VERSION
record: TRV-2026-1935
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-10-19T22:02:06.566944Z
status: published
lens: trace
sector: science
headline: Evaluating prompting strategies and large language models in systematic literature review screening: Relevance and task-stage classification
dek: This study quantifies how prompting strategies interact with large language models (LLMs) to automate the screening stage of systematic literature reviews (SLRs). We evaluate six LLMs (GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku, and Llama-4-Maverick) under five prompt types (zero-shot, few-shot, chain-of-thought [CoT], CoT-few-shot, and self-reflection) across relevance classification and six Level-2 tasks, using accuracy, precision, recall, and F1. Results show pronounced model-pr…
gain_title: Some model-prompt pairings screen abstracts effectively at low cost, with zero-shot maximizing recall and CoT-few-shot giving stronger precision and F1, while GPT-4o-mini stays cheap.
problem_title: Self-reflection prompting performed poorly because it was over-inclusive and unstable across models, so results are uneven and only validated on one corpus.
trace_subject: LLM prompting strategies for systematic literature review screening
gain_reading: Some model-prompt pairings screen abstracts effectively at low cost, with zero-shot maximizing recall and CoT-few-shot giving stronger precision and F1, while GPT-4o-mini stays cheap.
gain_evidence: zero-shot maximizes recall for high-sensitivity relevance screening | GPT-4o-mini performs competitively at a substantially lower dollar cost
problem_reading: Self-reflection prompting performed poorly because it was over-inclusive and unstable across models, so results are uneven and only validated on one corpus.
problem_evidence: self-reflection underperforms due to over-inclusivity and instability across models
quick_read: Researchers compared six large language models under five prompt styles on relevance classification and six further screening tasks in systematic literature reviews. They scored each pairing on accuracy, precision, recall and F1, and also estimated cost per 1,000 abstracts.

The work matters because screening is a laborious stage of evidence synthesis, and the results offer practical guidance on matching model and prompt to the goal. The authors describe the potential as uneven but promising, and say further validation is needed before the findings are generalised beyond the corpus studied.
limitation: Results come from a single SLR-automation corpus, and the authors state further validation is needed before generalising to broader systematic review screening tasks.
tag: Dual reading
key_points: Six LLMs were tested: GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku and Llama-4-Maverick. | Five prompt types were compared: zero-shot, few-shot, chain-of-thought, CoT-few-shot and self-reflection. | Zero-shot maximized recall, while CoT-few-shot gave stronger precision and F1. | GPT-4o and DeepSeek showed robust overall performance, and GPT-4o-mini was competitive at far lower cost. | Self-reflection was over-inclusive and unstable across models.
rundown: The evaluation covered relevance classification plus six Level-2 tasks, using accuracy, precision, recall and F1. It found pronounced interaction effects between model and prompt within the SLR-automation corpus.

A cost-performance analysis for relevance classification, per 1,000 abstracts, showed large absolute differences between pairings. GPT-4o-mini stayed low-cost across prompts, and structured prompts on it gave attractive F1 for a small added cost.

The authors suggest future workflows could be staged and cost-aware. Lower-cost, high-recall configurations would handle initial screening, and higher-cost or more precision-oriented configurations would be reserved for later triage or uncertain records.
sources:
- peer_reviewed | Research Synthesis Methods | https://doi.org/10.1017/rsm.2026.10110 | 2026-10-19
prev: 0000000000000000000000000000000000000000000000000000000000000000

sha256

52f97486c0948cc9ad4033f8507cd2203aa0d0278eb11473e2fbc9ad82d09974

previous

0000000000000000000000000000000000000000000000000000000000000000