Moderate evidence
More
Study finds AI screening cut researchers' workload by 96% across 51,281 records
Study title: Overcoming High Volume and Complexity: Developing a Large Language Model-Assisted Scoping Review Method for Identifying Long-term Caesarean Section Outcomes
- Evidence type
- Controlled or validated comparison
- Sample
- n = 51,281
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2953cite or check it
Study title: Overcoming High Volume and Complexity: Developing a Large Language Model-Assisted Scoping Review Method for Identifying Long-term Caesarean Section Outcomes
Researchers built a hybrid workflow for reviewing medical literature on long-term outcomes of caesarean section. Google's Gemini models pulled structured information from titles and abstracts, and a rule-based pipeline then decided which records were eligible. Applied to 51,281 records from six databases, it cut the human manual screening burden by 96%.
Read the study abstract
Rapid growth in medical literature limits traditional systematic review methods, particularly in high-volume fields like maternity care. This paper presents a novel methodological approach using a hybrid workflow that combines Large Language Model (LLM) semantic data extraction with programmatic filtering to automate title and abstract screening. The workflow was deployed on a high-volume clinical dataset (51,281 records retrieved from six databases) as a validation case study. Google Gemini 1.5-pro and 2.5-flas…
The gain
In a validation study of 51,281 caesarean-section literature records, a Gemini-based screening workflow cut human manual screening by 96% for US$458.73 in API costs, with accuracy comparable to dual-human review.
In brief
A manual audit checked the results. The workflow's estimated sensitivity was 89.9%, its specificity 96.4% and its false negative rate 0.18%, which the authors describe as comparable to dual-human review. Total API spending came to US$458.73.
Swapping the language model did not change screening accuracy in a subgroup analysis. The authors say separating data extraction from inclusion rules lets reviewers keep control over eligibility criteria and could support living systematic reviews in fast-growing fields such as maternity care.
Main points
- The workflow screened 51,281 records retrieved from six databases for a review of long-term caesarean section outcomes.
- Gemini 1.5-pro and 2.5-flash extracted 12 structured semantic variables from each record.
- A seven-stage rule-based filtering pipeline then determined eligibility.
- Estimated sensitivity was 89.9% (95% CI 87.3-92.1%) and specificity 96.4% (95% CI 96.2-96.5%).
- The false negative rate was 0.18% (95% CI 0.09-0.35%), and human screening burden fell by 96%.
- Total API expenditure was US$458.73, and a subgroup analysis found no change in accuracy after changing the language model.
The full story
The method targets a bottleneck: the rapid growth of medical literature limits traditional systematic review methods, particularly in high-volume fields like maternity care. The authors used long-term caesarean section outcomes as a validation case.
Instead of asking the model to make include-or-exclude calls, the workflow has it extract 12 structured variables, which a seven-stage rule-based filter then uses to decide eligibility. The authors say this keeps researchers in control of the inclusion logic.
Accuracy was checked against a large-scale manual audit. Sensitivity, specificity and false negative rate were reported with confidence intervals, and the authors say accuracy matched dual-human review. The run cost US$458.73 in API fees, and changing the model did not alter accuracy.
Sources
1 on file- Annals of Epidemiologydoi:10.1016/j.annepidem.2026.110319
Talk it over
The Gemini-based workflow cut human screening by 96% with an estimated sensitivity of 89.9%. How would you weigh the saved effort against the eligible records that might be missed?
Study finds AI screening cut researchers' workload by 96% across 51,281 recordshttps://truvace.com/item/overcoming-high-volume-and-complexity-developing-a-large-language-model-assisted
The Gemini-based workflow cut human screening by 96% with an estimated sensitivity of 89.9%. How would you weigh the saved effort against the eligible records that might be missed?
Answer in the threadThe workflow was validated on long-term caesarean section outcomes only. Which kinds of review topics would you want tested before trusting it, and why?
Answer in the threadIf you were a researcher running a review of 51,281 records, what finding from an audit would make you go back to dual-human screening?
Answer in the thread
If you have more time
The authors say separating data extraction from inclusion rules keeps reviewers in control. How would you test whether that holds up when a team changes its criteria?
Answer in the threadThe run cost US$458.73 in API fees. Who benefits if reviews get cheaper and faster, and who might lose out, such as small research teams or readers of the reviews?
Answer in the thread
ace
What readers say