The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2791
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T17:24:40.049844Z
status: published
lens: trace
sector: science
headline: Human versus artificial intelligence: evaluating ChatGPT’s performance in conducting published systematic reviews with meta-analysis in chronic pain research
dek: Introduction Artificial intelligence (AI), particularly large-language models like Chat Generative Pre-Trained Transformer (ChatGPT), has demonstrated potential in streamlining research methodologies. Systematic reviews and meta-analyses, often considered the pinnacle of evidence-based medicine, are inherently time-intensive and demand meticulous planning, rigorous data extraction, thorough analysis, and careful synthesis. Despite promising applications of AI, its utility in conducting systematic reviews with me…
gain_title: In a validation study replicating a published spinal cord stimulation meta-analysis, ChatGPT-4o pooled mean differences with 100% accuracy for four of five analyses and matched all 95% confidence intervals and I² heterogeneity values.
problem_title: In the same spinal cord stimulation meta-analysis replication, ChatGPT-4o screened 1,241 titles and abstracts with 54.9% sensitivity and 70.4% accuracy, producing 214 false negatives, and reached only 68.4% accuracy in full-text selection.
trace_subject: ChatGPT-4o performing systematic review and meta-analysis tasks on a spinal cord stimulation emotional functioning review
gain_reading: In a validation study replicating a published spinal cord stimulation meta-analysis, ChatGPT-4o pooled mean differences with 100% accuracy for four of five analyses and matched all 95% confidence intervals and I² heterogeneity values.
gain_evidence: ChatGPT provided pooled MDs with 100% accuracy for four out of five analyses | Calculation of 95% CI and heterogeneity estimate ( I 2 score) was performed with 100% accuracy for all five analyses
problem_reading: In the same spinal cord stimulation meta-analysis replication, ChatGPT-4o screened 1,241 titles and abstracts with 54.9% sensitivity and 70.4% accuracy, producing 214 false negatives, and reached only 68.4% accuracy in full-text selection.
problem_evidence: AI was found to have sensitivity of 54.9%, specificity of 80.1%, PPV of 63.0% and NPV of 41.7% | there were 260 true positives, 153 false positives, 614 true negatives, and 214 false negatives
quick_read: Researchers tested ChatGPT-4o on three steps of a published systematic review with meta-analysis on emotional functioning after spinal cord stimulator implantation in chronic pain patients. They compared its output with the human-executed steps, treated as the gold standard.

Screening was the weak point. On 1,241 titles and abstracts ChatGPT reached 70.4% accuracy with 54.9% sensitivity and 80.1% specificity. In full-text selection of 475 papers, accuracy was 68.4%, sensitivity 75.6% and specificity 66.8%. Data pooling went much better: all five forest plots were produced, pooled mean differences matched in four of five analyses, and tau-squared values differed by 0.01 to 0.05 in three.

The authors conclude that the tool could speed up the number-crunching part of systematic reviews, while selection of studies still needs human oversight. They note that the original meta-analysis may have been in the model's training data, so performance on new datasets is untested.
limitation: The test used one previously published pain-medicine meta-analysis that may have been in ChatGPT's training data, only the first response was recorded, and risk-of-bias assessment was not tested, so performance on new reviews is unknown.
tag: Dual reading
key_points: The test replicated one published meta-analysis (Klasova et al) on spinal cord stimulation and emotional functioning, using ChatGPT-4o and human results as the gold standard. | Title and abstract screening of 1,241 records gave 260 true positives, 153 false positives, 614 true negatives and 214 false negatives. | Title and abstract screening reached 70.4% accuracy, 54.9% sensitivity, 80.1% specificity, 63.0% positive predictive value and 41.7% negative predictive value. | Full-text selection of 475 papers reached 68.4% accuracy, 75.6% sensitivity, 66.8% specificity, 33.5% positive predictive value and 86.0% negative predictive value. | When excluding papers, ChatGPT gave the correct reason for exclusion 40.0% of the time. | ChatGPT pooled five outcomes (anxiety, depression, global functioning, mental well-being, pain catastrophizing) across 8 to 23 studies or subgroups, and the forest plots showed no gross discrepancies. | Pooled mean differences were exact in four of five analyses, with a 0.01 difference for the anxiety (HADS-A) outcome; tau-squared values differed by 0.01 to 0.05 in three analyses.
rundown: The study was a prospectively conducted validation exercise exempt from institutional review board review. Prompts were written to mirror the eligibility criteria of Klasova et al: adults over 18 with chronic pain, spinal cord stimulator implantation, and emotional functioning outcomes measured by validated questionnaires. The authors pasted the full list of titles and abstracts, and then each full manuscript, into ChatGPT-4o.

Before screening, 2,186 records were identified and duplicates were removed, including 43 extra duplicates the authors removed to help the AI, leaving 1,241 abstracts instead of the original 1,284. Of 495 papers sought for full-text review, two had no full text and 16 duplicates were removed, leaving 475.

For pooling, ChatGPT was given each study's mean difference, 95% confidence interval and weight and asked for a random-effects pooled estimate, I² and tau-squared. Pooled mean differences and confidence intervals matched the published analysis, heterogeneity estimates matched in all five, and tau-squared matched in two.

The authors list several constraints: only ChatGPT's first answer was recorded although repeated screening sometimes gave different answers, risk-of-bias and certainty-of-evidence steps were not tested, and only one pain-medicine review was used. They also say future work should test unpublished datasets alongside human reviews in progress.
sources:
- peer_reviewed | Regional Anesthesia & Pain Medicine | https://doi.org/10.1136/rapm-2024-106358 | 2025-02-16
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

7c36395a0ca7abf37c00aebe67691351a6c93fe831836c3dae217f7bbf3aad6c

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000