Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Review of 19 meta-analyses finds AI tools modestly lift student learning

Study title: A Second-Order Meta-Analysis on the Effects of Artificial Intelligence Applications on Student Outcomes

Evidence type
Systematic review or meta-analysis
Sample
n = 58,702
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2817cite or check it

Study title: A Second-Order Meta-Analysis on the Effects of Artificial Intelligence Applications on Student Outcomes

A second-order meta-analysis pooled results from 19 first-order meta-analyses published between 2020 and 2025, together involving 58,702 participants. Using a random-effects model, it found a moderate mean effect size of 0.67 (95% CI 0.55 to 0.78) for AI applications on student outcomes, including academic achievement and higher-order thinking skills.

Read the study abstract

Research on AI in education has expanded rapidly, resulting in numerous first-order meta-analyses, yet these studies are limited by inconsistent findings, heterogeneous methods, variable quality, and overlapping primary studies, requiring a second-order meta-analysis to obtain a more reliable estimate of AI’s effects. This study examined the overall effect of AI applications on student outcomes, including academic achievement and higher-order thinking skills. We included first-order meta-analyses published betwe…

The gain

A second-order meta-analysis of 19 earlier meta-analyses covering 58,702 students found AI applications raised student outcomes by a moderate average effect size of 0.67 (95% CI 0.55 to 0.78).

In brief

The authors undertook the work because first-order meta-analyses of AI in education give inconsistent findings, use varied methods, differ in quality and draw on overlapping primary studies. Pooling them was meant to give a more reliable overall estimate.

Effects varied by education level, education field and publication bias status, but were robust across AI types and learning outcome types. The authors call for informed AI integration and stronger pedagogical and institutional capacity.

Main points

  1. The analysis covered 19 first-order meta-analyses published between 2020 and 2025 with 58,702 participants in total.
  2. Those meta-analyses drew on primary studies published from 1993 to 2024.
  3. The random-effects mean effect size was 0.67 (95% CI 0.55 to 0.78), described as moderate and statistically significant.
  4. Education level, education field and publication bias status helped explain variation in effects.
  5. Effect sizes held across AI types and learning outcome types.
  6. In meta-regression, sample size and publication year did not predict effect size, but the number of primary studies did.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Student outcomes (academic achievement and higher-order thinking) with AI applications, second-order meta-analysis

    Effect size (ES) .67 (95% CI .55 to .78)

    58,702 people

    The review’s own words
    Using a random-effects model, we found a statistically significant moderate mean effect size (ES = .67, 95% CI [.55–.78]), indicating that AI technologies meaningfully contribute to student learning.

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The study is a second-order meta-analysis, which pools the results of earlier meta-analyses rather than individual experiments. The team searched electronic databases for first-order meta-analyses published from 2020 to 2025, which covered primary studies from 1993 to 2024.

Outcomes included academic achievement and higher-order thinking skills. The random-effects model gave a mean effect of 0.67 (95% CI 0.55 to 0.78) across the 19 included meta-analyses.

Moderator analysis showed that effects differed by education level, education field and publication bias status. Meta-regression found that sample size and publication year did not predict effect sizes, whereas the number of primary studies did.

Sources

1 on file
  1. Journal of Educational Computing Researchdoi:10.1177/07356331261424767
    Peer-reviewed2026-02-13

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Review of 19 meta-analyses finds AI tools modestly lift student learninghttps://truvace.com/item/a-second-order-meta-analysis-on-the-effects-of-artificial-intelligence-applicati

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A second-order meta-analysis of 19 earlier meta-analyses covering 58,702 students found AI applications raised student outcomes by a moderate average effect size of 0.67 (95% CI 0.55 to 0.78).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 58,702, from Journal of Educational Computing Research.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Effects varied significantly by education level, education field and publication bias status, so the average figure does not apply uniformly across settings.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education