Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Pooled results from 58 studies find higher scores with AI tools; the true gain may be smaller

Study title: A meta-analysis of the effects of generative artificial intelligence on learning, cognitive, and linguistic outcomes

Evidence type
Systematic review or meta-analysis
Sample
n = 6,941
Sources
1 sourcePeer-reviewed
Published
3 min read
Truvace record
TRV-2026-2704cite or check it

Study title: A meta-analysis of the effects of generative artificial intelligence on learning, cognitive, and linguistic outcomes

Researchers pooled 68 effect sizes from 58 experimental or quasi-experimental studies, covering 6,941 learners, and found that generative AI support was linked to a large average gain (Hedges' g = 0.864, 95% CI 0.720 to 1.009) across learning, cognitive and linguistic outcomes. Most of the evidence came from university students.

Read the study abstract

This meta-analysis examined the effect of generative artificial intelligence (GenAI) on students' cognitive, learning, and linguistic outcomes. A total of 68 effect sizes extracted from 58 independent primary studies were analyzed using a three-level random-effects model with cluster-robust variance estimation applied at the study level. Seven moderator variables were evaluated: outcome domain, educational level, GenAI tool type, experimental design, cultural orientation, course/subject domain, and comparison-co…

The gain

A meta-analysis of 58 studies covering 6,941 learners found generative AI use was associated with a large average improvement in learning, cognitive and linguistic outcomes (pooled g = 0.864).

In brief

The average hides large differences between studies. Heterogeneity was about 88%, and the 95% prediction interval ran from −0.213 to 1.942, so a new classroom could see no benefit or a negative effect. Tests for small-study bias suggested the true effect could be as low as g of roughly 0.26 to 0.45.

The pattern differed by outcome. Language outcomes showed the biggest gains (g = 1.102) and cognitive outcomes the smallest (g = 0.535). Most of the other differences between subgroups, including educational level and culture, did not hold up after statistical correction.

Main points

  1. The analysis pooled 68 effect sizes from 58 studies with 6,941 participants; 73.5% of effect sizes came from higher education or adult learners and 10.3% from primary-age children.
  2. Effect sizes by outcome: linguistic g = 1.102 (23 effect sizes), learning g = 0.861 (29) and cognitive g = 0.535 (16); the difference was significant before but not after Holm correction (adjusted p = .062).
  3. Primary-age learners showed g = 1.311 against 0.855 for higher education and 0.612 for secondary students, but only seven effect sizes covered primary learners and the difference lost significance after correction (adjusted p = .233).
  4. Studies from vertical-collectivist settings (g = 0.926, 61 effect sizes) showed larger effects than horizontal-individualist ones (g = 0.454, 6 effect sizes), a difference that did not survive Holm correction (adjusted p = .233).
  5. Gains were larger against passive baselines such as business-as-usual teaching (g = 1.014) than against active alternatives (g = 0.765), though this difference was not statistically significant (p = .161).
  6. ChatGPT-based tools accounted for 40 of 68 effect sizes (g = 0.959) against 0.739 for other or multi-model tools, a difference that was not significant.
  7. Egger's test indicated funnel-plot asymmetry (p = .002); PET and PEESE corrections gave g = 0.258 and 0.450, and a selection model gave 0.361 with a confidence interval that included zero.
  8. Removing studies one at a time moved the pooled estimate only between g = 0.834 and g = 0.888.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Overall effect of generative AI on students' cognitive, learning and linguistic outcomes

    Hedges' g 0.864 (95% CI 0.720 to 1.009)

    58 studies · 6,941 people

    The review’s own words
    The pooled effect was large, g = 0.864, 95% CI [0.720, 1.009]

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors searched Web of Science and Scopus in May 2026, starting from 3,254 records and 2,925 after removing duplicates. They kept peer-reviewed English-language experimental or quasi-experimental studies that reported enough statistics to calculate an effect size, leaving 58 studies.

They modelled effect sizes nested within studies using a three-level random-effects model. Positive values meant outcomes favoring the AI condition, and reduced cognitive load counted as positive. Eleven effect sizes came from one-group pre-post designs and 57 from independent-group comparisons; restricting the analysis to the group comparisons gave g = 0.882.

Outcome domain and educational level reached conventional significance before correction for multiple tests, but neither survived it. Cultural orientation remained significant in two of three adjusted models (p = .032 and .041), though the third model, adding experimental design and control type, did not (p = .115) and had few residual degrees of freedom.

The countries in the dataset skewed heavily: China supplied 22 effect sizes and Taiwan 9, while only 6 effect sizes came from horizontal-individualist settings such as Germany, the USA and Norway. The authors could not fit a model combining culture, educational level and subject because those studies had no primary-level or STEM samples.

Sources

1 on file
  1. Frontiers in Educationdoi:10.3389/feduc.2026.1971476
    Peer-reviewed2026-10-21

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Pooled results from 58 studies find higher scores with AI tools; the true gain may be smallerhttps://truvace.com/item/a-meta-analysis-of-the-effects-of-generative-artificial-intelligence-on-learning

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 58 studies covering 6,941 learners found generative AI use was associated with a large average improvement in learning, cognitive and linguistic outcomes (pooled g = 0.864).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 6,941, from Frontiers in Education.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The prediction interval of −0.213 to 1.942 means a new setting could show no or negative effect, and tests for small-study bias point to an inflated average; most subgroup differences did not hold after correction.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education