Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

A meta-analysis of 37 studies finds AI-assisted teaching improved results

Study title: The contingent impact of artificial intelligence on teaching effectiveness: a meta-analytic review of boundary conditions and moderating factors

Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2806cite or check it

Study title: The contingent impact of artificial intelligence on teaching effectiveness: a meta-analytic review of boundary conditions and moderating factors

A meta-analysis in Frontiers in Psychology pooled controlled studies of AI-enabled teaching published between 2015 and 2024. Studies using only post-test comparisons (37) gave a pooled effect of g = 0.586, while studies with pre-test and post-test data (35) gave a smaller gain of g = 0.136.

Read the study abstract

The digital transformation of education is reshaping pedagogical landscapes and driving the shift toward smart teaching. This meta-analysis examines the impact of Artificial Intelligence (AI)-enabled teaching on intelligent teaching assessment, synthesizing evidence from 35 pre-test-post-test designs and 37 post-test studies. Results show that AI-driven interventions yield significant positive effects, with the post-test effect size (g_p = 0.586) exceeding the pre-test-post-test gain (g_delta = 0.136), primarily…

The gain

A meta-analysis of 37 post-test studies found AI-assisted teaching scored 0.586 standard deviations higher than comparison teaching, while 35 pre-test/post-test studies that controlled for baseline found a smaller 0.136 gain.

In brief

The authors attribute the gap mainly to stricter baseline controls in the pre-test/post-test studies, and warn that post-test-only results may overstate real gains. Both estimates were statistically significant (p < 0.001), but results varied widely between studies.

What drove the variation was the type of AI tool and the direction of the reported results, not education stage, intervention type, publication year or location. The authors conclude that AI's benefit depends on how it is applied and on teachers' digital skills.

Main points

  1. The pre-test/post-test model covered 35 studies and gave a pooled effect of 0.136 (95% CI 0.071 to 0.200), which the authors call small but significant.
  2. The post-test-only model covered 37 studies and gave a pooled effect of 0.586 (95% CI 0.521 to 0.650).
  3. Heterogeneity was high: I² was 82.1% in the pre-test/post-test model and 74.3% in the post-test model, so effects differed widely between studies.
  4. AI type and results direction were the strongest moderators; education stage and intervention type did not reach significance.
  5. The literature search covered Web of Science, Scopus and ERIC for 2015 to 2024 and started with 1,681 records.
  6. In the quality assessment of 25 appraised studies, one was rated low bias, 16 moderate and eight high.
  7. The tools covered included generative AI and large language models, chatbots, adaptive learning systems and extended-reality or gamified learning.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Post-test learning outcomes with AI-assisted teaching versus traditional teaching

    SMD 0.586 (95% CI 0.521 to 0.650)

    37 studies

    The review’s own words
    The overall effect size g_p was 0.586 (95% confidence interval [0.521, 0.650], p < 0.001), showing an upward trend.

  • Learning outcomes gain from pre-test to post-test with AI-assisted teaching versus traditional teaching

    SMD 0.136 (95% CI 0.071 to 0.200)

    35 studies

    The review’s own words
    Figure 2 only included k equal to 35 studies that adopted a strict pretest-posttest design, and the cumulative analysis yielded a total effect size of 0.136 (95% confidence interval [0.071, 0.200], p < 0.001), which had a certain effect but was relatively small (i.e., the average increase was limited).

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

Researchers searched Web of Science, Scopus and ERIC for English-language studies from 2015 to 2024 using terms such as artificial intelligence, machine learning and teaching practice, combined with experimental-design terms. They began with 1,681 records and narrowed these to 345 after screening and removing duplicates; 169 studies with missing data and 113 without control groups were among those excluded.

The analysis ran two models using standardized mean differences. The pre-test/post-test model captures change relative to starting level and gave 0.136. The post-test-only model gave 0.586. The authors say this gap shows that looking only at final scores can overestimate AI's gains.

Moderator analysis found that AI type and results direction accounted for most of the variation between studies. Control-group sample size showed marginal significance, while publication year, region and intervention duration had no significant effect.

The review describes generative AI, LLM-based teaching assistants, chatbots, adaptive systems and augmented or virtual reality as the main approaches. Several included studies reported that heavy reliance on generative AI could weaken students' grasp of basic concepts, and the authors call for teacher training in digital skills.

Sources

1 on file
  1. Frontiers in Psychologydoi:10.3389/fpsyg.2026.1744690
    Peer-reviewed2026-04-29

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

A meta-analysis of 37 studies finds AI-assisted teaching improved resultshttps://truvace.com/item/the-contingent-impact-of-artificial-intelligence-on-teaching-effectiveness-a-met

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 37 post-test studies found AI-assisted teaching scored 0.586 standard deviations higher than comparison teaching, while 35 pre-test/post-test studies that controlled for baseline found a smaller 0.136 gain.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis, from Frontiers in Psychology.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The authors say the dataset is too small for reliable subgroup analysis, with many subgroups holding only one or two studies, and their funnel-plot section warns the pooled effect may overestimate AI's real impact.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education