aceModerate evidence
More
Meta-analysis of 13 studies finds AI tools tied to better student learning
Study title: Meta-Analysis of Artificial Intelligence in Education
- Evidence type
- Systematic review or meta-analysis
- Sample
- n = 13
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2838cite or check it
Study title: Meta-Analysis of Artificial Intelligence in Education
A meta-analysis of 13 empirical studies conducted in eight countries found that AI technologies were associated with better educational outcomes. The pooled effect size was Hedges' g = 0.86 (95% CI 0.45 to 1.27).
Read the study abstract
This meta-analysis examined the effectiveness of artificial intelligence (AI) technologies in educational settings through a systematic review of 13 empirical studies conducted across eight countries. We analysed the impact of various AI technologies on educational outcomes using PRISMA guidelines and multiple analytical approaches, including novel applications of Naive Bayes, TF-IDF, and BERT-based algorithms. The overall analysis revealed a significant positive effect size (Hedges' g = 0.86, 95% CI [0.45, 1.27…
The gain
A meta-analysis of 13 studies across eight countries found AI technologies in education were linked to better student learning outcomes (Hedges' g = 0.86), with chatbots and generative AI strongest at 1.02.
In brief
Chatbots and generative AI showed the largest effect, at 1.02 (95% CI 0.45 to 1.59). Learning management systems and AI platforms showed a smaller effect of 0.62, and online learning and virtual reality applications 0.79, with a confidence interval that crossed zero.
Results varied widely between the included studies, with heterogeneity (I²) ranging from 54.03% to 93.23%. The authors say implementation strategies and context matter for outcomes.
Main points
- The review covered 13 empirical studies from eight countries and followed PRISMA guidelines.
- The pooled effect of AI on educational outcomes was Hedges' g = 0.86 (95% CI 0.45 to 1.27, p < 0.0001).
- Chatbots and generative AI had the largest effect on student learning, at 1.02 (95% CI 0.45 to 1.59).
- Online learning and virtual reality applications had an effect of 0.79, but the interval (-0.04 to 1.62) included zero and p was below 0.07.
- Learning management systems and AI platforms showed a more modest effect of 0.62 (95% CI 0.03 to 1.21, p < 0.05).
- Heterogeneity across studies was high, with I² between 54.03% and 93.23%.
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Student learning outcomes with chatbots and generative AI
Effect size 1.02 (95% CI 0.45 to 1.59)
13 studies
The review’s own words
Particularly noteworthy were the effects of chatbots and generative AI (effect size = 1.02, 95% CI [0.45, 1.59], p < 0.0001), which demonstrated the most substantial positive impact on student learning outcomes.
Educational outcomes with learning management systems and AI platforms
Effect size 0.62 (95% CI 0.03 to 1.21)
13 studies
The review’s own words
Online learning and virtual reality applications showed moderate positive effects (effect size = 0.79, 95% CI [-0.04, 1.62], p < 0.07) while learning management systems and AI platforms demonstrated promising but more modest impacts (effect size = 0.62, 95% CI [0.03, 1.21], p < 0.05).
Educational outcomes with online learning and virtual reality applications
Effect size 0.79 (95% CI -0.04 to 1.62)
13 studies
The review’s own words
Online learning and virtual reality applications showed moderate positive effects (effect size = 0.79, 95% CI [-0.04, 1.62], p < 0.07) while learning management systems and AI platforms demonstrated promising but more modest impacts (effect size = 0.62, 95% CI [0.03, 1.21], p < 0.05).
Educational outcomes with AI technologies overall
Hedges' g 0.86 (95% CI 0.45 to 1.27)
13 studies
The review’s own words
The overall analysis revealed a significant positive effect size (Hedges' g = 0.86, 95% CI [0.45, 1.27], p < 0.0001), indicating substantial benefits of AI integration in education.
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
The full story
The authors pooled results from 13 empirical studies across eight countries, following PRISMA guidelines. They used Naive Bayes, TF-IDF and BERT-based algorithms alongside conventional analysis, and a weighted hybrid model that combines random and fixed effects approaches.
The overall effect was large and statistically significant (Hedges' g = 0.86, 95% CI 0.45 to 1.27). By technology type, chatbots and generative AI scored highest at 1.02, followed by online learning and virtual reality at 0.79 and learning management systems and AI platforms at 0.62.
The online learning and virtual reality estimate was not statistically significant at the conventional level (95% CI -0.04 to 1.62). The authors report significant heterogeneity across studies and say contextual factors and implementation strategies should be considered.
Sources
1 on file- Higher Education Studiesdoi:10.5539/hes.v15n2p189
Talk it over
Who gets this benefit, and who was left out of the study or story?
Meta-analysis of 13 studies finds AI tools tied to better student learninghttps://truvace.com/item/meta-analysis-of-artificial-intelligence-in-education
Who gets this benefit, and who was left out of the study or story?
The gain A meta-analysis of 13 studies across eight countries found AI technologies in education were linked to better student learning outcomes (Hedges' g = 0.86), with chatbots and generative AI strongest at 1.02.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a systematic review or meta-analysis with a sample of 13, from Higher Education Studies.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
The record’s own caveat Heterogeneity across the 13 studies was high (I² from 54.03% to 93.23%), and the online learning and virtual reality estimate had a confidence interval that crossed zero.
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
What readers say