Review of 15 controlled studies finds AI-supported STEM lessons tied to higher student scores

Results varied widely between studies: heterogeneity was substantial (Q = 72.467, I² = 73%), and the 95% prediction interval ran from -0.26 to 3.40. That means a new study in a comparable setting could show little or no benefit. Individual study effects ranged from 0.23 to 4.53.

Adjusting for possible missing studies cut the pooled effect from 1.57 to 0.89. The effect stayed positive, but the authors say some influence of publication bias cannot be ruled out. They conclude the evidence points to a generally positive association, not a uniform effect in every classroom.

The authors, from universities in Kazakhstan, pooled studies of intelligent tutoring systems, adaptive learning platforms, chatbots and generative AI tools used in STEM teaching. They calculated Hedges' g for each study and combined them with a random-effects model in Comprehensive Meta-Analysis 4.0.

Publication-bias checks gave mixed signals. Begg and Mazumdar (Kendall's tau = 0.000, p = 0.500) and Egger's test (intercept 0.987, p = 0.213) found no significant bias, and the fail-safe N was 790. Even so, trim-and-fill reduced the estimate to 0.89, and the authors say bias cannot be fully excluded.

A meta-regression tested outcome type, instructional tool and approach as explanations for the differences between studies. None was significant (p = 0.420, 0.872 and 0.427), and residual heterogeneity stayed at I² = 76.02%. The authors say factors such as educational level, learner characteristics, intervention duration and implementation quality still need testing.