Meta-analysis finds AI tools gave college students a medium boost in academic results

The authors began from the lack of academic consensus on whether generative AI helps college students learn. They examined how the effect varied by activity category, sample size, type of generated content, intervention duration, discipline and assessment tool.

The gains were largest when the generated content was text (g = 0.554), in studies with 21 to 40 participants (g = 0.776) and where students used independent learning styles (g = 0.600). Duration, discipline and assessment tools also showed moderate positive effects, though differences within those groups were not significant.

The authors set out to settle whether generative AI actually improves college students' academic achievement, since there was no academic consensus. They used a meta-analytic approach and tested several moderating variables.

Three moderators showed within-group differences: activity categories, sample size and generated content. Text content (g = 0.554), samples of 21 to 40 (g = 0.776) and independent learning styles (g = 0.600) had the strongest effects.

Intervention duration, discipline types and assessment tools also had moderate positive effects, but no significant within-group differences emerged for any of those moderators. The authors say the findings provide empirical evidence for applying generative AI and for educational technology policy.