aceModerate evidence
Published recordEducationGood Space
More
Do AI Chatbots Improve Students’ Learning Performance in Programming Education? Evidence from a Meta-Analysis
- Truvace record
- TRV-2026-2814raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- 11076d557e0e…check it
AI chatbots have emerged as innovative educational tools and drawn increasing attention from educators and researchers in programming education. Although previous research has highlighted potentials of applying AI chatbots in programming education, there is a lack of empirical evidence to understand the overall effects of using AI chatbots in programming learning as well as the critical factors that influence the effects. To fill this gap, this study conducted a meta-analysis of 32 empirical studies published be…
The gain
A meta-analysis of 32 studies from 2015 to 2025 found students who learned programming with AI chatbots scored higher, with an effect of g+ = 0.538 on posttests and g+ = 0.650 on practice tasks.
What this doesn’t fix
Effects were significantly larger in true experimental designs than in quasi-experimental designs, so the pooled estimates vary with how the underlying studies were run.
Sources
1 on file- Journal of Educational Computing Researchdoi:10.1177/07356331261424211
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Programming practice performance with AI chatbots compared with control conditions
Hedges' g 0.650 (95% CI .330 to .970)
32 studies
The review’s own words
a medium-to-large effect on practice performance ( g+ = 0.650, 95% CI [.330, .970], p < .001)
Programming posttest performance with AI chatbots compared with control conditions
Hedges' g 0.538 (95% CI .202 to .873)
32 studies
The review’s own words
The results indicated a small-to-medium effect on posttest performance ( g+ = 0.538, 95% CI [.202, .873], p < .01)
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
Cite this record
v1Truvace Impact Record TRV-2026-2814, v1: “Do AI Chatbots Improve Students’ Learning Performance in Programming Education? Evidence from a Meta-Analysis.” Truvace, 2026-11-12. /record/TRV-2026-2814 (accessed on the day you copy it). sha256 11076d557e0ea5af…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2814 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.