The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2814
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T17:28:09.069963Z
status: published
lens: g_space
sector: education
headline: Do AI Chatbots Improve Students’ Learning Performance in Programming Education? Evidence from a Meta-Analysis
dek: AI chatbots have emerged as innovative educational tools and drawn increasing attention from educators and researchers in programming education. Although previous research has highlighted potentials of applying AI chatbots in programming education, there is a lack of empirical evidence to understand the overall effects of using AI chatbots in programming learning as well as the critical factors that influence the effects. To fill this gap, this study conducted a meta-analysis of 32 empirical studies published be…
gain_title: A meta-analysis of 32 studies from 2015 to 2025 found students who learned programming with AI chatbots scored higher, with an effect of g+ = 0.538 on posttests and g+ = 0.650 on practice tasks.
problem_title: (none)
trace_subject: (none)
gain_reading: A meta-analysis of 32 studies from 2015 to 2025 found students who learned programming with AI chatbots scored higher, with an effect of g+ = 0.538 on posttests and g+ = 0.650 on practice tasks.
gain_evidence: a medium-to-large effect on practice performance ( g+ = 0.650, 95% CI [.330, .970], p < .001)
problem_reading: (none)
problem_evidence: (none)
quick_read: A meta-analysis of 32 empirical studies published between 2015 and 2025 found that AI chatbots improved students' programming learning. The pooled effect was g+ = 0.538 on posttest performance, which the authors call small-to-medium, and g+ = 0.650 on practice performance, medium-to-large.

The authors say earlier work pointed to the potential of chatbots in programming classes but lacked empirical evidence on their overall effect. The review also looked at which factors change the size of the benefit.

Two moderators mattered for posttest results: how the original study was designed and how many students shared each chatbot. True experiments showed larger effects than quasi-experiments, and one chatbot per student worked substantially better than one chatbot for many students.
limitation: Effects were significantly larger in true experimental designs than in quasi-experimental designs, so the pooled estimates vary with how the underlying studies were run.
tag: Evidence-backed gain
key_points: The meta-analysis pooled 32 empirical studies published between 2015 and 2025. | Posttest performance rose by g+ = 0.538 (95% CI .202 to .873, p < .01). | Practice performance rose by g+ = 0.650 (95% CI .330 to .970, p < .001). | Estimates came from robust variance estimation models. | True experimental designs produced significantly larger effects than quasi-experimental designs. | A 1:1 chatbot-to-student ratio was substantially more effective than a 1:N ratio.
rundown: The review pooled 32 empirical studies from 2015 to 2025 to estimate how much AI chatbots change programming learning performance. Using robust variance estimation, it reported g+ = 0.538 on posttests and g+ = 0.650 on practice performance.

Moderator analyses found that research design and chatbot-to-student ratio significantly influenced posttest performance. Studies with true experimental designs showed larger effects than quasi-experimental ones.

The authors found a 1:1 chatbot-student ratio substantially more effective than a 1:N ratio, and say the findings offer practical guidance for building chatbots into instructional design.
sources:
- peer_reviewed | Journal of Educational Computing Research | https://doi.org/10.1177/07356331261424211 | 2026-02-17
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

11076d557e0ea5af06fcdd6cd391f907866533e0c44e3a76d7c2eb65f1985cda

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000