aceModerate evidence
Published recordEducationGood Space
More
Meta-Analysis on the Effect Size of Chatbot Integration in Student Science Performance
- Truvace record
- TRV-2026-2822raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- dd5799230cf5…check it
This meta-analysis examines the effectiveness of chatbot integration in science education by synthesizing quantitative findings from selected peer-reviewed studies. Employing a rigorous meta-analytic approach, eight studies published between 2007 and 2024 were analyzed using a random-effects model. The inclusion criteria focused on quasi-experimental research measuring student performance in science subjects. Effect sizes were calculated using Hedges’ g, with additional analyses conducted to assess heterogeneity…
The gain
A meta-analysis of eight quasi-experimental studies found chatbot-assisted science instruction beat traditional teaching (pooled Hedges' g = 2.41, 95% CI 1.05–3.76), with AI-driven chatbots at g = 2.97 versus 0.82 for rule-based ones.
What this doesn’t fix
Heterogeneity was very high (I² = 97.67%) and Egger's test indicated small-study effects consistent with possible publication bias, so the pooled effect should be read cautiously.
Sources
1 on file- International Journal of Information and Education Technologydoi:10.18178/ijiet.2026.16.6.2633
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Student science performance with rule-based chatbots compared with control
Hedges' g 0.815 (95% CI −0.048 to 1.678)
search to first quarter of 2024
The review’s own words
The results indicated that AI-driven chatbots demonstrated a significantly larger effect size (Hedges’ g = 2.969, 95% CI: 1.375, 4.563) compared to rule-based chatbots (Hedges’ g = 0.815, 95% CI: −0.048, 1.678).
Student science performance with chatbot-assisted instruction compared with traditional teaching
Hedges' g 2.41 (95% CI 1.05 to 3.76)
search to first quarter of 2024
The review’s own words
The pooled effect size, Hedges’s g, was 2.41 (95% CI: 1.05–3.76), indicating a statistically significant and large effect of the intervention (Table 3).
Student science performance with AI-driven chatbots compared with control
Hedges' g 2.969 (95% CI 1.375 to 4.563)
search to first quarter of 2024
The review’s own words
The results indicated that AI-driven chatbots demonstrated a significantly larger effect size (Hedges’ g = 2.969, 95% CI: 1.375, 4.563) compared to rule-based chatbots (Hedges’ g = 0.815, 95% CI: −0.048, 1.678).
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
Cite this record
v1Truvace Impact Record TRV-2026-2822, v1: “Meta-Analysis on the Effect Size of Chatbot Integration in Student Science Performance.” Truvace, 2026-11-12. /record/TRV-2026-2822 (accessed on the day you copy it). sha256 dd5799230cf59464…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2822 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.