The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2822
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-12T17:29:52.216486Z
status: published
lens: g_space
sector: education
headline: Meta-Analysis on the Effect Size of Chatbot Integration in Student Science Performance
dek: This meta-analysis examines the effectiveness of chatbot integration in science education by synthesizing quantitative findings from selected peer-reviewed studies. Employing a rigorous meta-analytic approach, eight studies published between 2007 and 2024 were analyzed using a random-effects model. The inclusion criteria focused on quasi-experimental research measuring student performance in science subjects. Effect sizes were calculated using Hedges’ g, with additional analyses conducted to assess heterogeneity…
gain_title: A meta-analysis of eight quasi-experimental studies found chatbot-assisted science instruction beat traditional teaching (pooled Hedges' g = 2.41, 95% CI 1.05–3.76), with AI-driven chatbots at g = 2.97 versus 0.82 for rule-based ones.
problem_title: (none)
trace_subject: (none)
gain_reading: A meta-analysis of eight quasi-experimental studies found chatbot-assisted science instruction beat traditional teaching (pooled Hedges' g = 2.41, 95% CI 1.05–3.76), with AI-driven chatbots at g = 2.97 versus 0.82 for rule-based ones.
gain_evidence: The pooled effect size, Hedges’s g, was 2.41 (95% CI: 1.05–3.76) | AI-driven chatbots demonstrated a significantly larger effect size (Hedges’ g = 2.969, 95% CI: 1.375, 4.563) compared to rule-based chatbots (Hedges’ g = 0.815
problem_reading: (none)
problem_evidence: (none)
quick_read: A meta-analysis from National University in Manila pooled eight quasi-experimental studies on chatbot use in science classes, drawn from an initial 1,056 records. It found a pooled Hedges' g of 2.41 (95% CI 1.05–3.76, p < 0.001) favouring chatbot-assisted instruction over traditional teaching.

The studies covered Grade 6 to university students in Taiwan, South Korea, Pakistan, the USA and Ghana, with sample sizes from 32 to 192. AI-driven chatbots (six studies) averaged g = 2.97, while rule-based chatbots (two studies) averaged 0.82, a range that included zero. The group difference was statistically significant (p = 0.020).

The evidence is uneven. Individual effects ranged from 0.35 to 5.94, heterogeneity was I² = 97.67%, and Egger's test pointed to small-study effects consistent with publication bias. The authors say the findings call for cautious interpretation and structured teacher guidance.
limitation: Heterogeneity was very high (I² = 97.67%) and Egger's test indicated small-study effects consistent with possible publication bias, so the pooled effect should be read cautiously.
tag: Evidence-backed gain
key_points: A search of 1,056 records yielded nine eligible studies; one with an extreme effect (g = 19.283) was excluded, leaving eight in the final analysis. | The pooled effect was g = 2.41 (95% CI 1.05–3.76; z = 3.48, p < 0.001) in favour of chatbot-assisted groups. | Six AI-driven chatbot studies averaged g = 2.97, versus 0.82 (CI −0.048 to 1.678) for two rule-based chatbot studies. | The largest single effects were g = 5.94 (Chang, Kuo and Hwang, 32 students) and g = 4.53 (Essel et al., 68 students in Ghana); Lin and Ye (g = 0.35) and Prondoza and Panoy (g = 0.36) had no significant effect. | Leave-one-out checks kept the pooled effect between 1.96 and 2.71, and every run stayed significant. | Heterogeneity was very high (I² = 97.67%, Q(7) = 139.95), and sample size did not explain it in a meta-regression (p = 0.436). | Egger's test was significant (p = 0.0003), though trim-and-fill imputed no missing studies. | The authors report that university students and conceptual subjects such as biology and chemistry showed larger gains, though chatbots may be weaker for higher-order thinking.
rundown: The authors searched Google Scholar, ERIC, JSTOR and Publish or Perish for studies from 2007 to early 2024. They required quasi-experimental designs, science performance as the outcome and enough data to compute effect sizes. Of 1,056 records, nine qualified, and one was dropped as an outlier because it pushed heterogeneity to 99.66%.

The included studies were small: Chang et al. had 36 university students, Lin and Ye 34 Grade 7 students, Lee et al. 192 Grade 6 students and Essel et al. 68 university students in Ghana. Weights ranged only from 11.06% to 13.01%, so the pooled estimate leans on a handful of small trials.

The authors attribute AI-driven chatbots' edge to real-time feedback, adaptive responses and self-regulated learning support. They say university students and conceptual subjects such as biology and chemistry gained more, and studies using standardized exams showed larger effects than those using self-reported gains. These explanations are interpretive and were not separately tested in the reported subgroup analysis.

The authors recommend future work on long-term retention and critical thinking, standardized outcome measures and replication, and say chatbots work best inside a structured teaching framework with teacher guidance.
sources:
- peer_reviewed | International Journal of Information and Education Technology | https://doi.org/10.18178/ijiet.2026.16.6.2633 | 2026-01-01
prev: 0000000000000000000000000000000000000000000000000000000000000000

SHA-256 hash

dd5799230cf59464063560d7f4d50d10005290228ceaee1b959ab324b86aad17

Previous version’s hash

0000000000000000000000000000000000000000000000000000000000000000