aceModerate evidence
More
Meta-analysis finds AI tools lift second-language writing, especially in courses of 8+ weeks
Study title: Do AI Writing Tools Improve L2 Writing Performance? A Meta-Analysis of Experimental Evidence, Moderators, and Methodological Quality
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2749cite or check it
Study title: Do AI Writing Tools Improve L2 Writing Performance? A Meta-Analysis of Experimental Evidence, Moderators, and Methodological Quality
A meta-analysis of experimental and quasi-experimental studies found that AI writing tools improved second-language (L2) writing performance. The main model, built on 22 effects with directly extractable statistics, produced a pooled effect of g = 0.714 (95% CI 0.592 to 0.836), with low heterogeneity (I² = 24.3%).
Read the study abstract
Artificial intelligence (AI) writing tools—including large language model (LLM)-based systems, automated writing evaluation (AWE) platforms, and AI-powered writing assistants—have become an increasingly prominent feature of second language (L2) writing pedagogy. Yet whether these tools reliably improve L2 writing performance, and under what instructional conditions, remains an open question. This study reports a systematic meta-analysis of experimental and quasi-experimental evidence on the effects of AI writing…
The gain
A meta-analysis of experimental studies published 2021 to 2026 found AI writing tools improved second-language writing performance, with a pooled effect of g = 0.714 (95% CI 0.592 to 0.836) across 22 verified effect sizes.
In brief
Searches of Scopus, Web of Science and ERIC, plus citation tracing and hand-searching, identified 34 eligible effect sizes from studies published between 2021 and 2026. A second model using all 34 effects, including converted ones, served as a robustness check.
Exploratory moderator analyses linked larger effects to multidimensional feedback, interventions lasting eight weeks or more, and university settings. Single-session designs and feedback limited to lower-order concerns showed smaller gains. Many studies lacked delayed transfer measures and transparent effect-size reporting.
Main points
- Pooled effect across 22 verified effects was g = 0.714 (95% CI 0.592 to 0.836).
- Observed heterogeneity was low, with I² = 24.3%.
- 34 eligible effect sizes came from studies published between 2021 and 2026.
- The databases searched were Scopus, Web of Science and ERIC, with forward citation tracing and hand-searching added.
- Interventions lasting eight weeks or more, multidimensional feedback and university contexts were associated with larger effects.
- Single-session interventions and feedback on lower-order concerns only showed smaller gains.
- Methodological quality was generally adequate but uneven; many studies lacked delayed transfer measures.
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Second language writing performance with AI writing tools versus comparison conditions (verified subset)
Hedges' g 0.714 (95% CI 0.592 to 0.836)
22 studies
The review’s own words
The verified model produced a pooled effect of g = 0.714, 95% CI [0.592, 0.836], with low observed heterogeneity (I² = 24.3%).
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
The full story
The analysis covered AI writing tools broadly, including systems built on large language models, automated writing evaluation platforms and AI-powered writing assistants, in second-language writing instruction. Effect sizes were expressed as Hedges' g.
The main estimate used restricted maximum-likelihood (REML) with Knapp-Hartung adjustment on 22 effects with directly extractable statistics. A 34-effect model that included conversion-based effects was run as a robustness analysis.
The authors describe the tools as pedagogically contingent rather than inherently effective. They say educational value depends on how feedback literacy, teacher mediation and instructional design shape opportunities for noticing, revision and transferable writing development.
Sources
1 on file- Journal of Educational Technology and Innovationdoi:10.61414/awr9xk46
Talk it over
Who gets this benefit, and who was left out of the study or story?
Meta-analysis finds AI tools lift second-language writing, especially in courses of 8+ weekshttps://truvace.com/item/do-ai-writing-tools-improve-l2-writing-performance-a-meta-analysis-of-experiment
Who gets this benefit, and who was left out of the study or story?
The gain A meta-analysis of experimental studies published 2021 to 2026 found AI writing tools improved second-language writing performance, with a pooled effect of g = 0.714 (95% CI 0.592 to 0.836) across 22 verified effect sizes.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a systematic review or meta-analysis, from Journal of Educational Technology and Innovation.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
The record’s own caveat Moderator analyses were exploratory, and many included studies lacked delayed transfer measures and transparent effect-size reporting, so lasting effects are not established.
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
What readers say