Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

English learners using GenAI scored higher, a review of 25 studies finds

Study title: Applying Generative Artificial Intelligence to Task-based Language Teaching and Learning: A Systematic Review and Meta-analysis

Evidence type
Systematic review or meta-analysis
Sample
n = 2,431
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2827cite or check it

Study title: Applying Generative Artificial Intelligence to Task-based Language Teaching and Learning: A Systematic Review and Meta-analysis

A systematic review and meta-analysis pooled 25 empirical studies covering 2,431 English-as-a-second or foreign-language learners. Students who used generative AI in task-based language learning outperformed those who did not, with a pooled effect of Hedges's g = 0.75 (95% CI 0.41 to 1.10), which the authors describe as large.

Read the study abstract

Abstract Generative artificial intelligence (GenAI) has received increasing attention from language researchers and educators due to its capability to influence student language performance. Very limited previous meta-analysis studies have examined instructional factors in their analysis, while it deserves great attention. Task-based language teaching (TBLT) is a widely applied language instructional approach. Hence, this systematic review and meta-analysis study adopts TBLT to investigate how these instructiona…

62Gain side

How the two sides scoreClose call

59Problem side

The gain

A meta-analysis of 25 studies with 2,431 English learners found students using GenAI tools in task-based language learning performed significantly better than those without, with a pooled effect of Hedges's g = 0.75.

The problem

In the same 25-study meta-analysis of GenAI in task-based English learning, the benefit was not significant for students without AI literacy (g = 0.24), those not allowed task planning (g = 0.28) or those assessed by GenAI (g = 0.30) or by mixed assessors (g = -0.02).

A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.

What the scores mean

G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.

P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.

In brief

The benefit was uneven. Students with AI experience or training gained much more (g = 1.04, 16 studies) than those without (g = 0.24, 9 studies, not significant). Allowing task planning (g = 0.98) and having humans assess the work (g = 0.95) also went with larger gains.

Most studies used ChatGPT, a general-purpose tool not designed for language learning. The authors suggest teachers stay involved, for example as assessors, and that language-specific GenAI tools be built. Results are drawn mostly from university students in EFL settings.

Main points

  1. The review searched Web of Science, Scopus and ProQuest for peer-reviewed English-language studies from January 2022 to January 2025, narrowing 4,108 screened articles to 25.
  2. The overall effect was g = 0.75, with very high heterogeneity (I2 = 93.89), so results varied widely between studies.
  3. Students with AI experience or training showed g = 1.04 versus 0.24 (not significant) for those without.
  4. Task planning was linked to g = 0.98 versus 0.28 without planning.
  5. Human assessors (teachers or researchers) went with g = 0.95; GenAI-only assessment gave g = 0.30 and mixed assessment g = -0.02, neither significant.
  6. Twenty of the 25 studies involved higher education and 4 involved K-12; 16 studies focused on writing and 6 on speaking.
  7. Higher education showed g = 0.81 and K-12 g = 0.38, but the difference between subgroups was not statistically significant.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Student language performance with generative AI tools compared with learning without them

    Hedges' g 0.75 (95% CI 0.41 to 1.10)

    2,431 people

    The review’s own words
    There was an overall effect size of Hedges’s g = 0.75, 95% confidence interval (CI) [0.41, 1.10] without zero, indicating a large effect size at a significant level (Sullivan & Feinn, 2012).

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors followed PRISMA guidelines and searched three databases for studies from January 2022 to January 2025, after ChatGPT's public release. From 4,492 initial records, 384 duplicates were removed, 101 articles reached full-text screening and 25 were included. Two coders coded the studies independently with 92% intercoder reliability.

Study quality averaged 12.92 out of 18 on the MERSQI scale. Eleven studies were randomized controlled experiments, 11 were non-randomized two-group designs and 3 were single-group pre- and post-test designs. Nearly all (24 of 25) were run at a single institution.

The funnel plot looked asymmetric, suggesting possible publication bias. A fail-safe N test found 965 additional null studies would be needed to cancel the overall effect, and removing studies one at a time did not change the result.

Most studies used ChatGPT, with others using GPT-4 agents, QuillBot, Writerly and iFlytek Spark. The authors say general-purpose tools may give inappropriate feedback that confuses weaker L2 learners, so teachers need to explain how the tools work, encourage planning and act as assessors.

What this doesn’t fix

The review included only English-language, peer-reviewed journal articles from three databases, covered ESL/EFL learners rather than native speakers, and drew mostly on higher-education samples, so findings may not generalise to younger learners.

Sources

1 on file
  1. TechTrendsdoi:10.1007/s11528-025-01140-7
    Peer-reviewed2025-11-05

Talk it over

Four questions to start onThis record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?

Start with a question

English learners using GenAI scored higher, a review of 25 studies findshttps://truvace.com/item/applying-generative-artificial-intelligence-to-task-based-language-teaching-and-

  1. This record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?

    The gain A meta-analysis of 25 studies with 2,431 English learners found students using GenAI tools in task-based language learning performed significantly better than those without, with a pooled effect of Hedges's g = 0.75.

    The problem In the same 25-study meta-analysis of GenAI in task-based English learning, the benefit was not significant for students without AI literacy (g = 0.24), those not allowed task planning (g = 0.28) or those assessed by GenAI (g = 0.30) or by mixed assessors (g = -0.02).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 2,431, from TechTrends.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The review included only English-language, peer-reviewed journal articles from three databases, covered ESL/EFL learners rather than native speakers, and drew mostly on higher-education samples, so findings may not generalise to younger learners.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education