Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Review of 27 studies shows AI chatbots give students a moderate learning boost

Study title: Exploring the Impact of Generative AI-Powered Conversational Agents on Student Learning: A Systematic Review and Meta-Analysis Grounded in Activity Theory

Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
Published
1 min read
Truvace record
TRV-2026-2811cite or check it

Study title: Exploring the Impact of Generative AI-Powered Conversational Agents on Student Learning: A Systematic Review and Meta-Analysis Grounded in Activity Theory

A systematic review and meta-analysis following PRISMA guidelines pooled 27 empirical studies published between 2022 and 2025. It examined how generative AI-powered conversational agents affect students' learning outcomes.

Read the study abstract

Grounded in activity theory, this study conducted a systematic review and meta-analysis, following PRISMA guidelines, to evaluate the impact of generative AI-powered conversational agents on students’ learning outcomes in both cognitive and non-cognitive skills. Twenty-seven empirical studies published between 2022 and 2025 were analyzed. The results revealed that generative AI-powered conversational agents had moderately positive effects on both cognitive ( g = 0.462, p < .001) and non-cognitive skills ( g = 0.…

The gain

A systematic review and meta-analysis of 27 studies published from 2022 to 2025 shows students who learned with generative AI chatbots gained moderately in cognitive skills (g = 0.462) and non-cognitive skills (g = 0.519).

In brief

The pooled effect was moderately positive for cognitive skills (g = 0.462, p < .001) and slightly larger for non-cognitive skills (g = 0.519, p < .001).

Moderator analyses found that educational level, knowledge type, intervention duration, agent role and instructional mode were significant or marginally significant factors, so results differ by classroom context.

Main points

  1. The review followed PRISMA guidelines and analysed 27 empirical studies published between 2022 and 2025.
  2. The pooled effect on cognitive skills was g = 0.462 (p < .001).
  3. The pooled effect on non-cognitive skills was g = 0.519 (p < .001).
  4. Educational level, knowledge type, intervention duration, agent role and instructional mode were significant or marginally significant moderators.
  5. The analysis is grounded in activity theory and covers both cognitive and non-cognitive learning outcomes.

The full story

The authors synthesised empirical studies from 2022 to 2025, a period covering the spread of generative AI chatbots in education. They assessed outcomes in two groups: cognitive skills and non-cognitive skills.

Both groups showed moderate positive pooled effects, with g = 0.462 for cognitive and g = 0.519 for non-cognitive skills, both at p < .001.

Moderator analyses indicated that effects depend on context, including educational level, type of knowledge, how long the intervention ran, the role the agent played and the mode of instruction.

Sources

1 on file
  1. Journal of Educational Computing Researchdoi:10.1177/07356331261433396
    Peer-reviewed2026-03-20

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Review of 27 studies shows AI chatbots give students a moderate learning boosthttps://truvace.com/item/exploring-the-impact-of-generative-ai-powered-conversational-agents-on-student-l

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A systematic review and meta-analysis of 27 studies published from 2022 to 2025 shows students who learned with generative AI chatbots gained moderately in cognitive skills (g = 0.462) and non-cognitive skills (g = 0.519).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis, from Journal of Educational Computing Research.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education