Meta-analysis of 32 studies finds chatbots lift learning outcomes, with gaps in critical thinking

The effect held regardless of intervention duration, the role the chatbot played and the learning content. Chatbots significantly improved explicit reasoning, learning achievement, knowledge retention and learning interest.

Results were not uniformly positive: the analysis reported negative findings for critical thinking, learning engagement and motivation. The authors suggest future research should cover a wider range of learning components.

The authors say many studies have examined chatbots and learning, but few had summarised their effectiveness. They set out to test the effect of chatbot-assisted learning on different learning components and whether moderator variables changed it.

Moderators tested were intervention duration, chatbot roles and learning content. The medium-to-high overall effect held regardless of these variables.

By component, gains appeared in explicit reasoning, learning achievement, knowledge retention and learning interest, while critical thinking, learning engagement and motivation showed negative findings. The authors call for future research to include different learning components.