Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Meta-analysis of 29 studies finds AI tutoring tools modestly boost students' problem-solving and thinking skills

Study title: Influence of interactive, AI empowered learning tools on students’ higher-order thinking: evidence from a meta-analysis

Evidence type
Systematic review or meta-analysis
Sample
n = 29
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2745cite or check it

Study title: Influence of interactive, AI empowered learning tools on students’ higher-order thinking: evidence from a meta-analysis

A meta-analysis by Asil Derin of Kafkas University pooled 29 papers published between 2000 and 2025, yielding 37 effect sizes, to test whether AI learning tools improve students' higher-order thinking. The mean effect was moderate and positive (Hedges' g 0.628, 95% CI 0.463 to 0.793), in favour of AI-supported methods over traditional teaching.

Read the study abstract

This study aimed to determine the impact of artificial intelligence (AI) tools on students’ problem-solving, creative thinking, and critical thinking skills, since the role of AI in educational settings became increasingly important. The meta-analysis was used to quantitatively synthesize relevant research findings. The first question aimed to determine the impact of AI tools on students’ problem-solving, creative thinking, and critical thinking skills, classified as HOT skills. The results, based on 29 papers p…

The gain

A meta-analysis of 29 papers (37 effect sizes, 2000–2025) found AI learning tools were linked to a moderate gain in students' problem-solving, creative and critical thinking over traditional methods (mean Hedges' g 0.628, 95% CI 0.463 to 0.793).

In brief

Results varied widely between studies (I2 about 76%). The prediction interval reaches into negative territory (-0.246), so some AI applications performed worse than conventional teaching. The author attributes this to the quality of instructional design, not to AI use alone.

Effects differed significantly by country and by specific tool, but not by education level, type of thinking skill or AI branch. Problem-solving was by far the most studied skill (33 of 37 effect sizes), so evidence on creative and critical thinking is thin.

Main points

  1. Pooled evidence came from 29 papers and 37 effect sizes, searched in Web of Science, Scopus and ERIC through May 2025.
  2. The random-effects mean effect size was 0.628 (95% CI 0.463 to 0.793), described as a medium effect.
  3. Heterogeneity was high (Q = 151.51, I2 about 76%), and the prediction interval extends to -0.246.
  4. Country moderated results: Malaysia had the largest mean effect (2.005, 2 effect sizes) and the USA, with 14, had 0.346.
  5. Tool-level results ranged from 2.005 for the Flowchart-Based Intelligent Tutoring System to -0.292 for the ubiquitous learning management system.
  6. Problem-solving accounted for 33 of 37 effect sizes; creative thinking and critical thinking had two each.
  7. Higher education supplied 20 effect sizes, versus 5 for elementary school and 5 for high school.
  8. Publication-bias tests (Egger p = .18, Begg p = .15, trim-and-fill) found no sign of bias.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Students' higher-order thinking skills with AI tools compared with traditional methods

    Hedges' g .63 (95% CI .46 to .79)

    29 studies

    The review’s own words
    the mean effect size is .63 with a confidence interval of .46 to .79

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The study followed PRISMA reporting and the six-step meta-analytic process of Field and Gillett. Searches covered Web of Science, Scopus and ERIC with keywords spanning AI, chatbots, intelligent tutors, creativity and critical thinking. Non-peer-reviewed reports, books and non-empirical studies were excluded, and three doctoral-level researchers coded studies independently and resolved discrepancies.

Effect sizes were converted to Hedges' g and combined under a random-effects model, with dependent effect sizes aggregated by the Borenstein method. The mean effect was 0.628 with a confidence interval of 0.463 to 0.793. Between-study variation was high, and the prediction interval reached -0.246.

Subgroup analyses showed that country and the specific AI tool or agent significantly moderated effect sizes. Intelligent tutors or agents (27 effect sizes) averaged 0.693 and personalised learning systems (7) averaged 0.401, a difference that was not significant. Middle school (7 effect sizes) averaged 0.308 against 0.691 for higher education, also not significant.

The author notes that most research involved undergraduates and calls for more work at K-12 level and on creative and critical thinking. Several included tools, such as Animal Watch (-0.093) and SIMALE (-0.097), showed no benefit or a slightly negative effect.

Sources

1 on file
  1. Interpersona An International Journal on Personal Relationshipsdoi:10.54899/iij.v20i1.1796
    Peer-reviewed2026-10-13

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Meta-analysis of 29 studies finds AI tutoring tools modestly boost students' problem-solving and thinking skillshttps://truvace.com/item/influence-of-interactive-ai-empowered-learning-tools-on-students-higher-order-th

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 29 papers (37 effect sizes, 2000–2025) found AI learning tools were linked to a moderate gain in students' problem-solving, creative and critical thinking over traditional methods (mean Hedges' g 0.628, 95% CI 0.463 to 0.793).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 29, from Interpersona An International Journal on Personal Relationships.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Only 29 published papers were included, with books, theses and conference proceedings excluded.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education