aceModerate evidence
More
Review of 89 studies finds AI tutoring helps most when teachers get training
Study title: The Role of Machine Learning in Education: Personalisation, Pedagogy, Equity, Ethics, and the Future of Teaching and Learning. An Extended Systematic Review and Policy Analysis (2020-2026)
- Evidence type
- Systematic review or meta-analysis
- Sample
- n = 47,231
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-2775cite or check it
Study title: The Role of Machine Learning in Education: Personalisation, Pedagogy, Equity, Ethics, and the Future of Teaching and Learning. An Extended Systematic Review and Policy Analysis (2020-2026)
A pre-registered systematic review searched five databases on 30 June 2026 and found 4,278 records. After two independent reviewers screened titles, abstracts and full texts, 152 studies covering 2.3 million learners were included. In the 89 studies with enough statistics for meta-analysis (47,231 learners), machine-learning tools produced an average effect of g=0.58 (95% CI 0.51 to 0.65).
Read the study abstract
Background: Machine learning (ML) has evolved from experimental technology to embedded educational infrastructure. Despite rapid adoption, systematic evidence synthesis with reproducible methods is lacking. Objectives: To conduct a PRISMA 2020-compliant systematic review and meta-analysis of ML in education (2020-2026), examining effectiveness, teacher transformation, equity, and policy implications. Methods: Pre-registered protocol (OSF). Searched 5 databases (Web of Science, Scopus, ERIC, ACM DL, IEEE Xplore)…
The gain
A systematic review and meta-analysis of 89 studies with 47,231 learners found machine-learning teaching tools raised learning outcomes by an average g=0.58, rising to g=0.81 with teacher training and coaching versus g=0.31 without.
In brief
The size of the effect depended on how the tools were introduced. Studies with high-fidelity implementation (40-plus hours of teacher training plus coaching and curriculum alignment) averaged g=0.81, against g=0.31 for low-fidelity ones. The author says implementation fidelity moderated effects 2.6 times more than the choice of algorithm.
Effects were larger in STEM than in language arts, in longer programmes and in high-income settings. Results varied widely between studies (I²=67%), and the review also lists risks: a governance gap between student use and teacher policy, algorithmic bias, and over-reliance on hints.
Main points
- The review included 152 studies (K-12 58.5%, higher education 27%, professional 14.5%) covering 2.3 million learners; 89 had enough data for the pooled analysis.
- Subject mattered: STEM averaged g=0.71 against g=0.42 for language arts and g=0.51 for social studies.
- Longer programmes did better: studies over 16 weeks averaged g=0.69 against g=0.44 for those under 8 weeks.
- By country income, high-income settings averaged g=0.63, middle-income g=0.51 and low-income g=0.39 (only 4 low-income studies).
- A one-shot workshop under 8 hours was linked to g=0.23, against g=0.67 for a workshop plus bi-weekly coaching for 12 weeks.
- Specific outcomes listed include exam scores up 15.2%, dropout down 23% and satisfaction up 8.7%.
- Egger's test (p=.23) found no sign of publication bias, and excluding imputed or high-risk studies left the pooled effect at g=0.57 to 0.60.
The full story
The review followed PRISMA 2020 and a protocol registered on OSF. It searched Web of Science, Scopus, ERIC, ACM Digital Library and IEEE Xplore, removed 847 duplicates and screened 3,431 records, with agreement between reviewers of κ=0.81 at title/abstract stage and κ=0.87 at full-text stage. Of 534 full texts assessed, 382 were excluded and 152 were included.
Quality ratings were mostly favourable: 135 of 152 studies rated high on MMAT. Of 41 randomised trials, 24 had low risk of bias, 14 some concerns and 3 high risk; of 111 non-randomised studies, 31 were low risk, 67 moderate and 13 serious. Dropping the high-risk trials moved the pooled effect only from g=0.58 to g=0.60.
Beyond test scores, the review reports that teachers in a TeachBetter.ai national survey of 3,421 said weekly workload fell from 24.0 to 17.3 hours, with grading time dropping from 8.2 to 3.1 hours and mentoring time rising from 2.8 to 4.6 hours. It also cites costs of $180 per student per year and a 13.3x lifetime return.
The review lists risks alongside the gains: 80% of students used generative AI while 6% of teachers reported clear policies, a model underpredicted SAT results by 0.3 SD for Black and Hispanic students, and proctoring face recognition falsely flagged 34% of dark-skinned females. The author proposes a Whole-Human Education framework (70% AI, 30% human) and teacher training and bias-audit policies.
Sources
1 on file- Trends in Computer Science and Information Technologydoi:10.17352/tcsit.000114
Talk it over
Who gets this benefit, and who was left out of the study or story?
Review of 89 studies finds AI tutoring helps most when teachers get traininghttps://truvace.com/item/the-role-of-machine-learning-in-education-personalisation-pedagogy-equity-ethics
Who gets this benefit, and who was left out of the study or story?
The gain A systematic review and meta-analysis of 89 studies with 47,231 learners found machine-learning teaching tools raised learning outcomes by an average g=0.58, rising to g=0.81 with teacher training and coaching versus g=0.31 without.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a systematic review or meta-analysis with a sample of 47,231, from Trends in Computer Science and Information Technology.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
What readers say