aceModerate evidence
Published recordEducationThe Trace
More
The Effectiveness of Artificial Intelligence in Undergraduate Health Professions Education: Systematic Review and Meta-Analysis of Randomized Controlled Trials
- Truvace record
- TRV-2026-2810raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- c1a924f4b7e5…check it
Background: Health professions education faces increasing challenges from rising health care complexity, pedagogical shifts, and constrained curricular space, and rapidly expanding knowledge and technological advances. While artificial intelligence (AI) shows promise for transforming health professions education, evidence of its effectiveness remains unclear. Objective: This study synthesized evidence from randomized controlled trials (RCTs) on the effectiveness of AI in undergraduate health professions educatio…
61Gain side
How the two sides scoreProblem scores higher
67Problem side
The gain
A meta-analysis of 66 randomized trials (4,911 undergraduate health students) found AI chatbot-based personalized learning aids beat standard teaching on theoretical knowledge (SMD 0.53, 12 studies, 955 participants), though certainty was very low.
The problem
A meta-analysis of 66 randomized trials of AI in undergraduate health education found most had high risk of bias, certainty ranged from low to very low, and no trial measured real-life practice or patient outcomes.
A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.
What the scores mean
G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.
P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.
What this doesn’t fix
The authors say high risk of bias, heterogeneity, imprecision and the absence of higher-level outcomes limit conclusions, and effects were inconsistent and not reliably reproducible.
Sources
1 on file- JMIR Medical Educationdoi:10.2196/88933
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Theoretical knowledge of students using LLM-based personalized learning aids in health professions education
SMD 0.53 (95% CI 0.13 to 0.94)
12 studies · 955 people · search to January 26, 2026
The review’s own words
theoretical knowledge (SMD 0.53, 95% CI 0.13-0.94; 12 studies; 955 participants; I²=86%)
Student satisfaction with LLM-based personalized learning aids in undergraduate health professions education
SMD 0.93 (95% CI 0.40 to 1.46)
7 studies · 430 people · search to January 26, 2026
The review’s own words
Large language model (LLM)-based personalized learning aids comprised the largest evidence base and showed positive effects for satisfaction (standardized mean difference [SMD] 0.93, 95% CI 0.40-1.46; 7 studies; 430 participants; I²=74%)
Student confidence with LLM-based personalized learning aids in undergraduate health professions education
SMD 0.91 (95% CI 0.54 to 1.29)
7 studies · 609 people · search to January 26, 2026
The review’s own words
confidence (SMD 0.91, 95% CI 0.54-1.29; 7 studies; 609 participants; I²=64%)
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
Cite this record
v1Truvace Impact Record TRV-2026-2810, v1: “The Effectiveness of Artificial Intelligence in Undergraduate Health Professions Education: Systematic Review and Meta-Analysis of Randomized Controlled Trials.” Truvace, 2026-11-12. /record/TRV-2026-2810 (accessed on the day you copy it). sha256 c1a924f4b7e53368…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2810 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.