aceModerate evidence
Published recordHealthGood Space
More
Effectiveness of large language model-based teaching interventions in medical education: a systematic review and stratified meta-analysis of 14 studies
- Truvace record
- TRV-2026-2771raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- 129eac06306b…check it
BACKGROUND: In recent years, artificial intelligence has been adopted as an innovative educational approach and may substantially reshape health-care delivery systems. This systematic review and stratified meta-analysis compared large language model (LLM)-based teaching interventions with traditional teaching approaches in medical education. METHODS: Based on a four-database search conducted through 31 March 2026, we included comparative studies and feasibility studies evaluating LLM-based teaching tools in medi…
The gain
A systematic review of 14 studies with 1,207 medical learners found AI chatbot-based teaching was linked to higher knowledge scores (SMD 0.80, 95% CI 0.07 to 1.53, 5 studies) than traditional teaching, though certainty was low.
What this doesn’t fix
Pooled domains rested on only 2 to 5 studies, certainty was low to very low, and the clinical performance estimate may be inflated by small-study effects, selective reporting or measurement artifacts.
Sources
1 on file- BMC Medical Educationdoi:10.1186/s12909-026-09879-0
What the review found when it combined the studies
These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.
Strict knowledge acquisition of medical learners taught with LLM tools versus traditional teaching
SMD 0.80 (95% CI 0.07 to 1.53)
5 studies · search to 31 March 2026
The review’s own words
The strict knowledge acquisition sensitivity analysis included 5 studies (164 vs. 164 participants; Fig. 4 ) and yielded a pooled effect of SMD 0.80 (95% CI 0.07 to 1.53; I² = 88.6%).
Delayed retention of knowledge in medical learners taught with LLM tools versus traditional teaching
SMD 0.15 (95% CI -0.50 to 0.80)
2 studies · search to 31 March 2026
The review’s own words
Delayed retention included 2 studies (66 vs. 82 participants; Fig. 5 ), with a pooled effect of SMD 0.15 (95% CI -0.50 to 0.80).
Post-course scores in practice courses with LLM-based teaching versus non-AI teaching
SMD 1.60 (95% CI -0.21 to 3.41)
search to 31 March 2026
The review’s own words
Exploratory subgroup analysis showed a pooled effect of SMD 0.60 (95% CI -0.10 to 1.30) for theory courses and SMD 1.60 (95% CI -0.21 to 3.41) for practice courses.
Post-course score-based outcomes of medical learners taught with LLM tools versus non-AI teaching
SMD 0.90 (95% CI 0.18 to 1.62)
search to 31 March 2026
The review’s own words
The pooled analysis suggested that LLM-based teaching was associated with higher post-course score-based outcomes than non-AI control teaching, with a pooled effect size of SMD 0.90 (95% CI 0.18 to 1.62; I² = 92.5%; P = 0.015).
Post-course scores in theory courses with LLM-based teaching versus non-AI teaching
SMD 0.60 (95% CI -0.10 to 1.30)
search to 31 March 2026
The review’s own words
Exploratory subgroup analysis showed a pooled effect of SMD 0.60 (95% CI -0.10 to 1.30) for theory courses and SMD 1.60 (95% CI -0.21 to 3.41) for practice courses.
Structured clinical performance of medical learners taught with LLM tools versus traditional teaching
SMD 3.39 (95% CI 1.43 to 5.35)
2 studies · 77 people · search to 31 March 2026
The review’s own words
Structured clinical performance included 2 studies (38 vs. 39 participants), with a pooled effect of SMD 3.39 (95% CI 1.43 to 5.35) (Fig. 6 ).
Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results
Cite this record
v1Truvace Impact Record TRV-2026-2771, v1: “Effectiveness of large language model-based teaching interventions in medical education: a systematic review and stratified meta-analysis of 14 studies.” Truvace, 2026-11-12. /record/TRV-2026-2771 (accessed on the day you copy it). sha256 129eac06306b63d1…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2771 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.