Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Medical trainees taught with AI scored higher in 20 trials, but certainty is low to moderate

Study title: The impact of integrating generative artificial intelligence into medical education on short-term learning outcomes: a systematic review and meta-analysis of randomized controlled trials

Evidence type
Systematic review or meta-analysis
Sample
n = 1,413
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2794cite or check it

Study title: The impact of integrating generative artificial intelligence into medical education on short-term learning outcomes: a systematic review and meta-analysis of randomized controlled trials

A systematic review and meta-analysis pooled 20 randomized controlled trials with 1,413 medical students, interns and residents. Trainees taught with generative AI scored higher on knowledge after teaching (SMD 0.99, low certainty) and on clinical skills (SMD 1.09, moderate certainty) than those taught traditionally or with non-AI digital methods.

Read the study abstract

BACKGROUND: Generative artificial intelligence (GenAI) technologies —exemplified by large language models (LLMs) such as ChatGPT and DeepSeek—have attracted substantial interest in medical education for their capacity to generate content, adapt to individual learners, and support interactive dialogue. Whether these technologies produce measurable improvements in educational outcomes remains uncertain, and this evidence gap has constrained broader adoption. We conducted this systematic review and meta-analysis of…

The gain

A meta-analysis of 20 randomized trials (1,413 medical students, interns and residents) found clinical skill scores were higher with generative AI-assisted teaching than with traditional or non-AI digital teaching (SMD 1.09, moderate certainty).

In brief

Knowledge gains persisted at follow-up in four trials (SMD 0.51, moderate certainty). Publication bias was not detected, and trim-and-fill adjustment gave a similar estimate (SMD 0.90). Effects did not differ significantly by AI tool type, learner level, interaction modality or risk of bias.

The authors say the evidence is not definitive. The prediction interval for knowledge ran from -0.57 to 2.56, so a benefit in future settings cannot be assumed. They suggest generative AI may supplement conventional teaching but should not replace it until well-powered, pre-registered, multi-centre trials exist.

Main points

  1. The review covered 20 randomized trials with 1,413 participants, searched across six databases from January 2020 to February 2026.
  2. Post-intervention knowledge scores were higher in AI groups (SMD 0.99, 95% CI 0.49–1.49; 11 trials; low certainty).
  3. Clinical skill scores were higher in AI groups (SMD 1.09, 95% CI 0.81–1.38; 8 trials; moderate certainty).
  4. Knowledge retention at follow-up stayed significant (SMD 0.51, 95% CI 0.24–0.78; 4 trials; moderate certainty).
  5. The knowledge prediction interval crossed zero (-0.57 to 2.56), so benefit in future settings is not guaranteed.
  6. Knowledge results were highly heterogeneous (I2 = 86.7%), and trials lacked blinding.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Knowledge retention at follow-up in medical learners taught with generative AI versus control teaching

    SMD 0.51 (95% CI 0.24 to 0.78)

    4 studies · search to February 2026

    The review’s own words
    At follow-up, knowledge retention remained significant (SMD = 0.51, 95% CI: 0.24–0.78, P = 0.010; k = 4; I2 = 0%; GRADE: moderate certainty)

  • Post-intervention knowledge scores in medical learners taught with generative AI versus control teaching

    SMD 0.99 (95% CI 0.49 to 1.49)

    11 studies · search to February 2026

    The review’s own words
    Post-intervention knowledge scores were higher in GenAI groups (SMD = 0.99, 95% CI: 0.49–1.49, P = 0.001; k = 11; I2 = 86.7%; GRADE: low certainty)

  • Clinical skill scores in medical learners taught with generative AI versus control teaching

    SMD 1.09 (95% CI 0.81 to 1.38)

    8 studies · search to February 2026

    The review’s own words
    clinical skill scores (SMD = 1.09, 95% CI: 0.81–1.38, P < 0.001; k = 8; I2 = 41.0%; GRADE: moderate certainty)

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors followed PRISMA 2020 and registered the protocol on PROSPERO. They searched PubMed, Cochrane Library, Embase, Web of Science, CNKI and Wanfang. Included trials compared generative AI-integrated teaching with traditional teaching or non-AI digital teaching among medical students, interns or residents.

Risk of bias was assessed with Cochrane RoB 2 and certainty of evidence with GRADE. Pooling used random-effects models with REML estimation. Egger's test found no publication bias (P = 0.104).

Subgroup comparisons by AI tool type, learner level, interaction modality and risk of bias were all non-significant. The authors conclude evidence certainty is low to moderate because of heterogeneity, risk of bias and absence of blinding.

Sources

1 on file
  1. BMC Medical Educationdoi:10.1186/s12909-026-09320-6
    Peer-reviewed2026-04-30

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Medical trainees taught with AI scored higher in 20 trials, but certainty is low to moderatehttps://truvace.com/item/the-impact-of-integrating-generative-artificial-intelligence-into-medical-educat

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 20 randomized trials (1,413 medical students, interns and residents) found clinical skill scores were higher with generative AI-assisted teaching than with traditional or non-AI digital teaching (SMD 1.09, moderate certainty).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 1,413, from BMC Medical Education.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Certainty is low to moderate owing to high heterogeneity, risk of bias and lack of blinding, and only four studies assessed follow-up, which limits confidence in how long gains last.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Health