Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Medical students taught with AI scored higher on knowledge and skills, review of 14 trials shows

Study title: Meta-analysis of randomized controlled trials comparing outcomes of artificial intelligence-based teaching versus traditional-based teaching in medical education

Evidence type
Systematic review or meta-analysis
Sample
n = 1,116
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2786cite or check it

Study title: Meta-analysis of randomized controlled trials comparing outcomes of artificial intelligence-based teaching versus traditional-based teaching in medical education

A meta-analysis in the Postgraduate Medical Journal pooled 14 randomized controlled trials with 1,116 medical students, split evenly between AI-based teaching (558) and traditional teaching (558). AI-based teaching came out ahead on all three outcomes: knowledge (SMD 0.36), skills (SMD 0.78) and satisfaction (SMD 0.97).

Read the study abstract

AIMS: To evaluate comparative outcomes of artificial Intelligence (AI)-based and traditional-based teaching in medical education. METHODS: The literature search was carried out in CENTRAL, CINAHL, Web of Science, MEDLINE, and EMBASE to identify randomized controlled trials (RCTs) comparing AI-based versus traditional-based teachings in medical education. Estimate of effect size was determined for knowledge score, skills score, and teaching satisfaction score via fixed-effect modelling. RESULTS: Fourteen RCTs enr…

The gain

A meta-analysis of 14 randomized trials with 1,116 medical students found AI-based teaching produced higher knowledge scores (SMD 0.36), skills scores (SMD 0.78) and satisfaction scores (SMD 0.97) than traditional teaching.

In brief

The gains were largest for skills and satisfaction. Subgroup analyses by practical versus theoretical courses and by course length (shorter or longer than one week) matched the main results, and meta-regression showed practical courses added to the effect on knowledge (P = .002) and skills (P = .0001).

The authors rate this as level 1a evidence but caution that results may be affected by publication and reporting bias and by high variation between studies. They call for studies of AI-based teaching in postgraduate, specialty and subspecialty training.

Main points

  1. Fourteen randomized controlled trials were included, enrolling 1,116 medical students in total.
  2. Students were split evenly: 558 received AI-based teaching and 558 received traditional teaching.
  3. Knowledge scores were higher with AI-based teaching (SMD 0.36, 95% CI 0.24-0.49).
  4. Skills scores showed a larger difference (SMD 0.78, 95% CI 0.57-0.99).
  5. Satisfaction scores showed the largest difference (SMD 0.97, 95% CI 0.66-1.29).
  6. Meta-regression found practical courses significantly increased the effect on knowledge (P = .002) and skills (P = .0001).

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Medical student knowledge score with AI-based teaching compared with traditional teaching

    SMD 0.36 (95% CI 0.24 to 0.49)

    1,116 people

    The review’s own words
    The use of AI-based teaching was associated with significantly higher knowledge score (standardized mean difference (SMD): 0.36, 95% CI, 0.24-0.49, P < .00001), skills score (SMD: 0.78, 95% CI, 0.57-0.99, P < .00001), and teaching satisfaction score (SMD: 0.97, 95% CI, 0.66-1.29, P < .00001) compared to the traditional-based teaching.

  • Medical student teaching satisfaction score with AI-based teaching compared with traditional teaching

    SMD 0.97 (95% CI 0.66 to 1.29)

    1,116 people

    The review’s own words
    The use of AI-based teaching was associated with significantly higher knowledge score (standardized mean difference (SMD): 0.36, 95% CI, 0.24-0.49, P < .00001), skills score (SMD: 0.78, 95% CI, 0.57-0.99, P < .00001), and teaching satisfaction score (SMD: 0.97, 95% CI, 0.66-1.29, P < .00001) compared to the traditional-based teaching.

  • Medical student skills score with AI-based teaching compared with traditional teaching

    SMD 0.78 (95% CI 0.57 to 0.99)

    1,116 people

    The review’s own words
    The use of AI-based teaching was associated with significantly higher knowledge score (standardized mean difference (SMD): 0.36, 95% CI, 0.24-0.49, P < .00001), skills score (SMD: 0.78, 95% CI, 0.57-0.99, P < .00001), and teaching satisfaction score (SMD: 0.97, 95% CI, 0.66-1.29, P < .00001) compared to the traditional-based teaching.

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors searched CENTRAL, CINAHL, Web of Science, MEDLINE and EMBASE for randomized trials that compared AI-based and traditional teaching in medical education. Effects on knowledge, skills and satisfaction were pooled using fixed-effect modelling.

Subgroup analyses by practical course, theoretical course, and course duration under or over one week agreed with the overall findings. Practical courses showed a significantly larger effect on knowledge and skills scores.

The authors say future work should test AI-based teaching in postgraduate settings, including specialty and subspecialty training, and note that the evidence may be subject to publication and reporting bias with high between-study heterogeneity.

Sources

1 on file
  1. Postgraduate Medical Journaldoi:10.1093/postmj/qgag077
    Peer-reviewed2026-05-21

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Medical students taught with AI scored higher on knowledge and skills, review of 14 trials showshttps://truvace.com/item/meta-analysis-of-randomized-controlled-trials-comparing-outcomes-of-artificial-i

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 14 randomized trials with 1,116 medical students found AI-based teaching produced higher knowledge scores (SMD 0.36), skills scores (SMD 0.78) and satisfaction scores (SMD 0.97) than traditional teaching.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 1,116, from Postgraduate Medical Journal.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The authors say the evidence may be subject to publication and reporting bias, with high heterogeneity between studies, and the trials covered medical students rather than postgraduate trainees.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education