Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Medical students taught with ChatGPT scored 6 points higher in six trials, review shows

Study title: EP 254 ChatGTP in Undergraduate Medical Education- A Systematic Review and Meta-Analysis

Evidence type
Systematic review or meta-analysis
Sample
n = 361
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2768cite or check it

Study title: EP 254 ChatGTP in Undergraduate Medical Education- A Systematic Review and Meta-Analysis

A systematic review and meta-analysis pooled six randomised controlled trials comparing ChatGPT-assisted teaching with traditional classroom and bedside teaching. Together the trials covered 361 undergraduate medical students, 182 taught with ChatGPT and 179 taught traditionally. Students in the ChatGPT groups scored a pooled 6.21 points higher on assessments (95% CI 2.03–10.39, p = 0.004).

Read the study abstract

Abstract Background Artificial intelligence (AI) is increasingly embedded Gen-Z, with ChatGPT emerging as an educational tool. Its application has expanded rapidly across disciplines, including medical education, where it is widely used. Robust evidence synthesising its effectiveness in undergraduate medical education is still evolving. This study aimed to evaluate the impact of ChatGPT-assisted teaching compared with traditional educational methods in undergraduate medical students. Materials and Methods A syst…

The gain

A meta-analysis of six randomised trials with 361 undergraduate medical students found ChatGPT-assisted teaching produced assessment scores 6.21 points higher than traditional teaching (95% CI 2.03–10.39).

In brief

The review followed PRISMA 2020 guidelines and searched PubMed, Embase, Ovid full-text journals and the Cochrane Library up to 25 September 2025. Because it draws only on randomised trials, it offers a direct comparison of the two teaching methods in medical education, where ChatGPT is already widely used.

The results varied widely between trials (I² = 98%), and the authors flag potential publication bias. They say the findings need cautious interpretation and call for further high-quality trials to establish how ChatGPT teaching is best put into practice.

Main points

  1. Six randomised controlled trials met the inclusion criteria, covering 361 undergraduate medical students.
  2. 182 students were taught with ChatGPT and 179 with traditional classroom and bedside teaching.
  3. The pooled mean difference was 6.21 points in favour of ChatGPT (95% CI 2.03–10.39, p = 0.004).
  4. Heterogeneity between trials was substantial (I² = 98%).
  5. The authors cite potential publication bias and call for further high-quality trials to define optimal implementation.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Assessment scores of undergraduate medical students taught with ChatGPT assistance versus traditional teaching

    Mean difference 6.21 (95% CI 2.03 to 10.39)

    361 people · search to 25th September 2025

    The review’s own words
    ChatGPT-assisted learning was associated with significantly higher assessment scores, with a pooled mean difference of 6.21 points in favour of ChatGPT (95% CI 2.03–10.39, p = 0.004).

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The review was conducted under PRISMA 2020 guidelines. Searches of PubMed, Embase, Ovid full-text journals and the Cochrane Library used the terms 'ChatGPT' and 'medical education' in titles and abstracts, running until 25 September 2025.

Only randomised controlled trials in undergraduate medical students were included. The intervention was ChatGPT-assisted teaching and the comparator was traditional classroom and bedside teaching. Analysis used RevMan 5.0 with a random-effects model.

Six trials qualified, and the pooled assessment-score difference favoured ChatGPT by 6.21 points. An I² of 98% means the trials differed markedly from one another, so the pooled figure may not describe any single teaching setting. The authors say further high-quality trials are required to define how ChatGPT should be used in teaching.

Sources

1 on file
  1. British Journal of Surgerydoi:10.1093/bjs/znag087.567
    Peer-reviewed2026-08-01

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Medical students taught with ChatGPT scored 6 points higher in six trials, review showshttps://truvace.com/item/ep-254-chatgtp-in-undergraduate-medical-education-a-systematic-review-and-meta-a

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of six randomised trials with 361 undergraduate medical students found ChatGPT-assisted teaching produced assessment scores 6.21 points higher than traditional teaching (95% CI 2.03–10.39).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 361, from British Journal of Surgery.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Heterogeneity across the six trials was very high (I² = 98%) and potential publication bias was noted, so the authors say results need cautious interpretation.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Health