Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Review of 78 studies finds medical students taught with AI chatbots scored higher on exams

Study title: The effectiveness of GAI-assisted teaching methods in medical education: a systematic review and meta-analysis

Evidence type
Systematic review or meta-analysis
Sample
n = 7,566
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2801cite or check it

Study title: The effectiveness of GAI-assisted teaching methods in medical education: a systematic review and meta-analysis

A systematic review and meta-analysis pooled 78 controlled studies of generative AI in medical education, covering 3,635 students in AI-assisted groups and 3,931 in control groups. Students taught with AI scored higher on knowledge exams (SMD 0.95) and practical assessments (SMD 1.48) than those taught by traditional methods.

Read the study abstract

Objective The study evaluated the effectiveness of generative artificial intelligence (GAI)-assisted teaching methods on the medical educational outcomes. Methods Following the PRISMA guidelines, a systematic search was conducted for literature on AI-assisted educational interventions in medical education (e.g. clinical, nursing and dentistry medicine). PROSPERO registration number was CRD420251173150. Meta-analyses of the outcomes were performed using the Review Manager 5.4. Heterogeneity was evaluated using th…

The gain

A meta-analysis of 78 controlled studies (3,635 students taught with generative AI, 3,931 controls) found higher knowledge scores (SMD 0.95) and practical scores (SMD 1.48) than traditional teaching in clinical, nursing and dental education.

In brief

Gains also appeared in student satisfaction (SMD 1.52), self-directed learning ability (1.25), clinical thinking ability (1.18) and analytical and problem-solving skills (1.53). Critical thinking skills showed no significant difference.

Results varied widely between studies (I2 of 94% for both main outcomes), and 51 of the 78 studies came from mainland China. Knowledge gains were concentrated among Asian students and late-phase clinical learners, with little effect in Europe and the Americas or in pre-clinical stages.

Main points

  1. The review searched English and Chinese databases from 2005 to 14 November 2025, screened 4,412 titles and abstracts, and included 78 studies; it was registered on PROSPERO (CRD420251173150).
  2. The studies comprised 61 randomized trials and 17 non-randomized controlled trials, with clinical medicine (52 studies), nursing (17) and dental students (7).
  3. Knowledge scores came from 57 studies (SMD 0.95); practical scores from 50 studies (SMD 1.48).
  4. Binary outcomes favored AI-assisted teaching, with odds ratios from 3.50 to 10.97 for learning self-efficacy, initiative, self-directed learning, clinical thinking and problem-solving.
  5. Critical thinking skills showed no significant difference (SMD 0.82, 95% CI −0.97 to 2.61, p = 0.37).
  6. Of the randomized trials, 33 were rated low risk of bias, 25 some concern and 3 high risk; non-randomized studies were mostly medium quality (15 of 17).
  7. ChatGPT was the named tool in 20 studies and self-developed AI platforms in 22; 14 studies did not name the tool.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Medical students' analytical and problem-solving skills (continuous measures) with generative AI-assisted teaching versus control

    SMD 1.53 (95% CI 0.77 to 2.29)

    The review’s own words
    The results of the meta-analysis showed a significant improvement in analytical and problem-solving skills in the experimental group compared with the control group (SMD = 1.53, 95% CI: 0.77–2.29, p < 0.00001).

  • Medical students' knowledge examination scores with generative AI-assisted teaching versus control teaching

    SMD 0.95 (95% CI 0.72 to 1.18)

    57 studies

    The review’s own words
    Compared with the control group, the pooled effect of the studies (SMD = 0.95, 95% CI: 0.72-1.18, p < 0.00001) showed a significant improvement in knowledge scores in the GAI-assisted teaching group.

  • Medical students' learning initiative (continuous measures) with generative AI-assisted teaching versus control

    SMD 1.20 (95% CI 0.10 to 2.30)

    The review’s own words
    The results of the meta-analysis showed a significant improvement in learning initiative in the experimental group compared with the control group (SMD = 1.20, 95% CI: 0.10–2.30, p < 0.00001).

  • Medical students' teaching satisfaction (continuous measures) with generative AI-assisted teaching versus control

    SMD 1.52 (95% CI 1.01 to 2.02)

    The review’s own words
    The meta-analysis results showed a significant improvement in teaching satisfaction in the experimental group compared with the control group (SMD = 1.52, 95% CI: 1.01–2.02, p < 0.00001).

  • Medical students' critical thinking skills (continuous measures) with generative AI-assisted teaching versus control

    SMD 0.82 (95% CI −0.97 to 2.61)

    The review’s own words
    The results of the meta-analysis showed no difference in critical thinking skills between the two groups (SMD = 0.82, 95% CI: −0.97–2.61, p = 0.37).

  • Medical students' practical examination scores with generative AI-assisted teaching versus control teaching

    SMD 1.48 (95% CI 1.20 to 1.77)

    50 studies

    The review’s own words
    Compared with the control group, the pooled effect of the studies (SMD = 1.48, 95%CI: 1.20–1.77, p < 0.00001) showed a significant improving effect on experimental skills scores in the group of GAI-assisted experiment teaching group.

  • Medical students' learning self-efficacy with generative AI-assisted teaching versus control

    SMD 0.75 (95% CI 0.17 to 1.32)

    The review’s own words
    The results of the meta-analysis showed a significant improvement in learning self-efficacy in the experimental group compared with the control group (SMD = 0.75, 95% CI: 0.17–1.32, p < 0.00001).

  • Medical students' teaching satisfaction (dichotomous measures) with generative AI-assisted teaching versus control

    Odds ratio 5.58 (95% CI 4.27 to 7.28)

    23 studies

    The review’s own words
    Furthermore, studies using dichotomous variables revealed statistically significant differences between the two groups (OR = 5.58, 95% CI: 4.27–7.28, p < 0.0001).

  • Medical students' self-directed learning ability (continuous measures) with generative AI-assisted teaching versus control

    SMD 1.25 (95% CI 0.81 to 1.69)

    The review’s own words
    The results of the meta-analysis showed a significant improvement in self-directed learning ability in the experimental group compared with the control group (SMD = 1.25, 95% CI: 0.81–1.69, p < 0.00001).

  • Medical students' clinical thinking ability (continuous measures) with generative AI-assisted teaching versus control

    SMD 1.18 (95% CI 0.86 to 1.50)

    The review’s own words
    The results of the meta-analysis showed a significant improvement in clinical thinking ability in the experimental group compared with the control group (SMD = 1.18, 95% CI: 0.86–1.50, p < 0.00001).

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors searched PubMed, Web of Science, ScienceDirect, Embase, the Cochrane Library and four Chinese databases. Included studies compared undergraduate medical, nursing and dental students taught with generative AI against students taught without it, and reported exam or practical scores.

Pooled effects were large but heterogeneity was high (I2 of 94% for both knowledge and practical scores). Subgroup analyses showed that region, student major, knowledge level and course content changed the size of the effect. AI-assisted learning in the late clinical phase produced higher knowledge (SMD 1.32) and practical scores (SMD 1.78), while pre-clinical effects were low.

Knowledge gains were seen among Asian students and not in courses in Europe and the Americas, while practical gains appeared in China, Europe and the Americas but little in other Asian regions. Of 78 studies, 40 were in English and 38 in Chinese, and 52 ran longer than one week.

The authors recommend that policymakers and education administrators integrate AI into teacher training and medical curriculum design, tailored to learners' knowledge level and specialism.

Sources

1 on file
  1. Frontiers in Public Healthdoi:10.3389/fpubh.2026.1813108
    Peer-reviewed2026-04-22

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Review of 78 studies finds medical students taught with AI chatbots scored higher on examshttps://truvace.com/item/the-effectiveness-of-gai-assisted-teaching-methods-in-medical-education-a-system

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of 78 controlled studies (3,635 students taught with generative AI, 3,931 controls) found higher knowledge scores (SMD 0.95) and practical scores (SMD 1.48) than traditional teaching in clinical, nursing and dental education.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis with a sample of 7,566, from Frontiers in Public Health.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Results rest on highly heterogeneous studies, 51 of 78 from mainland China, and the authors found evidence of publication bias for the main knowledge and practical outcomes in Egger's test.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Health