Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Review of health-science exam studies shows GPT-4 beat students, GPT-3.5 did not

Study title: Can ChatGPT be used as an education assistant in the field of health sciences? Examining with the integration of meta-analysis and co-citation analysis

Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2845cite or check it

Study title: Can ChatGPT be used as an education assistant in the field of health sciences? Examining with the integration of meta-analysis and co-citation analysis

A meta-analysis of studies that pitted students against ChatGPT on health sciences exams found the answer depends on the version. Students beat ChatGPT 3.5 (OR = 3.173, 95% CI 1.132-8.869), while ChatGPT 4.0 beat the students (OR = 2.589, 95% CI 1.046-6.409).

Read the study abstract

Background: Meta-analysis studies are frequently included in the literature, but studies integrated with co-citation analysis are very few. There is no study that evaluates the success of ChatGPT in the field of health sciences, especially by integrating these two methods. Objective: This study aims to reveal whether ChatGPT can be used as an educational assistant by comparing the performance of students in exams in the field of health sciences with ChatGPT through the integration of meta-analysis and co-citatio…

The gain

A meta-analysis of studies comparing ChatGPT with students on health sciences exams found ChatGPT 4.0 performed better than students (OR = 2.589, 95% CI 1.046-6.409).

In brief

The authors searched Web of Science for articles up to May 2024. They also ran a co-citation analysis of the references in the included studies, to see how the studies relate to the work they cite.

The result matters for educators weighing ChatGPT as a study aid in health sciences. The wide confidence intervals, with lower bounds just above 1, mean the size of the gap is uncertain, and the GPT-4.0 comparison rests on only four studies.

Main points

  1. Students outperformed ChatGPT 3.5 on health sciences exams (OR = 3.173, 95% CI 1.132-8.869).
  2. ChatGPT 4.0 outperformed students on health sciences exams (OR = 2.589, 95% CI 1.046-6.409).
  3. The authors searched Web of Science for articles published until May 2024.
  4. Co-citation analysis of the ChatGPT 3.5 studies' references produced three clusters.
  5. For the four ChatGPT 4.0 studies, co-citation analysis produced two clusters; the most cited works (Kung et al., Aidan et al., Hopkins et al.) intersect in one cluster.
  6. The authors say combining meta-analysis with co-citation analysis adds depth and scope, and that their results agree with the existing literature.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Health sciences exam correct answers: ChatGPT 4.0 compared with students

    Odds ratio 2.589 (95% CI 1.046 to 6.409)

    search to May 2024

    The review’s own words
    Specifically, students correctly answered 1214 out of 2123 questions, while ChatGPT 4.0 answered 1665 questions correctly (OR = 2.589, 95 % CI [1.046–6.409]).

  • Health sciences exam correct answers: students compared with ChatGPT 3.5

    Odds ratio 3.173 (95% CI 1.132 to 8.869)

    search to May 2024

    The review’s own words
    Specifically, out of a total of 924 questions, students answered 716 correctly, while ChatGPT 3.5 correctly answered 590 questions (OR = 3.173, 95 % CI [1.132–8.869]).

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The study included only research that compared students' exam performance directly with ChatGPT 3.5 or 4.0 in health sciences. The pooled results run in opposite directions: students scored better than the older model, but the newer model scored better than students.

Co-citation analysis examined the references of the included studies. For ChatGPT 3.5 it identified three clusters. For ChatGPT 4.0 it identified two clusters across the references of four studies, with Kung et al., Aidan et al. and Hopkins et al. intersecting in one.

The authors conclude that the findings agree with the literature for both versions and that using ChatGPT as an educational assistant in health sciences is increasing day by day.

Sources

1 on file
  1. Heliyondoi:10.1016/j.heliyon.2025.e41925
    Peer-reviewed2025-01-25

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Review of health-science exam studies shows GPT-4 beat students, GPT-3.5 did nothttps://truvace.com/item/can-chatgpt-be-used-as-an-education-assistant-in-the-field-of-health-sciences-ex

  1. Who gets this benefit, and who was left out of the study or story?

    The gain A meta-analysis of studies comparing ChatGPT with students on health sciences exams found ChatGPT 4.0 performed better than students (OR = 2.589, 95% CI 1.046-6.409).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis, from Heliyon.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat The ChatGPT 4.0 result is pooled from only four studies, and its confidence interval (1.046-6.409) is wide, so the size of the advantage over students is uncertain.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Education