Truace
Subscribe
70/100

Moderate evidence

ProblemPublished recordHealthProblem Space

Full story
More
Export record JSONEmailLinkedInX

Performance of ChatGPT in medical examinations: A systematic review and a meta‐analysis

Truvace record
TRV-2026-2852raw data (JSON)
Version
v1Published
First published
Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
sha256
a6f3798a6e57…check it

The use of ChatGPT, an artificial intelligence (AI) language model, has been described in various scientific and medical applications.1 With its human-like conversation capacity and large quantity of data, ChatGPT has the potential to become an important medical education tool.2 ChatGPT's performance in different medical knowledge examinations has been recently studied in various medical disciplines. However, reported rates of correct answers vary dramatically across different examinations and medical fields.3,…

The problem

A meta-analysis of 19 studies found ChatGPT 3.5 answered a mean 61.1% of medical multiple-choice exam questions correctly, roughly a passing grade and well below the 95% the authors say a reliable educational tool would need.

What this doesn’t fix

The review included only studies of ChatGPT version 3.5, the included studies were heterogeneous with the number of answer choices per exam unreported, and the authors say tool performance may be overestimated.

Sources

1 on file
  1. BJOG: An International Journal of Obstetrics & Gynaecologydoi:10.1111/1471-0528.17641
    Peer-reviewed2023-08-21

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • ChatGPT (version 3.5) correct-answer rate on medical multiple-choice examinations

    Percentage 61.1% (95% CI 56.1% to 66.0%)

    19 studies · search to 2 June 2023

    The review’s own words
    The mean performance of ChatGPT was 61.1% (95% CI 56.1%–66.0%

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

Cite this record

v1

Truvace Impact Record TRV-2026-2852, v1: “Performance of ChatGPT in medical examinations: A systematic review and a meta‐analysis.” Truvace, 2026-11-12. /record/TRV-2026-2852 (accessed on the day you copy it). sha256 a6f3798a6e5762bb…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv1a6f3798a6e57…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2852 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.