Truace
Subscribe
48/100

Emerging evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

AI chatbots pass Taiwan's lab scientist exam; students rated AI-made practice questions higher

Study title: Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

Evidence type
Benchmark or lab evaluation
Sample
n = 30
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2864cite or check it

Study title: Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

Researchers tested four AI models (GPT-3.5, GPT-4o, GPT-5.2 and Gemini 3 Pro) on six administrations of the Taiwanese Medical Laboratory Scientist Examination from 2021 to 2023. All four cleared the 60% passing threshold, with GPT-5.2 scoring 93.5%, Gemini 3 Pro 92.1%, GPT-4o 85.8% and GPT-3.5 67.1%.

Read the study abstract

The rapid evolution of generative artificial intelligence (GAI), particularly large language models (LLMs), offers transformative potential for health professions education. However, their performance in specialized clinical domains, such as medical laboratory science as well as educational utility, remain insufficiently characterized. This study evaluated multiple LLMs on the Taiwanese Medical Laboratory Scientist Examination (TMLSE) and investigated the effectiveness of AI-generated assessments in undergraduat…

The gain

In a study testing four AI models on six Taiwanese Medical Laboratory Scientist Examination administrations, GPT-5.2 scored 93.5% and Gemini 3 Pro 92.1%, and 30 students rated AI-generated items significantly higher than real exam questions for overall satisfaction (p=0.0089).

In brief

The team also had experts validate 180 AI-generated exam items and gave them to 30 undergraduate-year medical laboratory students. The students rated the AI items higher than real exam questions for overall satisfaction and for usefulness in exam review, and scores on the AI items tracked clinical internship outcomes.

Image interpretation remained the weak point: GPT-5.2 reached 73.3% on image-based items. Informational gaps accounted for more than 95% of wrong answers across all models.

Main points

  1. Four models were tested on six exam administrations from 2021 to 2023, across 10 disciplines plus image-based items.
  2. All four models passed the 60% threshold; GPT-3.5 scored lowest at 67.1%.
  3. GPT-5.2 (93.5%) and Gemini 3 Pro (92.1%) significantly outperformed GPT-4o (85.8%) and GPT-3.5 (67.1%).
  4. Informational deficiencies caused more than 95% of incorrect answers; the rest were logical or statistical errors.
  5. Students rated AI-generated items higher than real exam questions for satisfaction (p=0.0089) and exam-review utility (p=0.0405).
  6. Scores on AI-generated assessments correlated with clinical internship outcomes (Spearman's r=0.3662, p=0.0465); traditional exam scores showed no significant association.

The full story

The evaluation covered six administrations of Taiwan's national medical laboratory scientist exam. Wrong answers were sorted into informational, logical or statistical errors, and informational gaps dominated at more than 95% across all models.

For the education part, 180 AI-generated items were validated by experts and given to 30 undergraduate-year students. They rated these items significantly higher than authentic exam questions for overall satisfaction (p=0.0089) and usefulness for review (p=0.0405).

Performance on the AI-generated assessments was positively correlated with clinical internship outcomes (r=0.3662, p=0.0465), while traditional exam scores showed no significant association. The authors conclude that image interpretation remains a limitation for the models.

Sources

1 on file
  1. Journal of the Chinese Medical Associationdoi:10.1097/jcma.0000000000001445
    Peer-reviewed2026-11-12

Talk it over

Three questions to start on, two more if you have timeGPT-5.2 scored 93.5% on the exam, but reached 73.3% on image-based items, and informational gaps caused more than 95% of wrong answers. How would you weigh these results against each other?

Start with a question

AI chatbots pass Taiwan's lab scientist exam; students rated AI-made practice questions higherhttps://truvace.com/item/generative-ai-in-the-taiwan-medical-laboratory-scientist-examination-performance

  1. GPT-5.2 scored 93.5% on the exam, but reached 73.3% on image-based items, and informational gaps caused more than 95% of wrong answers. How would you weigh these results against each other?

    Answer in the thread

  2. Only 30 undergraduate-year students rated the 180 AI-generated items, and the link with clinical internship outcomes was r=0.3662. What can that sample tell us about AI-made practice questions?

    Answer in the thread

  3. If you were one of the 30 students, which would you rather study from: the AI-generated items or real exam questions, and why?

    Answer in the thread
If you have more time

  • Look up how the Taiwanese Medical Laboratory Scientist Examination is used in licensing. What would it mean if a chatbot could pass it?

    Answer in the thread

  • Experts validated the 180 AI-generated items. How would a school check such items at a larger scale, and who should do that checking?

    Answer in the thread

What readers say

More from Health