70/100

Moderate evidence

ProblemPublished recordEducationProblem Space

Large language model-generated vs. human-authored assessment items in education: systematic review and meta-analysis with a focus on health professional education

Truvace record
TRV-2026-3058raw data (JSON)
Version
v1Published
First published
Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
sha256
3b33ce8b3038…check it

Purpose Assessment is crucial in health professional education (HPE) but is resource-intensive to create. Large language models (LLMs) offer a potential solution by generating assessment items; however, a comprehensive synthesis of their psychometric quality is lacking in HPE. Design/methodology/approach We systematically searched the electronic databases and included studies generating assessment items using any LLM and reporting psychometric outcomes. The primary outcomes were item difficulty index, discrimina…

The problem

A systematic review and meta-analysis of 54 studies found AI-generated assessment items were easier than human-written ones (difficulty index g 0.40) and had higher odds of containing non-functioning distractors (OR 1.86).

What this doesn’t fix

The certainty of the evidence ranged between very low and moderate, so the pooled differences between AI and human items carry real uncertainty.

Sources

1 on file
  1. Peer-reviewed2026-11-18

Cite this record

v1

Truvace Impact Record TRV-2026-3058, v1: “Large language model-generated vs. human-authored assessment items in education: systematic review and meta-analysis with a focus on health professional education.” Truvace, 2026-11-19. /record/TRV-2026-3058 (accessed on the day you copy it). sha256 3b33ce8b30386ef8…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv13b33ce8b3038…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3058 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.