Moderate evidence
Published recordEducationProblem Space
More
Large language model-generated vs. human-authored assessment items in education: systematic review and meta-analysis with a focus on health professional education
- Truvace record
- TRV-2026-3058raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Systematic review or meta-analysis
- Sources
- 1 sourcePeer-reviewed
- sha256
- 3b33ce8b3038…check it
Purpose Assessment is crucial in health professional education (HPE) but is resource-intensive to create. Large language models (LLMs) offer a potential solution by generating assessment items; however, a comprehensive synthesis of their psychometric quality is lacking in HPE. Design/methodology/approach We systematically searched the electronic databases and included studies generating assessment items using any LLM and reporting psychometric outcomes. The primary outcomes were item difficulty index, discrimina…
The problem
A systematic review and meta-analysis of 54 studies found AI-generated assessment items were easier than human-written ones (difficulty index g 0.40) and had higher odds of containing non-functioning distractors (OR 1.86).
What this doesn’t fix
The certainty of the evidence ranged between very low and moderate, so the pooled differences between AI and human items carry real uncertainty.
Sources
1 on file- Journal of Applied Research in Higher Educationdoi:10.1108/jarhe-05-2026-0919
Cite this record
v1Truvace Impact Record TRV-2026-3058, v1: “Large language model-generated vs. human-authored assessment items in education: systematic review and meta-analysis with a focus on health professional education.” Truvace, 2026-11-19. /record/TRV-2026-3058 (accessed on the day you copy it). sha256 3b33ce8b30386ef8…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-3058 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace