Certified into the record
The exact text we saved
This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.
TRUVACE RECORD VERSION record: TRV-2026-3058 version: 1 kind: certified reason: Certified into the record timestamp: 2026-11-19T18:02:16.936883Z status: published lens: p_space sector: education headline: Large language model-generated vs. human-authored assessment items in education: systematic review and meta-analysis with a focus on health professional education dek: Purpose Assessment is crucial in health professional education (HPE) but is resource-intensive to create. Large language models (LLMs) offer a potential solution by generating assessment items; however, a comprehensive synthesis of their psychometric quality is lacking in HPE. Design/methodology/approach We systematically searched the electronic databases and included studies generating assessment items using any LLM and reporting psychometric outcomes. The primary outcomes were item difficulty index, discrimina… gain_title: (none) problem_title: A systematic review and meta-analysis of 54 studies found AI-generated assessment items were easier than human-written ones (difficulty index g 0.40) and had higher odds of containing non-functioning distractors (OR 1.86). trace_subject: (none) gain_reading: (none) gain_evidence: (none) problem_reading: A systematic review and meta-analysis of 54 studies found AI-generated assessment items were easier than human-written ones (difficulty index g 0.40) and had higher odds of containing non-functioning distractors (OR 1.86). problem_evidence: LLM-generated items had a significantly higher difficulty index (pooled g: 0.40, 95% CI: 0.23 to 0.57), indicating they were easier and had significantly higher odds of containing NFDs (OR: 1.86, 95% CI: 1.27 to 2.73) quick_read: A systematic review and meta-analysis pooled 54 studies that used large language models to write assessment items and reported psychometric results, mostly in health professional education. In direct comparisons with human-written items, the AI items were significantly easier and more often contained non-functioning distractors, meaning wrong answer options that test-takers rarely pick. The pooled difficulty index was higher for AI items (g 0.40) and the odds of non-functioning distractors were 1.86 times those of human items. Discrimination index, which shows how well an item separates strong from weak students, did not differ significantly (g -0.21), but distractor effectiveness was poorer (g -0.65). When AI items were assessed on their own, 67% were accurate or needed only minor revisions, yet 37% of distractors were non-functioning. Rankings across all main psychometric measures placed human-authored items first, and the authors say expert review and careful prompting are needed to produce valid, reliable tests. limitation: The certainty of the evidence ranged between very low and moderate, so the pooled differences between AI and human items carry real uncertainty. tag: Evidence-backed problem key_points: The review included 54 studies, and 92.8% of those with direct comparisons were in health professional education. | AI-generated items had a higher difficulty index than human items (pooled g 0.40, 95% CI 0.23 to 0.57), meaning they were easier. | The odds of containing non-functioning distractors were 1.86 times higher for AI items (95% CI 1.27 to 2.73). | Discrimination index showed no significant difference (g -0.21, 95% CI -0.49 to 0.07), while distractor effectiveness was poorer for AI items (g -0.65). | Assessed alone, 67% of AI items were accurate or needed only minor edits, but 37% of their distractors were non-functioning. | Surface under the cumulative ranking curve analyses ranked human-generated items superior on every critical psychometric parameter. rundown: Researchers searched electronic databases for studies in which any large language model generated assessment items and psychometric outcomes were reported. The primary outcomes were item difficulty index, discrimination index, distractor effectiveness and the risk of non-functioning distractors, pooled with random-effects models. In head-to-head comparisons, AI items were easier and had more flawed answer options than human-written ones, though they separated strong from weak test-takers about as well. Ranking analyses consistently favoured human-authored items. The authors conclude that AI can produce assessment items efficiently and that many are usable with minor edits, but that sophisticated prompt engineering and expert-led review are essential for valid and reliable assessments in health professional education. sources: - peer_reviewed | Journal of Applied Research in Higher Education | https://doi.org/10.1108/jarhe-05-2026-0919 | 2026-11-18 prev: 0000000000000000000000000000000000000000000000000000000000000000
- SHA-256 hash
- 3b33ce8b30386ef8364ec6ed303e36e67ae8ae07e81b9e6288fd0730cf7ba236
- Previous version’s hash
- 0000000000000000000000000000000000000000000000000000000000000000
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-3058 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace