Developing validity arguments for artificial intelligence-based assessment: Balancing affordances and threats
Background Artificial intelligence (AI) is increasingly used to generate, score and interpret educational assessment, yet these applications are being adopted in a largely unregulated environment. This creates a paradox: Whereas AI systems used in clinical care are subject to formal scrutiny for safety, performance and monitoring, AI systems used to inform consequential decisions about learner progression and future clinical practice are not. Existing validity frameworks remain useful but may not fully account f…
AI systems can generate, score and interpret educational assessments that inform learner progression, with design and governance determining whether cross-cutting mechanisms function as affordances.
AI-based assessment introduces distinct validity threats across scoring, generalisation, extrapolation and implications, including contamination, instability, inequities, automation bias and deskilling when used for consequential learner progression decisions.
Findings derive from a conceptual review synthesizing literature rather than new empirical multi-site validation of an AI assessment system.
Evidence
- Peer-reviewedMedical Education2026-09-08
How should this claim be treated?
Truvace Impact Record TRV-2026-1026, v1: “Developing validity arguments for artificial intelligence-based assessment: Balancing affordances and threats.” Truvace, 2026-09-09. /record/TRV-2026-1026 (accessed at citation time). sha256 3b03b3e18455a5a6…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-1026 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.
ace