Assessment Design Needs: A Preliminary Study on Using Generative Artificial Intelligence Tools for Educators

Assessment is considered as a key feature and central pillar in the teaching and learning process, particularly in making of judgement and measurement of student achievement and mastery of learning outcomes. The traditional methods of designing assessments items face several challenges, such as the need and requirement to design the assessment items manually, the time-consuming nature of the design process and the required cognitive skills through formal training provided by various institutions. As the result,…

By the Truvace deskOctober 21, 20262 min read1 sourcePeer-reviewedTRV-2026-2090· cite & verify

Assessment Design Needs: A Preliminary Study on Using Generative Artificial Intelligence Tools for Educators — "DSCF1183" by YHA CONWY is licensed under CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/.

In brief

A preliminary analysis-phase study surveyed forty-six educators about their understanding of Outcome-Based Education, assessment literacy, and generative AI literacy. It found that most respondents had low levels of generative AI tool use and prompt engineering skill.

Use was concentrated in general teaching and learning and information searching, while assessment design was where educators struggled most. This matters because traditional manual assessment design is slow and can yield items lacking validity, reliability and consistent scoring. The sample is small, and the study does not test whether AI tools actually improve assessment quality.

Main points

Forty-six educators took part in the preliminary study.
43.5% reported using generative AI for teaching and learning, and 34.8% for searching information.
Writing assessment test items was the most difficult task at 28.3%, followed by practical tests at 21.7%.
Presentations were cited as difficult by 17.4% and projects by 10.9%.
The authors propose a standardized guideline to incorporate AI ethically in assessment.

The problem

Educators in a 46-person preliminary survey showed low generative AI and prompt engineering skills, and found assessment test items and practical tests among the hardest tasks to design.

The rundown

The study sets out from the view that assessment is a central pillar of teaching and learning, used to judge student achievement and mastery of learning outcomes. Traditional item design is manual, time-consuming, and requires formal training, which can leave items lacking context-specific validity and reliability.

Survey results showed other uses were small: idea collection at 6.5%, reflection or journaling at 4.3%, writing programming code at 4.3%, image generation at 4.3%, and other tasks at 2.2%. Simulations (6.5%) and fieldwork (5.4%) were the least frequently cited difficult tasks.

The authors conclude that educators lack skills in designing assessment materials with generative AI. They suggest a guideline as an innovative assessment tool that would standardize practice and support ethical use of AI.