Moderate evidence
Published recordHealthGood Space
More
The Use of Artificial Intelligence and Its Reliability in Grading Drug Information Responses
- Truvace record
- TRV-2026-2984raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Controlled or validated comparison
- Sources
- 1 sourcePeer-reviewed
- sha256
- fe8cd196eb92…check it
Objective: To compare the reliability of two artificial intelligence (AI) models and pharmacy faculty in grading drug information (DI) responses. Methods: We retrospectively collected 55 deidentified DI responses written by second-year pharmacy students. A 32-point grading rubric was developed and five faculty graders completed two calibration sessions to ensure grading consistency. Each DI response was graded by three randomly assigned faculty and two AI models (OpenAI ChatGPT 5.2 and Google Gemini 2.5 Flash).…
The gain
In a study of 55 pharmacy student drug information responses, ChatGPT 5.2 (ICC .90) and Gemini 2.5 Flash (ICC .72) graded more consistently than faculty graders (ICC .39) using the same 32-point rubric.
What this doesn’t fix
The study analysed only 55 responses from second-year pharmacy students, graded against a single rubric by five faculty, and both AI models scored slightly lower than faculty.
Sources
1 on file- American Journal of Pharmaceutical Educationdoi:10.1016/j.ajpe.2026.102124
Cite this record
v1Truvace Impact Record TRV-2026-2984, v1: “The Use of Artificial Intelligence and Its Reliability in Grading Drug Information Responses.” Truvace, 2026-11-17. /record/TRV-2026-2984 (accessed on the day you copy it). sha256 fe8cd196eb928ed5…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-2984 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace