TRV-2026-2984Version 1 · Published

Saved · see the current record

Why this version exists

Certified into the record

The exact text we saved

This is the version exactly as it was saved, character for character. The SHA-256 hash below is a 64-character code worked out from that text. Run the same text through SHA-256 and you get the same code; change a single letter and the code comes out completely different. Each version also stores the previous version’s hash, which chains them together so no version can be swapped out quietly.

TRUVACE RECORD VERSION
record: TRV-2026-2984
version: 1
kind: certified
reason: Certified into the record
timestamp: 2026-11-17T06:02:20.836872Z
status: published
lens: g_space
sector: health
headline: The Use of Artificial Intelligence and Its Reliability in Grading Drug Information Responses
dek: Objective: To compare the reliability of two artificial intelligence (AI) models and pharmacy faculty in grading drug information (DI) responses.
Methods: We retrospectively collected 55 deidentified DI responses written by second-year pharmacy students. A 32-point grading rubric was developed and five faculty graders completed two calibration sessions to ensure grading consistency. Each DI response was graded by three randomly assigned faculty and two AI models (OpenAI ChatGPT 5.2 and Google Gemini 2.5 Flash).…
gain_title: In a study of 55 pharmacy student drug information responses, ChatGPT 5.2 (ICC .90) and Gemini 2.5 Flash (ICC .72) graded more consistently than faculty graders (ICC .39) using the same 32-point rubric.
problem_title: (none)
trace_subject: (none)
gain_reading: In a study of 55 pharmacy student drug information responses, ChatGPT 5.2 (ICC .90) and Gemini 2.5 Flash (ICC .72) graded more consistently than faculty graders (ICC .39) using the same 32-point rubric.
gain_evidence: Reliability was significantly higher for ChatGPT (ICC=.90) and Gemini (ICC=.72) compared to faculty evaluators (ICC=.39).
problem_reading: (none)
problem_evidence: (none)
quick_read: A study in the American Journal of Pharmaceutical Education compared two AI models with pharmacy faculty on grading 55 deidentified drug information responses written by second-year pharmacy students. ChatGPT 5.2 reached an intraclass correlation of .90 and Gemini 2.5 Flash reached .72, against .39 for faculty.

Each response was graded by three randomly assigned faculty and by both AI models on a 32-point rubric. Faculty had completed two calibration sessions beforehand, yet their agreement with one another stayed lower than that of the AI models.

Median rubric scores were 25.9 points for faculty, 24.2 for ChatGPT and 24.7 for Gemini. Both AI models graded slightly harsher, by 1.2 to 1.7 points, about 5% of the rubric, and showed no proportional bias across essay quality.
limitation: The study analysed only 55 responses from second-year pharmacy students, graded against a single rubric by five faculty, and both AI models scored slightly lower than faculty.
tag: Evidence-backed gain
key_points: The study used 55 retrospectively collected, deidentified drug information responses from second-year pharmacy students. | A 32-point rubric was developed, and five faculty graders completed two calibration sessions; each response was graded by three of them. | The AI graders were OpenAI's ChatGPT 5.2 and Google's Gemini 2.5 Flash. | Reliability was ICC .90 for ChatGPT, .72 for Gemini and .39 for faculty. | Against the mean faculty score, both models had a mean absolute error of 2.1 points. | Median scores did not differ between the two AI models. | Neither model showed proportional bias, and grading differences were spread evenly regardless of essay quality.
rundown: Researchers built a 32-point rubric for drug information responses and had five faculty graders complete two calibration sessions to align their scoring. Each of the 55 student responses was then scored by three randomly assigned faculty and by both AI models.

Reliability was measured with intraclass correlation, and the mean faculty score served as a consensus benchmark for mean absolute error. Both ChatGPT and Gemini differed from that benchmark by 2.1 points on average.

The AI models scored lower than faculty: median 24.2 for ChatGPT and 24.7 for Gemini versus 25.9 for faculty. The authors describe this as about a 5% difference on the rubric, and Spearman's correlation found no proportional bias.
sources:
- peer_reviewed | American Journal of Pharmaceutical Education | https://doi.org/10.1016/j.ajpe.2026.102124 | 2026-11-16
prev: 0000000000000000000000000000000000000000000000000000000000000000
SHA-256 hash
fe8cd196eb928ed53f0a06b53fd8959570097e162bc53066122d60c758e65bec
Previous version’s hash
0000000000000000000000000000000000000000000000000000000000000000
Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-2984 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.