Moderate evidence
Published recordEducationGood Space
More
Using Artificial Intelligence to Evaluate the Quality of Narrative Feedback in Surgical Entrustable Professional Activities
- Truvace record
- TRV-2026-3033raw data (JSON)
- Version
- v1Published
- First published
- Evidence type
- Observational study
- Sources
- 1 sourcePeer-reviewed
- sha256
- f5d62d00af8f…check it
Background: Narrative feedback quality in surgical Entrustable Professional Activity (EPA) assessments is variable, limiting consistent evaluation and opportunities for improvement. We evaluated whether fine-tuned large language models (LLMs) could reproduce expert-assigned feedback quality scores and whether a simplified scoring rubric improved assessment consistency. Study design: This observational study evaluated 1,509 narrative comments from EPA micro-assessments in a general surgery residency program durin…
The gain
In an observational study of 1,509 surgical residency feedback comments, fine-tuned encoder language models matched expert quality scores with accuracy of 0.93 for evidence, 0.96 for suggestion and 0.95 for connection.
What this doesn’t fix
The observational study used comments from a single general surgery residency program, and the authors call only for further evaluation of AI-assisted feedback assessment in residency programs.
Sources
1 on file- Journal of the American College of Surgeonsdoi:10.1097/xcs.0000000000002297
Cite this record
v1Truvace Impact Record TRV-2026-3033, v1: “Using Artificial Intelligence to Evaluate the Quality of Narrative Feedback in Surgical Entrustable Professional Activities.” Truvace, 2026-11-18. /record/TRV-2026-3033 (accessed on the day you copy it). sha256 f5d62d00af8fb959…
Change history
1 versionEvery change made to this record since it was first published, and why. Nothing has changed since it went up.
Certified into the record
Check this record hasn’t been altered
How to check it yourself, without trusting this page
Download any version from /api/record/TRV-2026-3033 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.
ace