63/100

Moderate evidence

GainPublished recordEducationGood Space

Using Artificial Intelligence to Evaluate the Quality of Narrative Feedback in Surgical Entrustable Professional Activities

Truvace record
TRV-2026-3033raw data (JSON)
Version
v1Published
First published
Evidence type
Observational study
Sources
1 sourcePeer-reviewed
sha256
f5d62d00af8f…check it

Background: Narrative feedback quality in surgical Entrustable Professional Activity (EPA) assessments is variable, limiting consistent evaluation and opportunities for improvement. We evaluated whether fine-tuned large language models (LLMs) could reproduce expert-assigned feedback quality scores and whether a simplified scoring rubric improved assessment consistency. Study design: This observational study evaluated 1,509 narrative comments from EPA micro-assessments in a general surgery residency program durin…

The gain

In an observational study of 1,509 surgical residency feedback comments, fine-tuned encoder language models matched expert quality scores with accuracy of 0.93 for evidence, 0.96 for suggestion and 0.95 for connection.

What this doesn’t fix

The observational study used comments from a single general surgery residency program, and the authors call only for further evaluation of AI-assisted feedback assessment in residency programs.

Sources

1 on file
  1. Journal of the American College of Surgeonsdoi:10.1097/xcs.0000000000002297
    Peer-reviewed2026-11-18

Cite this record

v1

Truvace Impact Record TRV-2026-3033, v1: “Using Artificial Intelligence to Evaluate the Quality of Narrative Feedback in Surgical Entrustable Professional Activities.” Truvace, 2026-11-18. /record/TRV-2026-3033 (accessed on the day you copy it). sha256 f5d62d00af8fb959…

Change history

1 version

Every change made to this record since it was first published, and why. Nothing has changed since it went up.

  1. Publishedv1f5d62d00af8f…

    Certified into the record

Check this record hasn’t been altered

Check this record hasn’t been altered
How to check it yourself, without trusting this page

Download any version from /api/record/TRV-2026-3033 and hash its canonical field yourself, for example with shasum -a 256. You should get its content_hash. Each version’s text also ends with prev:and the hash of the version before it (version 1 points to 64 zeros). If anyone had changed a single character of any version after it was saved, those hashes wouldn’t line up.