Moderate evidence
More
GPT-4 caught more surgical complications in 1,454 spine operation notes than human reviewers, study finds
Study title: GPT-4-Assisted Versus Human Detection of Intraoperative Adverse Events From Spine Surgery Operative Reports
- Evidence type
- Controlled or validated comparison
- Sample
- n = 1,454
- Sources
- 1 sourcePeer-reviewed
- Published
- 2 min read
- Truvace record
- TRV-2026-3080cite or check it
Study title: GPT-4-Assisted Versus Human Detection of Intraoperative Adverse Events From Spine Surgery Operative Reports
A retrospective study of 1,454 laminectomy cases at one academic institution, performed between 2016 and 2022, compared GPT-4 with four trained human reviewers at spotting intraoperative adverse events in operative notes. For incidental durotomy, which occurred in 88 cases (6.1%), GPT-4 found every one (100.0% sensitivity) while humans found 89.8%.
Read the study abstract
Background and objectives: Manual chart review remains the primary method of clinical data collection in spine surgery research but is time intensive and prone to human error. Large language models may enable automated extraction of clinically meaningful events from unstructured operative text. We evaluated the performance of generative pre-trained transformer 4 (GPT-4) in detecting intraoperative adverse events from spine surgery operative notes compared with trained human reviewers. Methods: A retrospective co…

The gain
In a retrospective study of 1,454 laminectomy operative notes, GPT-4 detected 100% of incidental durotomies versus 89.8% for human reviewers and 90.5% of other adverse events versus 57.1%, in about 2 hours versus 35.
In brief
GPT-4 also did better on other adverse events (21 cases), with 90.5% sensitivity against 57.1% for humans. Human sensitivity fell on longer notes, whereas GPT-4 stayed stable across note-length quartiles.
Review took humans about 35 hours for all notes and GPT-4 about 2 hours, a cut of more than 90%. The authors describe the approach as a human-in-the-loop adjunct to chart review rather than a replacement.
Main points
- The study analysed 1,454 laminectomy cases from a single academic institution between 2016 and 2022.
- Four trained human reviewers and GPT-4 independently read deidentified operative notes using standardized prompts.
- A senior reviewer adjudicated discordant classifications to set the ground truth.
- GPT-4 had 100.0% sensitivity for incidental durotomy versus 89.8% for humans, but positive predictive value of 88.0% versus 100.0%.
- Specificity for durotomy was 99.1% for GPT-4 and 100.0% for humans.
- For nondurotomy adverse events (21 cases), GPT-4 sensitivity was 90.5% versus 57.1% for humans, with equal 100.0% specificity.
- Human sensitivity declined on longer notes while GPT-4 held steady across note-length quartiles.
- Review time fell from about 35 hours to about 2 hours.
The full story
Manual chart review is the main way spine surgery researchers collect clinical data, but it is slow and prone to human error. The team tested whether GPT-4 could extract adverse events from unstructured operative notes.
Both humans and GPT-4 used standardized prompts on deidentified notes. Discordant calls went to a senior reviewer, and the team calculated sensitivity, specificity, predictive values and F1 score, and stratified results by note length.
GPT-4 flagged all 88 incidental durotomies but produced more false positives, with positive predictive value of 88.0% against 100.0% for humans. On the 21 other adverse events it found 90.5% against 57.1%.
The authors conclude that model-assisted review could make neurosurgical outcomes research more complete and efficient when paired with human oversight.
Sources
1 on file- Neurosurgery Practicedoi:10.1227/neuprac.0000000000000333
Talk it over
GPT-4 found all 88 incidental durotomies but had a positive predictive value of 88.0% against 100.0% for humans. How do you weigh missed events against false alarms in operative notes?
GPT-4 caught more surgical complications in 1,454 spine operation notes than human reviewers, study findshttps://truvace.com/item/gpt-4-assisted-versus-human-detection-of-intraoperative-adverse-events-from-spin
GPT-4 found all 88 incidental durotomies but had a positive predictive value of 88.0% against 100.0% for humans. How do you weigh missed events against false alarms in operative notes?
Answer in the threadThe 1,454 laminectomy notes came from one academic institution, and a senior reviewer settled disagreements to set the ground truth. How might that design affect what we learn about GPT-4 versus the four trained reviewers?
Answer in the threadIf you were the senior reviewer adjudicating the disagreements between GPT-4 and the human reviewers, what would you want to know about each flagged event before deciding?
Answer in the thread
If you have more time
Human sensitivity fell on longer notes while GPT-4 held steady. What would you look up to find out whether other kinds of surgery or other hospitals show the same pattern?
Answer in the threadReview time fell from about 35 hours to about 2 hours. Who should decide how the time saved is used, and what checks should remain in place?
Answer in the thread
ace
What readers say