70/100

Moderate evidence

Cite
More
EmailLinkedInX

GPT-4 caught more surgical complications in 1,454 spine operation notes than human reviewers, study finds

Study title: GPT-4-Assisted Versus Human Detection of Intraoperative Adverse Events From Spine Surgery Operative Reports

Evidence type
Controlled or validated comparison
Sample
n = 1,454
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: GPT-4-Assisted Versus Human Detection of Intraoperative Adverse Events From Spine Surgery Operative Reports

A retrospective study of 1,454 laminectomy cases at one academic institution, performed between 2016 and 2022, compared GPT-4 with four trained human reviewers at spotting intraoperative adverse events in operative notes. For incidental durotomy, which occurred in 88 cases (6.1%), GPT-4 found every one (100.0% sensitivity) while humans found 89.8%.

Read the study abstract

Background and objectives: Manual chart review remains the primary method of clinical data collection in spine surgery research but is time intensive and prone to human error. Large language models may enable automated extraction of clinically meaningful events from unstructured operative text. We evaluated the performance of generative pre-trained transformer 4 (GPT-4) in detecting intraoperative adverse events from spine surgery operative notes compared with trained human reviewers. Methods: A retrospective co…

GPT-4-Assisted Versus Human Detection of Intraoperative Adverse Events From Spine Surgery Operative Reports
"US Army, Burundian surgeons work together in operating room" by 1st Lt. Katherine Sibilla, U.S. Army, public domain.

The gain

In a retrospective study of 1,454 laminectomy operative notes, GPT-4 detected 100% of incidental durotomies versus 89.8% for human reviewers and 90.5% of other adverse events versus 57.1%, in about 2 hours versus 35.

In brief

GPT-4 also did better on other adverse events (21 cases), with 90.5% sensitivity against 57.1% for humans. Human sensitivity fell on longer notes, whereas GPT-4 stayed stable across note-length quartiles.

Review took humans about 35 hours for all notes and GPT-4 about 2 hours, a cut of more than 90%. The authors describe the approach as a human-in-the-loop adjunct to chart review rather than a replacement.

Main points

  1. The study analysed 1,454 laminectomy cases from a single academic institution between 2016 and 2022.
  2. Four trained human reviewers and GPT-4 independently read deidentified operative notes using standardized prompts.
  3. A senior reviewer adjudicated discordant classifications to set the ground truth.
  4. GPT-4 had 100.0% sensitivity for incidental durotomy versus 89.8% for humans, but positive predictive value of 88.0% versus 100.0%.
  5. Specificity for durotomy was 99.1% for GPT-4 and 100.0% for humans.
  6. For nondurotomy adverse events (21 cases), GPT-4 sensitivity was 90.5% versus 57.1% for humans, with equal 100.0% specificity.
  7. Human sensitivity declined on longer notes while GPT-4 held steady across note-length quartiles.
  8. Review time fell from about 35 hours to about 2 hours.

The full story

Manual chart review is the main way spine surgery researchers collect clinical data, but it is slow and prone to human error. The team tested whether GPT-4 could extract adverse events from unstructured operative notes.

Both humans and GPT-4 used standardized prompts on deidentified notes. Discordant calls went to a senior reviewer, and the team calculated sensitivity, specificity, predictive values and F1 score, and stratified results by note length.

GPT-4 flagged all 88 incidental durotomies but produced more false positives, with positive predictive value of 88.0% against 100.0% for humans. On the 21 other adverse events it found 90.5% against 57.1%.

The authors conclude that model-assisted review could make neurosurgical outcomes research more complete and efficient when paired with human oversight.

Sources

1 on file
  1. Neurosurgery Practicedoi:10.1227/neuprac.0000000000000333
    Peer-reviewed2026-11-20

Talk it over

Three questions to start on, two more if you have time
GPT-4 found all 88 incidental durotomies but had a positive predictive value of 88.0% against 100.0% for humans. How do you weigh missed events against false alarms in operative notes?

Start with a question

GPT-4 caught more surgical complications in 1,454 spine operation notes than human reviewers, study findshttps://truvace.com/item/gpt-4-assisted-versus-human-detection-of-intraoperative-adverse-events-from-spin

  1. GPT-4 found all 88 incidental durotomies but had a positive predictive value of 88.0% against 100.0% for humans. How do you weigh missed events against false alarms in operative notes?

    Answer in the thread
  2. The 1,454 laminectomy notes came from one academic institution, and a senior reviewer settled disagreements to set the ground truth. How might that design affect what we learn about GPT-4 versus the four trained reviewers?

    Answer in the thread
  3. If you were the senior reviewer adjudicating the disagreements between GPT-4 and the human reviewers, what would you want to know about each flagged event before deciding?

    Answer in the thread
If you have more time
  • Human sensitivity fell on longer notes while GPT-4 held steady. What would you look up to find out whether other kinds of surgery or other hospitals show the same pattern?

    Answer in the thread
  • Review time fell from about 35 hours to about 2 hours. Who should decide how the time saved is used, and what checks should remain in place?

    Answer in the thread

What readers say