Truace
Subscribe

AI chatbot abstracted surgical records with 99.33% accuracy versus 98.19% for human reviewers

Study title: Large Language Model Data Abstraction Demonstrates Accuracy and Reliability for NSQIP

Background National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast reconstruction variables compared with conventional human abstraction.

Read the study abstract

Background National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast reconstruction variables compared with conventional human abstraction. Study design Clinical notes from patients enrolled in the NSQIP Breast Reconstruction pilot program (July 1, 2024-February 28, 2…

By the Truvace desk1 min read1 sourceEvidence: Moderate · 68/100 — Controlled or validated comparison · 1 independent sourcePeer-reviewedTRV-2026-1141· cite & verify
Large Language Model Data Abstraction Demonstrates Accuracy and Reliability for NSQIP
Abstraction in the INTEL iAPX-432 prototype systems implementation language by MacLennan, Bruce J.. Public domain

In brief

Study design Clinical notes from patients enrolled in the NSQIP Breast Reconstruction pilot program (July 1, 2024-February 28, 2025) were manually de-identified and processed using a customized ChatGPT 4.1 workflow targeting individual variables. Overall accuracy of LLM and human abstraction was compared using McNemar's and Chi-square tests.

Main points

  1. Background National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling.
  2. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast reconstruction variables compared with conventional human abstraction.
  3. Study design Clinical notes from patients enrolled in the NSQIP Breast Reconstruction pilot program (July 1, 2024-February 28, 2025) were manually de-identified and processed using a customized ChatGPT 4.1 workflow targeting individual variables.

The gain

A customized ChatGPT 4.1 workflow abstracted NSQIP surgical records with 99.33% accuracy versus 98.19% for human abstraction, beating the human baseline.

The rundown

Background National Surgical Quality Improvement Program (NSQIP) data collection depends on labor-intensive manual chart abstraction, limiting efficiency, increasing cost, and necessitating patient sampling. This study evaluated whether a large language model (LLM) could accurately abstract unstructured NSQIP breast reconstruction variables compared with conventional human abstraction.

Study design Clinical notes from patients enrolled in the NSQIP Breast Reconstruction pilot program (July 1, 2024-February 28, 2025) were manually de-identified and processed using a customized ChatGPT 4.1 workflow targeting individual variables. A faculty plastic surgeon established the reference standard.

Sources

  1. Peer-reviewedJournal of the American College of Surgeons2026-09-18

The debate