AI simplification of pediatric orthopaedic caregiver discharge documents

Source article: Readability and Content Integrity of Discharge Documents for Common Pediatric Orthopaedic Procedures Before and After Artificial Intelligence Simplification

Background: Caregivers of pediatric orthopaedic surgical patients routinely receive discharge documents written at reading levels exceeding established health literacy recommendations. This disparity creates a significant barrier to comprehension and adherence to postoperative instructions. While artificial intelligence (AI) language models offer a scalable solution for simplification of medical text, a systematic evaluation of the ability to preserve critical content in surgical discharge materials remains lack…

By the Truvace deskOctober 19, 20262 min read1 sourcePeer-reviewedTRV-2026-1916· cite & verify

Trace impact readingNegative state

P 74

Both sides are scored from claims and sources, not community votes.

G 63

In brief

Researchers collected 15 publicly available caregiver discharge documents from institutions in 9 US states, covering supracondylar humerus fractures, pediatric forearm fractures, and slipped capital femoral epiphysis pinning. They had ChatGPT simplify each with a standardized prompt and compared readability before and after using six validated indices. Two independent reviewers then checked each simplified document for errors.

Discharge instructions that are too hard to read can hinder caregivers' understanding of and adherence to postoperative care, so automated simplification is attractive. This study suggests it cannot be used unchecked, because the same process that eased reading also altered safety-critical content. The sample was small and limited to three conditions, so how the findings extend to other procedures or tools remains open.

Main points

Mean Flesch-Kincaid grade level fell by 3.94 grade levels (95% CI 2.65-5.23), with a large effect size (Cohen d=1.69).
All six readability indices improved across all fifteen documents, and 73% reached the FKGL 6 target versus none originally.
Review flagged 35 issues in 14 of 15 documents: 12 safety-critical errors, 16 completeness gaps, 7 accuracy concerns.
Greater word-count reduction correlated with more safety-critical flags (Spearman rho 0.63; P=0.012).
Reviewer agreement was high, with Cohen kappa of 0.85 to 0.89 across judgments.

The gain

Simplifying caregiver discharge documents with ChatGPT lowered mean reading level by about four grade levels, bringing 11 of 15 documents to a sixth-grade level or below.

The problem

The same simplified documents contained 35 content flags, including 12 safety-critical errors, and three needed clinician revision because of omissions before they could be used clinically.

The rundown

The documents came from institutions across 9 US states and covered three common procedures: closed reduction and percutaneous pinning for supracondylar humerus fractures, ORIF or CRPP for forearm fractures, and SCFE pinning. All were simplified with ChatGPT version 5.2 using one standardized prompt.

Baseline documents averaged a grade 8.91 reading level, above health literacy recommendations. After simplification the mean was 4.97, and none of the originals met the FKGL 6 or lower threshold versus 11 of 15 simplified versions.

Two independent reviewers assessed safety-critical errors, completeness gaps, and factual accuracy. Kappa was 0.87 for whether a change was clinically meaningful, 0.85 for severity, and 0.89 for flag category.

The authors conclude AI simplification should be used only with structured clinician review. Because larger word-count cuts were linked to more safety-critical alterations, the size of the reduction could help decide which documents get the closest review.

What this doesn’t fix

The study used only 15 documents covering three pediatric orthopaedic conditions and one model with a single standardized prompt, so results may not generalize; the source also classifies it as Level III evidence.