Evaluation of an Artificial Intelligence-Driven Language Model for Age-Specific Patient Education in Pediatric Orthopaedics

Background: Describing complex orthopaedic surgical procedures and diagnoses to pediatric patients across a large age range and at varying levels of health literacy can be quite challenging. Generative artificial intelligence (GenAI) offers a potential solution by providing real-time age specific explanations for pediatric patients. The use of GenAI large language models (LLMs) could potentially allow providers to more effectively communicate complex medical problems in an age-appropriate and comprehensible mann…

By the Truvace deskOctober 16, 20262 min read1 sourcePeer-reviewedTRV-2026-1604· cite & verify

In brief

Researchers prompted ChatGPT-4 to explain 43 pediatric orthopaedic procedures and tests for imagined patients aged 6, 10 and 14. Six quantitative readability indices scored the outputs, and seven board-certified pediatric orthopaedic surgeons rated accuracy and comfort, comparing the text with their own clinical explanations.

The findings suggest a language model could help clinicians explain complex care to children with differing health literacy, supporting shared decision-making. The evidence is a physician review of generated text rather than a trial with real patients, so effects on children's understanding or outcomes remain untested.

Main points

All six readability indices differed significantly across the 6-, 10- and 14-year-old prompts (P < .001).
Younger-age prompts yielded the most readable text, and 14-year-old prompts the least readable.
Mean attending-rated accuracy was 4.15 out of 5, with no significant differences by procedure (P = .051) or age group (P = .27).
Mean attending-rated comfort was 3.95, with inter-rater reliability of 0.92 for accuracy and 0.94 for comfort.

The gain

ChatGPT-4 produced explanations that became progressively less readable for older age prompts, and seven pediatric orthopaedic surgeons rated their accuracy highly and felt comfortable using them in clinical settings.

The rundown

The study tested ChatGPT-4 on 43 pediatric orthopaedic procedures and tests, with each one prompted for three ages: 6, 10 and 14 years. This design allowed comparison of how the model adjusted its language across a wide developmental range.

Readability was measured with six quantitative indices, and every index showed significant separation between the age groups. Seven board-certified pediatric orthopaedic surgeons separately scored accuracy and comprehensiveness and compared the output with the explanations they use in practice.

The authors conclude that, when reviewed by a physician, generative AI could be a helpful tool for patient education, health literacy and shared decision-making. The study is labeled level of evidence III/IV.