Truace
Subscribe

Readability of hand surgery patient education materials from the ASSH website versus AI-generated versions

Source article: Comparative Analysis of Readability Levels of Hand Surgery Patient Information on the ASSH Website and Artificial Intelligence

Background The American Medical Association (AMA) recommends patient education materials (PEMs) be written at a sixth-grade level to ensure accessibility. This study aims to compare hand surgery topics from the American Society for Surgery of the Hand (ASSH) website and artificial intelligence (AI)-generated PEMs to evaluate their alignment with the AMA's recommendations. Methods Ninety-eight hand surgery topics from the ASSH Hand Care "The Upper Extremity Expert" website were analyzed. Artificial intelligence (…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1734· cite & verify
Comparative Analysis of Readability Levels of Hand Surgery Patient Information on the ASSH Website and Artificial Intelligence
Reading Wikipedia in the Classroom for Secondary School Students 10 by James Rhoda. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0
Trace impact readingContested
P 70The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 68The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Researchers analyzed 98 hand surgery topics from the American Society for Surgery of the Hand patient website and compared them with materials generated by Google Gemini 2.0 Flash and ChatGPT-4. The AI tools were asked to write new materials, rewrite the ASSH topics at a sixth-grade level, and write new materials at that level, with readability scored using Flesch measures.

The findings matter because patient materials that are too hard to read can undermine informed decision-making, and the American Medical Association recommends a sixth-grade level. The study shows AI can lower reading difficulty when prompted, but the authors say the accuracy of AI-written content and its fit for patients with varying health literacy still need to be established.

Main points

  1. The original ASSH materials ranged from 10th to 12th-grade readability, with a mean FKGL of 9.4.
  2. Without a reading-level instruction, Gemini and ChatGPT outputs were at college level, with mean FKGL scores of 9.6 and 10.4.
  3. AI-rewritten ASSH content showed improved readability compared with the originals.
  4. AI materials created at a sixth-grade target had fewer words and better Flesch scores than the ASSH originals.
  5. The authors call for future work on the accuracy of AI-generated materials and on accessibility for patients with varying health literacy.

The gain

When asked to target a sixth-grade level, Gemini 2.0 Flash and ChatGPT-4 produced hand surgery patient materials that were easier to read and shorter than the original ASSH website content.

The problem

Default AI-generated patient materials and the existing ASSH materials were written above the recommended sixth-grade level, with AI's initial output at college level, which limits accessibility for patients.

The rundown

The study used the ASSH Hand Care website, called The Upper Extremity Expert, as its baseline. Its 98 topics were scored with Flesch Reading Ease and Flesch-Kincaid Grade Level, and the original materials averaged an FRE of 57.5 and an FKGL of 9.4.

Three prompt conditions were tested on both AI platforms: creating original materials from ASSH topics, rewriting ASSH topics at a sixth-grade level, and creating original materials at a sixth-grade level. Initial Gemini output averaged an FRE of 49.7 and an FKGL of 9.6, while ChatGPT averaged an FRE of 47.4 and an FKGL of 10.4.

The authors conclude that AI can serve as a tool for producing hand surgery patient materials at the recommended level, while stressing that the accuracy of such materials must be ensured.

What this doesn’t fix

The study measured readability only, and the authors state that the accuracy of AI-generated patient materials and their accessibility to patients with varying health literacy still need to be ensured.

Sources

  1. Peer-reviewedHAND2026-10-08

The debate