Truace
Subscribe

LLM-based Injury Severity Score calculation from free-text trauma discharge diagnoses

Source article: Automating Injury Severity Score Calculation From Free-Text Discharge Diagnoses Using Domain-Specific Retrieval-Augmented Large Language Models: Retrospective Validation Study

Background: Trauma registries are crucial for evaluating the prognosis of injuries and the quality of trauma care. However, the calculation of Injury Severity Score (ISS) remains a predominantly manual process, which is labor-intensive, potentially error-prone, and difficult to scale. Automating this process is a critical step toward enhancing the efficiency and quality of trauma registry data. Objective: This study conducts a feasibility test on the use of large language models (LLMs) to automatically calculate…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1356· cite & verify
Automating Injury Severity Score Calculation From Free-Text Discharge Diagnoses Using Domain-Specific Retrieval-Augmented Large Language Models: Retrospective Validation Study
Hospital Universitari Doctor Peset, València 11 by 19Tarrestnom65. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0
Trace impact readingContested
P 68The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 69The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Researchers tested whether large language models could calculate the Injury Severity Score from free-text discharge diagnoses of trauma patients at one Taiwanese hospital in 2022. They compared plain LLMs, LLMs with a standard retrieval system, and LLMs with a retriever fine-tuned on trauma diagnosis codes, scoring all of them against physician-verified answers.

The work matters because manual scoring for trauma registries is slow, error-prone and hard to scale, so automation could ease administrative burden and improve registry data. Results come from a single center and a retrospective test set, and the authors say multi-institution validation is still needed before the approach can be trusted more widely.

Main points

  1. Baseline LLMs reached exact-match accuracy of 47.3% to 64.7% on Injury Severity Score calculation.
  2. Standard retrieval augmentation produced inconsistent results, ranging from 54.3% to 60.3%.
  3. The domain-specific retriever, fine-tuned on trauma ICD-10 codes, lifted accuracy to between 62.7% and 75.3%.
  4. Gains were most pronounced in patients with moderate to severe injuries.
  5. The best system, GPT-4.1 with domain-specific retrieval, was compared with a 70.0% deterministic ICDPIC-R baseline.

The gain

A domain-specific retriever fine-tuned on trauma ICD-10 codes raised LLM exact-match accuracy for Injury Severity Score calculation, reaching 75.3% with GPT-4.1 versus 70.0% for the ICDPIC-R baseline.

The problem

Standard retrieval augmentation gave no consistent gains, and baseline LLMs reached only 47.3% to 64.7% exact-match accuracy, so off-the-shelf models are unreliable for this task.

The rundown

The data were deidentified free-text discharge diagnoses from trauma patients at Chang Gung Memorial Hospital in Linkou, Taiwan, from 2022. Performance was judged against a physician-verified gold standard, using exact score prediction, tolerance-based accuracy and severity categories.

The retrieval component drew on knowledge from the Abbreviated Injury Scale, and the domain-specific version had its retriever fine-tuned on trauma-related ICD-10 codes. This was contrasted with a standard retrieval setup that did not help consistently.

The authors conclude that accuracy was comparable overall to the deterministic ICDPIC-R tool, with the advantage concentrated in the most severe injuries. They suggest this could reduce coding workload and improve standardization of trauma registry data.

What this doesn’t fix

The study is a single-center retrospective feasibility test, so the findings need confirmation across multiple institutions.

Sources

  1. Peer-reviewedJMIR Medical Informatics2026-10-13

The debate