65/100

Moderate evidence

Cite
More
EmailLinkedInX

Hospital AI score flagged sick children earlier and more precisely than old tool, study finds

Study title: Electronic Children's Hospital Early Warning Score: Development and Validation of a Novel Machine Learning Algorithm Predicting Critical Deterioration Events in a Retrospective, Single-Center Cohort

Evidence type
Observational study
Sample
n = 179,074
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Electronic Children's Hospital Early Warning Score: Development and Validation of a Novel Machine Learning Algorithm Predicting Critical Deterioration Events in a Retrospective, Single-Center Cohort

Researchers at a single quaternary care pediatric hospital built a machine learning model, electronic CHEWS (eCHEWS), to predict critical deterioration in children on acute care wards. At about 44% sensitivity, it had a positive predictive value of 11.4%, against 1.8% for the hospital's existing Children's Hospital Early Warning Score.

Read the study abstract

Objective To develop and validate a novel, machine learning prediction model for critical deterioration events. Design Retrospective cohort study. Setting Single quaternary care freestanding pediatric hospital. Patients All patients up to 21 years old admitted to acute care wards (excluding cardiac and psychiatry units) between January 2012 and June 2023. Interventions None. Measurements and main results The primary outcome was a critical deterioration, defined as utilization of advanced respiratory support or c…

The gain

In a retrospective study of 179,074 pediatric hospitalizations at one hospital, the eCHEWS machine learning score had a positive predictive value of 11.4% versus 1.8% for the legacy CHEWS at about 44% sensitivity, cutting the number needed to alert from 55 to 9.

In brief

The model was trained and tested on data from 179,074 hospitalizations between January 2012 and June 2023, covering 2,803,017 scoring windows. Only 10,656 hospitalizations (0.4%) had a later critical deterioration, so the event is rare and false alarms are a central concern for clinicians.

The authors say eCHEWS can detect deterioration at least 6 hours before it occurs. They note it still needs prospective validation before use as an electronic health record trigger for proactive rounding, escalation and timely ICU transfer.

Main points

  1. The study was a retrospective cohort at one freestanding quaternary pediatric hospital, covering patients up to 21 years old on acute care wards from January 2012 to June 2023.
  2. Cardiac and psychiatry units were excluded from the analysis.
  3. Critical deterioration was defined as advanced respiratory support or continuous IV medication infusions given outside acute care units; ICU transfer was a secondary outcome.
  4. Outcomes were assessed 6 to 30 hours after each prediction score.
  5. Of 179,074 hospitalizations, 10,656 (0.4%) had a subsequent critical deterioration event.
  6. Area under the precision-recall curve was 0.157 for eCHEWS versus 0.017 for CHEWS.
  7. At about 44% sensitivity, the number needed to alert fell from 55 with CHEWS to 9 with eCHEWS.

The full story

The study compared eCHEWS directly with the legacy CHEWS at one hospital. Data covered 179,074 unique hospitalizations and 2,803,017 windows of scoring data. Each window was scored and checked for critical deterioration between 6 and 30 hours later.

On the precision-recall measure, eCHEWS scored 0.157 (95% CI 0.150-0.164) against 0.017 (95% CI 0.013-0.027) for CHEWS. Because only 0.4% of hospitalizations had an event, precision-recall is a key measure of how many alerts are true.

At roughly 44% sensitivity, CHEWS had a positive predictive value of 1.8%, meaning 55 alerts for each true event. eCHEWS reached 11.4%, or 9 alerts per true event. The authors conclude it outperforms traditional early warning scores and could support rounding, escalation and ICU transfer decisions if validated prospectively.

Sources

1 on file
  1. Pediatric Critical Care Medicinedoi:10.1097/pcc.0000000000004112
    Peer-reviewed2026-11-13

Talk it over

Three questions to start on, two more if you have time
At about 44% sensitivity, eCHEWS meant 9 alerts per true event against 55 with CHEWS. How would you weigh that gain against the fact that most alerts would still be false?

Start with a question

Hospital AI score flagged sick children earlier and more precisely than old tool, study findshttps://truvace.com/item/electronic-childrens-hospital-early-warning-score-development-and-validation-of-

  1. At about 44% sensitivity, eCHEWS meant 9 alerts per true event against 55 with CHEWS. How would you weigh that gain against the fact that most alerts would still be false?

    Answer in the thread
  2. The study looked back at 179,074 hospitalizations at one quaternary pediatric hospital, with cardiac and psychiatry units excluded. What could that setting tell us about how eCHEWS would work in other hospitals or on those units?

    Answer in the thread
  3. If you were a nurse on an acute care ward getting an eCHEWS alert, what would you want to know before deciding whether to escalate to proactive rounding or ICU transfer?

    Answer in the thread
If you have more time
  • The authors say eCHEWS still needs prospective validation. What would a prospective test in a hospital you know look like, and which result would settle whether to use it as a trigger?

    Answer in the thread
  • Only 10,656 hospitalizations (0.4%) had a critical deterioration. How does a rare event change how useful an alert is, and how many false alarms could a ward team live with?

    Answer in the thread

What readers say