Truace
Subscribe

Study finds AI model missed most child ICU deaths in the only open dataset

Study title: Evaluating the AI-predictive potential of the PIC database for pediatric ICU mortality

Researchers tested how well the Pediatric Intensive Care (PIC) database, the only open-access resource focused on pediatric critical care, supports machine learning prediction of death in critically ill children. The best model, CatBoost, reached an AUC of 0.8521 on a hold-out test set but identified only 42% of non-survivors.

Read the study abstract

Background Reliable Artificial Intelligence (AI) tools in healthcare depend on both data quality and model design. However, the predictive integrity of datasets is rarely assessed. Publicly available resources for critically ill children remain scarce. The Pediatric Intensive Care (PIC) database remains, to the best of current knowledge, the only open-access resource focused on pediatric critical care research. Methods This study evaluates the predictive potential of the PIC database. After applying strict cohor…

Truvace record2 min read1 sourceEvidence: Moderate · 52/100 — Benchmark or lab evaluation · 1 independent sourcePeer-reviewedTRV-2026-2389 · cite or check it
Evaluating the AI-predictive potential of the PIC database for pediatric ICU mortality"Medical laboratories and clinics in Kinwat" by Mouryan, CC BY-SA 4.0.

In brief

The authors say the weak sensitivity points to limits in the dataset itself rather than only in model design. They report that PIC lacks essential physiological data, which reduces the clinical reliability of models built on it.

The finding matters because publicly available data on critically ill children are scarce, so researchers building pediatric ICU prediction tools have few alternatives. The authors call for richer multicenter datasets that include physiological and neurological indicators.

Main points

  1. CatBoost was the best-performing model, with an AUC of 0.8521 (95% range 0.8414-0.8627) on the hold-out test set.
  2. Sensitivity was 42%, so the model missed most children who died.
  3. The most influential variables were minimum lactate, maximum lactate, mean respiratory rate, maximum hematocrit and minimum urea.
  4. The team applied strict cohort selection and preprocessing, then used univariate odds ratios and SHAP values to examine associations and feature importance.
  5. The authors describe this as the first comprehensive evaluation of PIC for machine learning prediction.
  6. They recommend multicenter pediatric ICU databases that include essential physiological and neurological indicators.

The problem

In an evaluation of the open Pediatric Intensive Care database, the best machine learning model caught only 42% of non-survivors, which researchers say limits its clinical reliability because the data lack essential physiological measures.

The full story

The study evaluated the predictive integrity of the PIC database, which the authors describe as, to the best of current knowledge, the only open-access resource focused on pediatric critical care research. They note that the predictive integrity of datasets is rarely assessed even though reliable healthcare AI depends on data quality as well as model design.

After strict cohort selection and preprocessing, the team combined univariate odds ratios with machine learning models interpreted through SHAP values. Performance was described as moderate but limited, with CatBoost reaching an AUC of 0.8521 but sensitivity of 42%.

Lactate measures, respiratory rate, hematocrit and urea were the most relevant variables. The authors conclude that missing physiological data in PIC limits current pediatric databases and call for richer multicenter datasets to support more robust, clinically applicable AI models.

Sources

  1. Peer-reviewedPediatric Research2026-11-02

The debate