65/100

Moderate evidence

Cite
More
EmailLinkedInX

Study of 51,475 child heart surgeries finds AI risk models no better than standard ones

Study title: Machine-Learning Derived Individual Risk Prediction in Congenital Heart Surgery: Development and internal validation of risk prediction models for in-hospital mortality after congenital heart surgery using data from the Pediatric Cardiac Critical Care Consortium (PC4) registry

Evidence type
Observational study
Sample
n = 51,475
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Machine-Learning Derived Individual Risk Prediction in Congenital Heart Surgery: Development and internal validation of risk prediction models for in-hospital mortality after congenital heart surgery using data from the Pediatric Cardiac Critical Care Consortium (PC4) registry

Researchers using the Pediatric Cardiac Critical Care Consortium registry compared machine-learning models with a traditional logistic regression model for predicting in-hospital death after congenital heart surgery. The data covered 51,475 operative encounters at 52 hospitals between September 2014 and December 2019, in patients aged 18 or younger.

Read the study abstract

Background Machine learning (ML) methods may improve prediction of individual risk of mortality risk after congenital heart surgery (CHD) by identifying nonlinear relationships/novel combinations of variables. Improved risk discrimination may improve outcomes by optimizing resource allocation and patient surveillance. Methods Children and adolescents (age≤18 years) undergoing CHD surgery at Pediatric Cardiac Critical Care Consortium hospitals 9/1/2014-12/30/2019 were studied. Non-collinear potential predictors w…

Machine-Learning Derived Individual Risk Prediction in Congenital Heart Surgery: Development and internal validation of risk prediction models for in-hospital mortality after congenital heart surgery using data from the Pediatric Cardiac Critical Care Consortium (PC4) registry
"Facial Plastic Surgeon in Operating Room" by Richard Balikian MD, CC BY 4.0.

The problem

In a registry study of 51,475 congenital heart operations at 52 US-linked hospitals, random forest and gradient boosting models predicted in-hospital death no better than logistic regression, with reclassification gains that were at best modest and not significant.

In brief

Overall fit measures (C-statistics and Brier scores) did not differ significantly between the machine-learning and logistic regression models. Against a baseline model using only the STAT surgical complexity category, the machine-learning models had a net reclassification index of 0.76 for risk categories, versus 0.75 for logistic regression.

The finding matters because better risk discrimination had been expected to help hospitals allocate resources and monitor patients. The authors say future work should test whether more comprehensive data or more advanced computational methods add value.

Main points

  1. The study covered 51,475 operative encounters at 52 hospitals from 1 September 2014 to 30 December 2019.
  2. Patients were 45% female, 20% were neonates and 5% underwent STAT 5 operations, the highest complexity category.
  3. C-statistics and Brier scores did not differ significantly between machine-learning and logistic regression models.
  4. For absolute risk categories, net reclassification was 0.76 (95% CI 0.68-0.83) for machine learning and 0.75 (95% CI 0.67-0.83) for logistic regression.
  5. For ranking of risk within the cohort, net reclassification was 0.26 (95% CI 0.19-0.32) for machine learning and 0.24 (95% CI 0.17-0.33) for logistic regression.

The full story

The team identified non-collinear predictors empirically and built three models: a logistic regression meant to mimic traditional risk-adjustment tools, plus random forest and gradient boosting machine-learning models. All were compared with a simple model based only on STAT surgical category.

The main measure was net reclassification index, which captures how often a model changes a clinician's view of a patient's risk, either by moving the patient to a different absolute risk category or by changing their rank relative to other patients. Secondary analyses compared overall model fit.

Results showed the machine-learning models changed risk views only slightly more than logistic regression, and the differences were not statistically significant on either the category or ranking measure. The authors conclude that the benefit of machine learning here was at best modest.

Sources

1 on file
  1. American Heart Journaldoi:10.1016/j.ahj.2026.107655
    Peer-reviewed2026-11-14

Talk it over

Three questions to start on, two more if you have time
Across 51,475 operations at 52 hospitals, random forest and gradient boosting predicted in-hospital death no better than logistic regression. How would you weigh the effort of building machine-learning tools against a gain that was at best modest?

Start with a question

Study of 51,475 child heart surgeries finds AI risk models no better than standard oneshttps://truvace.com/item/machine-learning-derived-individual-risk-prediction-in-congenital-heart-surgery-

  1. Across 51,475 operations at 52 hospitals, random forest and gradient boosting predicted in-hospital death no better than logistic regression. How would you weigh the effort of building machine-learning tools against a gain that was at best modest?

    Answer in the thread
  2. The models were built and internally validated on registry data from one consortium. What could that design miss about how they would work in a hospital outside it?

    Answer in the thread
  3. If you were a clinician at one of these hospitals, what result would make you rely on a machine-learning risk score over the STAT surgical complexity category?

    Answer in the thread
If you have more time
  • The authors suggest testing whether more comprehensive data or more advanced methods add value. What data would you want added, and why?

    Answer in the thread
  • Net reclassification was 0.76 for machine learning and 0.75 for logistic regression. How would you explain to a parent what that gap means for a child's risk estimate?

    Answer in the thread

What readers say