External validation of a machine learning pediatric surgical risk calculator in Ethiopia and Nigeria

Source article: Does it Fit? Testing a Novel Pediatric Surgical Risk Calculator in Africa

Background Low- and middle-income countries (LMICs), particularly Sub-Saharan Africa, experience a high burden of surgical disease and a disproportionately high postoperative morbidity and mortality. Although machine learning based pediatric surgical risk calculators (SRCs) have been developed to support clinical decision making, most are derived from high-income settings and may not generalize well to LMIC populations. We previously developed a pediatric surgical risk calculator using multinational LMIC data; h…

By the Truvace deskOctober 16, 20262 min read1 sourcePeer-reviewedTRV-2026-1631· cite & verify

Does it Fit? Testing a Novel Pediatric Surgical Risk Calculator in Africa — "Dr. Jianendra Singh doing pediatric surgery in Aastha" by Bhakua is licensed under CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/.

Trace impact readingContested

P 68

Both sides are scored from claims and sources, not community votes.

G 67

In brief

Researchers tested a previously developed machine learning pediatric surgical risk calculator on prospective data from children operated on at four tertiary referral hospitals in Ethiopia and Nigeria between July 2024 and February 2025. The tool had been built from multinational data from low- and middle-income countries but had not been checked in independent African groups.

The result matters because most surgical risk tools come from high-income settings and may not transfer to places with high postoperative mortality. The validation suggests the tool can help in these settings, but its weak performance in newborns and the large share of patients without a useful risk category limit how far it can guide care without further refinement.

Main points

The cohort had 1,670 pediatric surgical patients, and in-hospital mortality was 6.8%.
Overall discrimination reached an AUC of 0.84, compared with 0.77 for ASA classification alone.
Discrimination was 0.83 for general pediatric surgery and 0.89 for emergency cases, but only 0.64 in neonates.
Initial miscalibration improved after logistic recalibration.
Likelihood ratios were 0.1 for low-risk and 5.4 for high-risk patients, while 66.5% fell in an intermediate group with a ratio of 1.0.

The gain

In a 1,670-child validation cohort, the machine learning calculator predicted in-hospital mortality better than ASA class alone and separated low-risk from high-risk children.

The problem

The calculator performed poorly in neonates and left most patients in an intermediate risk group that gives no useful guidance for decisions.

The rundown

The study was an external validation using prospective data from four tertiary referral hospitals in Ethiopia and Nigeria. Model performance was judged by discrimination and by calibration slope and intercept, and predicted risk was split into low, intermediate and high categories to make it easier to apply clinically.

Mortality differed significantly by site, age, weight, sex, prematurity, preoperative sepsis, transfusion, supplemental oxygen use, antibiotic use, ASA class, emergent versus elective surgery, postoperative ICU need and reoperation.

The authors conclude that the calculator performed well in the African context and became more useful after recalibration and risk stratification. They say neonates may need additional risk factors and possibly a separate model, and that future work should support risk communication, clinical decision making and resource prioritization.

What this doesn’t fix

Performance was low in neonates, the model was systematically miscalibrated until recalibrated, and about two thirds of patients fell in an intermediate risk group that did not help decision making.