Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

24 studies show machine learning no better than a standard model at predicting kidney injury

Study title: Comparison of machine learning and logistic regression models in predicting acute kidney injury: A systematic review and meta-analysis

Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2770cite or check it

Study title: Comparison of machine learning and logistic regression models in predicting acute kidney injury: A systematic review and meta-analysis

A systematic review and meta-analysis pooled 24 studies containing 84 prediction models to test whether machine learning predicts acute kidney injury (AKI) better than logistic regression, the conventional statistical approach. Average accuracy was similar: an AUC of 0.736 for machine learning against 0.748 for logistic regression (p = 0.538).

Read the study abstract

INTRODUCTION: We aimed to assess whether machine learning models are superior at predicting acute kidney injury (AKI) compared to logistic regression (LR), a conventional prediction model. METHODS: Eligible studies were identified using PubMed and Embase. A total of 24 studies consisting of 84 prediction models met inclusion criteria. Independent samples t-test was performed to detect mean differences in area under the curve (AUC) between ML and LR models. One-way ANOVA and post-hoc t-tests were performed to ass…

The problem

A systematic review and meta-analysis of 24 studies (84 models) found machine learning predicted acute kidney injury no better than logistic regression, with average AUC of 0.736 versus 0.748.

In brief

The averages hide wide differences between methods. Gradient boosting models reached an average AUC of 0.838, which was better than other machine learning models in the literature (p < 0.05). The review did not report that gradient boosting beat logistic regression directly.

Creatinine and urine output, the standard variables used to stage AKI, came up as significant predictors across multiple machine learning models. Most other significant predictors were unique to individual studies, so it is unclear which inputs generalise across hospitals.

Main points

  1. Eligible studies were found through PubMed and Embase; 24 studies with 84 prediction models met the inclusion criteria.
  2. Average AUC was 0.736 ± 0.116 for machine learning models and 0.748 ± 0.057 for logistic regression, a difference that was not statistically significant (p = 0.538).
  3. Gradient boosting averaged an AUC of 0.838 ± 0.077, outperforming other machine learning models (p < 0.05).
  4. Creatinine and urine output were significant predictors across multiple machine learning models.
  5. Most significant predictors were unique and study specific, and model performance varied with the model used, variable selection and processing, and how models were trained, validated, tested and calibrated.

The full story

The authors compared mean AUC between machine learning and logistic regression models with an independent samples t-test, then used one-way ANOVA and post-hoc t-tests to compare differences between machine learning methods.

Their conclusion is that machine learning models perform equally to logistic regression on average, but with variable performance, and some models displayed exceptional performance. Gradient boosting was the example reported, with an AUC of 0.838.

The authors attribute the variability in machine learning predictions of AKI in part to the specific model used, variable selection and processing, study and subject characteristics, and the steps of model training, validation, testing and calibration.

Sources

1 on file
  1. International Journal of Medical Informaticsdoi:10.1016/j.ijmedinf.2021.104484
    Peer-reviewed2021-05-08

Talk it over

Four questions to start onWho carries this cost, and what would have to change to prevent it?

Start with a question

24 studies show machine learning no better than a standard model at predicting kidney injuryhttps://truvace.com/item/comparison-of-machine-learning-and-logistic-regression-models-in-predicting-acut

  1. Who carries this cost, and what would have to change to prevent it?

    The problem A systematic review and meta-analysis of 24 studies (84 models) found machine learning predicted acute kidney injury no better than logistic regression, with average AUC of 0.736 versus 0.748.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis, from International Journal of Medical Informatics.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Performance varied widely between studies and models, and most significant predictors were unique and study specific, so the pooled averages do not show how any single model would perform in a given hospital.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Health