acequality, risk of bias, and applicability assessment of prediction models using regression or AI in healthcare
Source article: PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods
The Prediction model Risk Of Bias ASsessment Tool (PROBAST) is used to assess the quality, risk of bias, and applicability of prediction models or algorithms and of prediction model/algorithm studies. Since PROBAST’s introduction in 2019, much progress has been made in the methodology for prediction modelling and in the use of artificial intelligence, including machine learning, techniques. An update to PROBAST-2019 is thus needed. This article describes the development of PROBAST+AI. PROBAST+AI consists of two…
Contested: both sides are scored from claims and sources, not community votes.
Business ethics 2 by Jan Cardino13. CC BY-SA 4.0 · https://creativecommons.org/licenses/by-sa/4.0
Published March 24 2025 in the BMJ, this methods article describes PROBAST+AI, an updated assessment tool for prediction models built with regression or artificial intelligence methods. It splits assessment into model development and model evaluation, each organized around participants and data sources, predictors, outcome, and analysis domains.
The update matters because clinical prediction models increasingly use AI and machine learning, raising stakes for biased or non-applicable models in patient care. The paper offers a standardized way to scrutinize them, but does not report empirical validation, adoption rates, or impact on clinical outcomes, leaving real-world effectiveness uncertain.
- PROBAST+AI consists of two distinctive parts: model development with 16 signalling questions and model evaluation with 18 signalling questions.
- Both parts contain four domains: participants and data sources, predictors, outcome, and analysis, with applicability rated for three domains.
- Tool is intended to replace PROBAST-2019 and apply irrespective of whether regression modelling or AI techniques are used.
PROBAST+AI provides a unified tool that lets stakeholders assess quality, bias, and applicability of both regression and AI-based prediction models in healthcare.
The original 2019 PROBAST tool had become outdated given rapid progress in prediction modelling methodology and AI including machine learning.
The rundown
The authors describe development of PROBAST+AI as an update to the 2019 PROBAST tool, structured into model development and model evaluation components with 16 and 18 targeted signalling questions respectively.
The tool retains four domains across both parts and is positioned to be used by model developers, AI companies, researchers, editors, reviewers, healthcare professionals, guideline developers and policy organisations for any prediction model type in healthcare.
Sources
- Peer-reviewedBMJ2025-03-24
How should this claim be treated?
The debate