Best AI in glioblastoma slide challenge reached only 69% accuracy on tumor sub-regions

The sub-regions were cellular tumor, pseudopalisading necrosis, microvascular proliferation, geographic necrosis, cortical infiltration and white matter penetration. The organizers describe it as the first international benchmarking environment and public spatially annotated dataset for this task, which they say remains an open research problem.

Performance suffered from severe class imbalance, which hurt minority classes, and from disagreement among annotators, especially on ambiguous regions such as pseudopalisading necrosis and white matter penetration. The authors say the lack of a consistent gold-standard annotation protocol limits reliability.

BraTS-Path 2024 is the first international benchmark for AI classification of glioblastoma histopathologic sub-regions. It was organized by the AI-RANO and RANO-Pathology working groups and uses a public, spatially annotated dataset built from multi-institutional H&E-stained whole slide images.

Eight expert neuropathologists created the reference standard following a peer review protocol, yielding 309,118 samples split into 96,485 for training, 62,387 for validation and 150,246 for hold-out testing. Six AI approaches were scored on metrics including accuracy and MCC, with a statistically aware ranking strategy.

The winning approaches shared large pre-trained models, ensembling and heavy data augmentation, yet topped out at accuracy 0.69 and MCC 0.58. Rare sub-regions and disagreement among annotators on pseudopalisading necrosis and white matter penetration held results back, and the authors point to foundation models as a route to better generalization.