Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Luxembourg floods: AI forecasts cut temperature error up to 19%; physics led on heaviest rain

Study title: Physics vs. AI in weather prediction: evaluating GraphCast, AIFS, and FuXi against an observation-corrected WRF model for flash floods

Evidence type
Controlled or validated comparison
Sample
n = 92,119
Sources
1 sourcePeer-reviewed
Published
3 min read
Truvace record
TRV-2026-2525cite or check it

Study title: Physics vs. AI in weather prediction: evaluating GraphCast, AIFS, and FuXi against an observation-corrected WRF model for flash floods

Researchers at the University of Luxembourg compared three AI weather models (GraphCast, AIFS Single 1.0 and FuXi) with a regional physics-based WRF model that was corrected using observations. They tested all four systems on flood events in 2016, 2018 and 2021, scoring them against 92,119 matched forecast–observation pairs from stations across four countries.

Read the study abstract

Data-driven artificial intelligence (AI) weather models are increasingly positioned as alternatives or complements to physics-based numerical weather prediction (NWP) systems, yet systematic model evaluation studies that compare both paradigms under controlled experimental conditions remain limited. Here we present a structured model evaluation framework applying standardised verification metrics to assess a regional WRF configuration against three publicly available AI models – GraphCast, AIFS Single 1.0, and F…

58Gain side

How the two sides scoreClose call

59Problem side

The gain

In a hindcast study of three floods in Luxembourg and the Greater Region, AIFS and GraphCast cut temperature error by 19.0% and 17.6% against an observation-corrected physics model and led it on light-rain forecast skill.

The problem

In the same study, at 20 mm of rain per 6 hours the physics model scored highest (CSI 0.160 against 0.141 for AIFS and 0.114 for GraphCast), and FuXi forecast no such rainfall at all.

A fixed formula sets both scores from the evidence and its sources. Reader votes don’t move them.

What the scores mean

G is the gain side, out of 100. It goes up when the gain is specific, when it measurably helped people and when the sources behind it are strong.

P is the problem side, out of 100. It goes up when the problem is specific, when it measurably hurt people and when the sources behind it are strong.

In brief

At light rain of 1 mm per 6 hours, GraphCast had the best overall skill (CSI 0.424) and AIFS detected the most rain events (POD 0.835), both ahead of the assimilated WRF (CSI 0.389). AIFS and GraphCast also cut temperature error by 19.0% and 17.6% against assimilated WRF.

The order flipped for heavy rain, which matters most for flash-flood warnings. At 20 mm per 6 hours WRF scored highest (CSI 0.160), ahead of AIFS (0.141) and GraphCast (0.114), and FuXi produced no exceedances. Only three events were tested, and AIFS was fine-tuned on data covering all three, so its scores may be optimistic.

Main points

  1. The test covered three floods (July 2016, the 2018 Mullerthal flash flood and the July 2021 European flood), with 345 valid 6-hour times and 259, 280 and 264 independent precipitation stations per event.
  2. Assimilating observations raised WRF's 1 mm CSI from 0.364 to 0.389 and moved temperature bias close to zero, but at 10 mm the assimilated run scored below the unassimilated one (CSI 0.189 against 0.212).
  3. AIFS was best at 5 and 10 mm; at 10 mm its CSI was 0.237 against 0.189 for assimilated WRF.
  4. GraphCast's skill fell from CSI 0.424 at 1 mm to 0.114 at 20 mm, and FuXi produced no 20 mm exceedances anywhere in the verification set.
  5. AIFS had the lowest temperature error (RMSE 1.552 °C) and GraphCast followed (1.580 °C), against 1.916 °C for assimilated WRF; FuXi had the highest errors and a warm bias of +0.609 °C.
  6. GraphCast's skill was highest on the 2021 event (CSI 0.493 against 0.430 in 2016), though that event lies after its 1979–2017 training period.
  7. On one A100 GPU a 6-hour AI forecast took about 1.5 minutes, roughly 140 times faster than the WRF chain on 52 CPU cores (34–37 minutes per cycle).
  8. In a radar comparison over 27 six-hour periods in July 2021, AIFS led at 1 mm, but at 20 mm both WRF configurations were ahead of all three AI models.

The full story

The study ran a Luxembourg-tuned WRF model at 12 km grid spacing with 3D-VAR assimilation of GNSS zenith total delay and conventional observations every 6 hours. It was compared with GraphCast, FuXi and AIFS Single 1.0, all started from the same ERA5 reanalysis and re-initialised every 6 hours. Stations within 12 km of any assimilated surface observation were excluded, and none of the systems received bias correction.

On light rain and temperature the AI models led. GraphCast's CSI was 0.424 at 1 mm, AIFS reached a detection rate of 0.835, and AIFS and GraphCast beat assimilated WRF on temperature RMSE by 0.364 °C and 0.337 °C, differences the bootstrap resolved at every block length tested. Their 1 mm CSI advantage over WRF was +0.035 for GraphCast and +0.026 for AIFS.

Heavy rain reversed the order. WRF's 20 mm CSI of 0.160 beat GraphCast by +0.046, a gap the bootstrap resolved at all block lengths. Its edge over AIFS (+0.019) was resolved only for 24 and 48 hour blocks. Radar checks for July 2021 showed the same reversal in space, with both WRF configurations ahead of the three AI models at 20 mm.

Assimilation itself helped light-rain detection (POD 0.527 to 0.611) but lowered CSI at 5 and 10 mm, and the 10 mm decrease appeared in all three events (largest in 2018, −0.027). In 2018 the assimilated run issued 338 heavy forecasts against 263 but scored 114 hits against 132. The authors report this without attributing a cause.

What this doesn’t fix

Only three flood events were tested, and AIFS was fine-tuned on 2016–2022 data covering all of them, so its scores may be optimistic and cannot show generalisation.

Sources

1 on file
  1. Geoscientific Model Developmentdoi:10.5194/gmd-19-9831-2026
    Peer-reviewed2026-10-13

Talk it over

Four questions to start onThis record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?

Start with a question

Luxembourg floods: AI forecasts cut temperature error up to 19%; physics led on heaviest rainhttps://truvace.com/item/physics-vs-ai-in-weather-prediction-evaluating-graphcast-aifs-and-fuxi-against-a

  1. This record found a gain and a problem in the same place. Which matters more to you, and who might answer differently?

    The gain In a hindcast study of three floods in Luxembourg and the Greater Region, AIFS and GraphCast cut temperature error by 19.0% and 17.6% against an observation-corrected physics model and led it on light-rain forecast skill.

    The problem In the same study, at 20 mm of rain per 6 hours the physics model scored highest (CSI 0.160 against 0.141 for AIFS and 0.114 for GraphCast), and FuXi forecast no such rainfall at all.

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a controlled comparison with a sample of 92,119, from Geoscientific Model Development.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Only three flood events were tested, and AIFS was fine-tuned on 2016–2022 data covering all of them, so its scores may be optimistic and cannot show generalisation.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Science