The best weather forecast may come from models that disagree
A weaker forecast model can improve the strongest one—if it makes different mistakes. That is the promise of blending AI and physics-based weather ensembles.
A weaker model improved the stronger one—because its errors were different.
A weather forecast can improve when a weaker model joins a stronger one—not because the newcomer is secretly better, but because it makes different mistakes. In this study, a blend of the UK Met Office’s high-resolution ensemble and an AI-based ensemble beat the stronger of the two on average across forecast lead times. The implication is bigger than a contest between artificial intelligence and physics: forecasting systems may gain more from diversity than from simply making one model more powerful.
That matters because forecasts are decisions in disguise. A temperature outlook can affect whether councils grit roads, whether energy suppliers prepare for demand, or whether people working outdoors face heat or cold safely. Forecasters need not just a single best guess, but a credible range of possible outcomes. Ensembles—sets of slightly different forecasts—help describe that uncertainty. Combining ensembles from distinct systems could make those ranges more useful, especially when the models see the atmosphere differently.
The paper tests that idea with screen-temperature forecasts at sites across the United Kingdom. Its central result is consistent across average probabilistic scores, selected temperature thresholds and mapped locations: adding AIFS-CRPS to the Met Office’s MOGREPS-UK ensemble improved the blend, even though MOGREPS-UK was usually the more accurate model on its own. The authors also report that the blend of MOGREPS-UK with either AIFS-CRPS or another physics-based model, IFS, outperformed a blend of IFS and AIFS-CRPS. The pairing, not the AI label, appears to be the point.
The Science
Weather models are imperfect maps of a turbulent system. Physics-based models calculate how the atmosphere evolves by solving equations; AI-based models learn patterns from historical weather data. Neither approach captures every process or error. A physics-based model may resolve local detail well but still carry systematic biases. An AI model may be fast and competitive, but its errors can reflect its training data and design.
The study brings together the Met Office’s MOGREPS-UK, a convection-permitting physics-based ensemble designed for the UK, and ECMWF’s AIFS-CRPS, an AI-based ensemble. “Ensemble” means that a model produces multiple plausible forecasts rather than a single outcome. MOGREPS-UK supplies 18 time-lagged members, while AIFS-CRPS has 50 members. The researchers also compare forecasts from ECMWF’s physics-based Integrated Forecasting System, or IFS, which has 50 members.
Ensemble members in the forecast systems
Member counts described in the study. MOGREPS-UK members are formed from six time-lagged cycles; IFS and AIFS-CRPS each contain 50 members.
| Label | Value |
|---|---|
| MOGREPS-UK | 18 members |
| MOGREPS-G | 36 members |
| IFS | 50 members |
| AIFS-CRPS | 50 members |
The analysis focuses on screen-level temperature at 130 UK sites, with wind speed examined in supplementary results. Forecasts cover 24 hourly lead times and are verified against observations at 00Z, the standard time reference used in meteorology. The main dataset runs from 1 October 2024 to 31 January 2025: October is used to train the calibration, and November through January to evaluate performance. The authors also report a separate July 2025 temperature comparison in the appendix. The scope is therefore a detailed UK case study, not a global test across all seasons and climates.
Scope of the main evaluation
Three dimensions of the UK study design: number of forecast sites, evaluation days, and probability thresholds. These are separate counts, not a common sample-size measure.
| Label | Value |
|---|---|
| Forecast sites | 130 sites |
| Evaluation period | 92 sites |
| Temperature thresholds | 41 sites |
To make the models comparable at sites, the team regrids forecasts onto a shared UK grid, selects the nearest land grid point and adjusts for differences in elevation. The Met Office’s high-resolution MOGREPS-UK grid is about 2 kilometres across; the global forecasts are coarser and are interpolated to that grid. The adjustment matters: a forecast for a hilltop and an observation in a nearby valley should not be compared as if they were at the same altitude.
The blend combines forecast probabilities, rather than simply averaging the models’ expected temperatures. For each temperature threshold, the system estimates the chance that the forecast will exceed it, then takes a weighted average of the models’ probabilities. In the main comparisons, the more skillful model at a given lead time receives 80% of the weight and the less skillful one 20%. This deliberately simple weighting tests whether a second model adds value; it is not presented as the final operational recipe.
The main score is the continuous ranked probability score, or CRPS. It measures the distance between a forecast’s whole probability distribution and what actually happened; lower scores mean better forecasts. The paper also examines the Brier score at selected thresholds, which measures the quality of probability forecasts for events such as temperature above or below a cutoff. Its decomposition separates reliability—whether, for example, events forecast at 70% happen about 70% of the time—from resolution, the ability to distinguish situations with different outcomes.
Calibration is a further step. A model can be systematically too warm or too confident even when it captures useful patterns. The researchers use ensemble model output statistics (EMOS), a statistical method that adjusts the forecast distribution using the model’s ensemble average and spread. They test calibration before blending, after blending, and at both stages.
What They Found
Across lead times, the MOGREPS-UK ensemble usually outperformed AIFS-CRPS on its own for UK temperature, with an exception at day five, when AIFS-CRPS became stronger. Yet the weighted blend performed better than the best individual model at every lead time in the overall CRPS and root mean squared error comparisons. Root mean squared error evaluates errors in the expected, or average, forecast value; it does not capture the full forecast distribution, so the agreement between it and CRPS is notable. The paper’s Figure 1 shows these lead-time comparisons.
The same broad pattern appears when the forecasts are judged at temperature thresholds. At a lower threshold, MOGREPS-UK beats AIFS-CRPS throughout the forecast range, while the blend has the best Brier score at every lead time. At middle and higher thresholds, the Met Office model is again generally stronger except at day five, but the blend still beats the best individual model. The authors report a similar improvement for wind speed in the appendix.
The gain is not confined to an average across the country. When the team maps CRPS skill by site, AIFS-CRPS trails MOGREPS-UK across many locations at short lead times; that gap shrinks as forecasts extend further into the future. By day five, AIFS-CRPS is better at many sites. More strikingly, the blend exceeds the MOGREPS-UK reference at almost every mapped location on day one, despite the reference being the stronger individual model overall at that lead. At longer leads, many sites show larger blend gains, especially in central England and northern Scotland. Figure 4 makes visible what a national average can hide: the value of a model depends partly on where the forecast is being made.
The scores also suggest that blending can improve more than one attribute at once. The blend’s resolution—the capacity to distinguish different weather situations—exceeds that of the best model across the reported lead times and selected thresholds. Its reliability is similar to that of the better model. In practical terms, the combination appears to become more informative without paying for that extra discrimination through a comparable deterioration in probability calibration.
To test whether AI itself explains the gain, the authors compare different pairings. MOGREPS-UK blended with IFS performs about as well as MOGREPS-UK blended with AIFS-CRPS, with only a slight difference in overall skill. But IFS blended with AIFS-CRPS performs worse than those pairings across almost all lead times. The authors suggest a reason: IFS and AIFS-CRPS may make more similar errors than either does with the regional MOGREPS-UK. AIFS-CRPS was trained on ERA5, a historical atmospheric dataset produced using IFS, which may contribute to overlap in their errors. Figure 5 compares these model combinations.
This points to a useful distinction. A model can be less accurate on average and still improve a blend if it provides information the stronger model lacks. Conversely, two individually capable models may add little together if their errors move in step. The researchers formalize this intuition by decomposing the blend’s CRPS into contributions from the component models’ average skill and a diversity term. Their analysis supports giving greater weight to the more skillful source while retaining a contribution from a model whose errors are complementary.
The calibration experiments add another piece. According to the abstract, calibrating AIFS-CRPS improves its skill by an amount comparable to calibrating physics-based ensemble forecasts. Calibrating each source before blending, then calibrating the blend itself, further improves overall accuracy. The supplied paper text does not include the numerical values behind these calibration comparisons, so the size of the gain cannot be stated here. Still, the result challenges the idea that statistical correction is a mature tool only for traditional numerical models.
Why This Changes Things
Forecasting has often been framed as a race: which model has the best score? This paper suggests a more operational question: which set of models, with their different strengths and weaknesses, gives the best decision-ready forecast? The answer can favor a model that loses the head-to-head comparison. That is a subtle but consequential shift, because it changes how institutions might assess investment in new forecasting systems.
The work also gives a reason to keep distinct development paths alive. AI models can be cheaper to run after training and can potentially generate large ensembles. Physics-based models encode explicit atmospheric processes and, at regional scales, can represent local phenomena at high resolution. The study does not establish that either family is universally superior. It shows instead that distinct systems can complement one another, and that this complementarity may be an asset in itself.
There is a real-world route from better probabilities to better choices. A forecast that more accurately expresses the chance of a temperature threshold being crossed can help public services plan for cold, heat, or demand spikes. Better uncertainty information can also make decisions more proportionate: a small chance of a severe event may justify preparation, while a high-confidence benign forecast may allow resources to be redirected. The paper does not measure such downstream benefits, but it improves one of the ingredients they require—forecasts that are both informative and probabilistically trustworthy.
The findings are particularly relevant to organizations already operating multi-model blends. The study uses IMPROVER, the Met Office’s open-source post-processing system, and a simple fixed 80/20 weighting scheme. Operational blending can instead adjust weights by lead time, and potentially by location or conditions. Since the blend’s relative benefit varied geographically, the results make a case for testing more local or lead-dependent weights. But that is a next experiment, not a proven improvement in this paper.
What’s Next
The evidence is promising, but it has boundaries. The main evaluation covers a few winter months, with 130 sites and a short training period. The separate July comparison is encouraging, yet it does not substitute for systematic tests across multiple years, seasons, extreme events and other regions. UK site temperatures also do not represent every forecast variable: the authors include wind speed in supplementary analysis, but the paper’s central investigation is temperature.
The weighting choice deserves scrutiny too. Choosing the 80% model according to its mean skill in the evaluation period is useful for an exploratory comparison, but a deployed system must choose weights without looking into the future. It must also avoid abrupt changes from one forecast lead to the next. The paper notes that a gradual, or gradient, weighting scheme can reduce such jumps. Local weighting might unlock further gains, but it would need enough independent data to avoid chasing noise at individual sites.
Another qualification is ensemble size. The study uses ensembles of different sizes and explicitly does not correct for the way small ensembles can bias score estimates. It also does not provide, in the supplied text, numerical effect sizes or uncertainty intervals for the reported improvements. That makes it difficult to judge how large the gains are, or how stable they would be under repeated samples. Operational decisions should therefore rely on longer and broader verification, including checks of rare but consequential thresholds.
The broader lesson survives those caveats: weather forecasting may advance not only by building a more accurate model, but by assembling models that fail in different ways—and then calibrating their combined probabilities carefully. The next step is to see whether the advantage holds across longer records, more variables and genuinely operational weighting strategies. If it does, AI’s most valuable contribution may not be replacing physics-based forecasting, but helping make a diverse forecasting system better than any one of its parts.
Sign in to join the conversation.
Comments (0)
No comments yet. Be the first to share your thoughts.