A forecast can lose skill—and reveal where its errors are going
A forecast can lose error energy at one scale while still being seeded by errors at another. A new coordinate system makes both processes visible.
200 forecasts: slow-level errors lost energy to faster levels even as fast-only errors seeded them immediately.
A slow variable in a chaotic forecast lost error energy to faster variables at a nearly steady relative rate of about 1.2 per model time unit. Yet forecasts that began with errors only in the fast variables still seeded errors in the slow variable immediately. Those two results sound contradictory only if “error growth” is treated as one number. Malaquias Peña’s new framework tracks both how far a forecast error has advanced toward saturation and how its energy is shifting among scales.
That distinction matters because a forecast can have the same total error as another forecast and a very different error spectrum. One may be failing first at small, fast scales; another may already have lost the large-scale patterns that matter for weather, oceans or other complex systems. Peña’s coordinates make those changes visible together, then connect them to the physical processes that move error energy around.
The study tests the approach in the three-level Lorenz–96 model, a deliberately simplified system used to investigate weather-like predictability. It is not a forecast of the real atmosphere, and its numerical rates should not be read as operational weather results. The broader contribution is a way to ask a practical scientific question more precisely: Is a forecast getting worse because its error is growing, because error is moving into a consequential scale, or both?
The Science
A familiar picture of forecast error is a curve that starts small, grows rapidly, then levels off. Logistic growth describes that shape: error accelerates at first, then slows as it approaches a reference level. The reference level here is not a universal maximum. It is a fixed climatological energy for each measured band, based on the variability of the model’s long-running “truth” simulation.
Peña applies a mathematical transformation called the logit to the error energy divided by that reference. If this normalized error is called , its logit is . Under the logistic reference model, the logit increases at a constant rate. That makes it a clock: instead of saying only that an error is large or small, it gives a scale for how far through a standard growth-and-saturation process the error has moved. The transformation can also be inverted, so the clock reading maps back to normalized error energy.
The “Koopman” in Logit–Koopman refers to a way of describing nonlinear dynamics through observables whose evolution is simple. For logistic growth, the logit advances linearly and the odds, , grow exponentially. Peña does not claim that this simple law exactly governs the full chaotic model. Rather, it supplies a prescribed reference geometry: deviations from its steady clock speed become measurable instead of being hidden inside an aggregate curve.
Now divide the slow model’s Fourier error into three bands: wavenumbers 1–5, 6–9, and 10–18. A Fourier band groups patterns by their spatial scale; low wavenumbers represent broad patterns and high wavenumbers finer ones. Each band gets its own logit clock. A shared, or longitudinal, coordinate measures progress along a reference relation among the clocks. Transverse coordinates measure how the bands depart from that relation. In plain terms, the method separates “how far along is this forecast’s error growth?” from “which scales are running ahead or behind?”
That separation is more than a visualization. The paper derives an exact transformation of the error-energy budget. Physical processes such as production, dissipation, transfers among Fourier bands, and exchanges between model levels can each be translated into contributions to clock speed or to transverse movement. A process that speeds all band clocks together advances the forecast along the reference path; one that affects bands unequally bends its trajectory away from it.
The experiments use a three-level Lorenz–96 system: a slow ring of variables, a faster intermediate ring, and a still faster ring. In one regime, the levels are uncoupled and only the slow ring remains active. In the other, all levels interact. For each regime, the researchers launched 200 deterministic forecasts from states produced by a local ensemble transform Kalman filter, a method that blends model forecasts with observations to estimate the system’s state. Each forecast ran for six model time units, with forecast errors checked repeatedly against a simulated truth.
Initial normalized errors in the coupled experiment
Mean launch error energy in each model level, normalized by its fixed climatological reference energy, as reported for the coupled Run C.
| Label | Value |
|---|---|
| Slow X | 0.05 normalized reference energy |
| Middle Y | 0.56 normalized reference energy |
| Fast Z | 0.48 normalized reference energy |
In the coupled experiment, the forecast began with the slow level much better constrained than the faster levels: its mean normalized error was 0.05, while the middle and fast levels were already at 0.56 and 0.48 of their reference energies. The imbalance is central to the predictability question the paper studies: what happens to a relatively well-known slow state when the faster parts of the system are already uncertain?
What They Found
First, the logit clocks tracked the mean error growth remarkably well. Across the reported fits, the band-mean logits had values from 0.965 to 0.993. An near one means a straight-line clock accounts for most of the observed variation over the fitted interval. This does not prove that the full dynamics are logistic; it says that the chosen reference offers a useful, compact description of the mean band growth in these experiments.
Reported linearity of band-mean error clocks
The lowest and highest R² values reported across the fitted band-mean logits in the two regimes; the paper gives this as a range, not as a regime-by-regime breakdown.
| Label | Value |
|---|---|
| Lowest reported fit | 0.965 R² |
| Highest reported fit | 0.993 R² |
The relationship among clocks carried information too, but correlation needed careful interpretation. If several independent bands follow logistic curves with different constant rates, their logit clocks can rise in near lockstep simply because they share the same passage of time. High temporal correlation alone therefore does not show that scales interact. Peña compares correlations in mean clock timing, in residuals after removing each band’s fitted straight-line trend, and across individual forecast launches. These diagnostics ask separate questions: do bands progress together, do they depart from their average growth together, and do forecasts with unusually high error in one band also tend to have high error in another?
The paper’s Figure 4 puts those three views side by side for the uncoupled and coupled experiments. Its key interpretive point is that “the bands are correlated” is not one finding: timing, deviations from timing, and forecast-to-forecast alignment can tell different stories. The results show distinct patterns across those measures, rather than a single correlation that can be treated as evidence of a common physical cause.
Second, the transformed budget helped identify the direction of physical exchange. In the coupled control, recovered forecast-error fields showed net error-energy transfer from the slow level toward the faster levels. For the slow-level error, this exchange acted like an additional damping: it removed error energy at a nearly steady relative rate of about 1.2 per model time unit.
Slow-level error-energy removal by net exchange
Approximate nearly steady relative rate at which net slow-to-fast interlevel exchange removed slow-level forecast-error energy in the coupled control.
| Label | Value |
|---|---|
| Coupled control | 1.2 per model time unit |
That result describes the net exchange in the control forecasts, not every instant or every component of the system. It also concerns error energy, not the transfer of the model’s total physical energy. In the Lorenz–96 equations, interlevel exchanges cancel in the budget for total state energy; for forecast errors, the signed exchange can nevertheless redistribute error among levels. Keeping those two budgets distinct is essential to interpreting the result.
Third, “errors move from slow to fast” does not mean fast-level errors cannot affect slow-level predictability. In a separate experiment, the researchers initialized perturbations in the fast variables only. Those perturbations seeded slow-level errors immediately. After that initial transient, the average slow-level error approximately traced the control’s growth curve, but with a delay of about 0.7 model time units. The fast levels could therefore influence the slow level even while the control’s net exchange tended to remove slow-level error energy.
Figure 6 shows the scale-resolved error spectra from forecast replays, alongside the saturation ratios across wavenumbers. It reinforces why a single total-error curve can miss important detail: bands and levels need not approach their reference energies at the same pace. The paper also reports a persistent error deficit at dominant wavenumbers in the middle band. That deficit helps explain why its clock was slower; the explanation comes from the full physical balance, not simply from whichever transformed exchange term looked largest.
Reported linearity of band-mean error clocks
The lowest and highest R² values reported across the fitted band-mean logits in the two regimes; the paper gives this as a range, not as a regime-by-regime breakdown.
| Label | Value |
|---|---|
| Lowest reported fit | 0.965 R² |
| Highest reported fit | 0.993 R² |
The straight-line fits summarize mean growth, but the physical budget clarifies what the clocks alone cannot: which signed terms speed a band up, slow it down, or move its trajectory sideways.
Why This Changes Things
The central conceptual change is to treat forecast-error growth as a path through a space of scale-resolved clocks, not just a climb up a single error curve. The longitudinal coordinate provides a common measure of advancement; the transverse coordinates show whether the distribution of error across bands is evolving in an unusual way. Because the coordinates remain tied to measured error energies, they are interpretable in physical terms rather than being abstract latent variables learned from data.
That can help address a persistent challenge in multiscale prediction. A coarse model or observing network may be judged by total error, while the practical consequence depends on which patterns are wrong. A large error in a rapidly saturating small scale may matter differently from a smaller error in a broad, slowly varying component. Tracking scale-specific clocks could make it easier to compare forecast systems at equivalent stages of error growth, or to identify when a model’s scale interactions cause one band to depart from its expected trajectory.
The approach also cautions against a common shortcut: interpreting correlated growth as evidence of coupling. Independent logistic bands with constant growth rates can have highly correlated clocks because each advances with time. Conversely, physical exchange can be present even when its effect on clock speeds is obscured by other terms. Separating mean timing, residual motion, and launch-to-launch association makes the evidence more discriminating.
There is a useful bridge here to finite-size Lyapunov analysis, which estimates growth rates at specified error amplitudes rather than only in the infinitesimal-error limit. The logit speed is related to the instantaneous growth rate of normalized error, with a correction that becomes important as saturation approaches. It is not itself a measured finite-size Lyapunov exponent, and the authors are careful about that distinction. But the coordinates offer a way to place finite-amplitude growth and scale composition in one diagnostic framework.
A second contribution is methodological transparency. Many data-driven coordinate systems can produce a convenient low-dimensional description, but the researcher may have to infer afterward what the coordinates mean. Here the normalization and reference growth law are chosen explicitly. The energy budget then says how known physical terms contribute. That makes the assumptions inspectable—and makes it possible to change the assumed reference law if logistic growth is not the right baseline.
What’s Next
The strongest caveat is that the demonstration is a controlled model experiment, not a real-world forecast evaluation. Lorenz–96 is valuable precisely because its interacting scales and energy transfers can be isolated, but real atmospheres and oceans have more complicated physics, observation networks, and sources of model error. The two regimes also differ in their climatologies, initial errors, and observation errors, so their comparison does not isolate coupling as the sole changed factor.
The logit itself has a domain restriction. It is defined for normalized error between zero and one, while individual forecasts can exceed their fixed reference energy. Analyses must therefore specify which population of forecasts is included and whether the mean is taken before or after transforming. Those operations are not interchangeable: variation among forecasts adds a heterogeneity correction to the logit of the ensemble mean. The paper derives that correction and warns against counting it twice in the physical budget.
A further open question is how robust the conclusions are to choices of reference energy, band boundaries, and the assumed growth law. The authors motivate the logistic reference theoretically and physically, rather than selecting it only because it fits. Still, a good straight-line fit over a finite interval does not guarantee the same geometry will work across models, forecast lead times, or different definitions of error. The paper explicitly allows other reference laws, such as Gompertz growth, which would produce different clocks.
The next useful test is therefore not simply to apply the transform to more data, but to see whether these coordinates help explain or improve actual forecast decisions: whether they identify which scales are losing predictability, whether their physical budget terms remain stable under changes in resolution and observing systems, and whether they reveal interventions that reduce consequential error rather than merely shifting it between bands. In this study, the coordinates make a subtle point measurable: the direction of net energy exchange and the direction of causal influence need not be the same. That distinction is a promising starting point for understanding how forecast uncertainty travels through systems built from many interacting scales.
Sign in to join the conversation.
Comments (0)
No comments yet. Be the first to share your thoughts.