On London’s first COVID-19 wave, a physics-informed neural network forecast better than a stochastic mechanistic model built around the same disease equations. But neither model beat a simple data-driven baseline. In simulated outbreaks, the ranking flipped: the mechanistic model was slightly more accurate. The lesson is not that one kind of model has won. It is that a forecasting method can look strong in a controlled world and weaker in the one public health officials actually face.

That distinction matters. Forecasts guide decisions about staffing, hospital capacity and interventions, often when the data are incomplete and the epidemic is changing. A model that is elegant, interpretable or technologically sophisticated is not necessarily the model that will make the most useful forecast. Martina Rama and colleagues set up a deliberately shared comparison to ask whether a physics-informed neural network, or PINN, offers a real advantage over a more established stochastic model. Their result is a useful mixture of promise and restraint.

The Science

A compartmental epidemic model divides a population into groups according to disease status. In the familiar SIR model, people are susceptible, infectious or recovered. This study used a more detailed SEIRD model: susceptible, exposed, infectious, recovered and dead. “Exposed” accounts for the delay between infection and becoming infectious; a further compartment tracked deaths that had occurred but were not yet officially reported. The model also split the population into 16 age groups, because contact patterns and fatality risks differ by age.

The two approaches shared this epidemiological core. They also used the same forecasting rhythm: take the observations available up to a given week, estimate unknown parameters, forecast future weeks, then repeat when new data arrive. Such a rolling forecast resembles the real task of updating an outlook as surveillance data accumulate. For the London analysis, the models used weekly deaths; for the simulated cases, they used weekly new infections.

The approaches differed in how they solved and calibrated the model. The mechanistic framework treated disease spread as a stochastic process: outcomes could vary from one simulation to another even with the same parameters. It calibrated its model using Approximate Bayesian Computation Sequential Monte Carlo, a way to infer plausible parameters by comparing simulated outcomes with observed data rather than writing down a conventional likelihood.

The PINN approach learned a fast approximation to the solutions of the same differential equations. A neural network is “physics-informed” when its training is constrained not only by observed data but also by the equations that describe the system. Here, the network was trained in advance across a range of epidemic parameters. During each forecasting round, the researchers estimated parameters using Markov chain Monte Carlo, then evaluated the trained network to generate predictions.

That design makes the comparison more informative than a contest between unrelated models. If one method had used age structure and the other had not, or if their forecast pipelines differed, it would be difficult to know what explained a performance gap. By sharing core assumptions, the authors aimed to isolate how the two computational frameworks behaved.

There is a trade-off in that choice. A shared model structure makes the methods easier to compare, but it does not represent every force that shaped COVID-19. The study deliberately excluded changing interventions and mobility patterns. Both methods therefore had to infer the epidemic from observed incidence under a simplified account of transmission, rather than being given policy changes or movement data as additional inputs.

What They Found

The researchers first tested two synthetic outbreaks, generated from a known age-structured epidemic model. One had a basic reproduction number, , of 1.5; the other had an of 2.0. The basic reproduction number is the average number of people one infected person would infect in a largely susceptible population. In broad terms, the higher value produced a faster, larger wave. Because the data-generating process was known, the authors could check whether the models recovered the parameters and forecast the trajectories.

In these controlled scenarios, both frameworks estimated accurately and performed better than the baseline. The stochastic mechanistic method was slightly more accurate overall. The PINN, meanwhile, produced narrower uncertainty intervals and sharper parameter estimates. That sounds like a strength—but a narrow interval is useful only if it still covers plausible outcomes. The authors describe the PINN forecasts in the synthetic settings as overconfident. Precision and reliability are not the same thing.

The real-data test changed the picture. The team examined weekly deaths during London’s first COVID-19 wave, from mid-February to mid-July 2020. The PINN achieved lower forecasting errors than the stochastic mechanistic approach. Yet both underperformed the simple data-driven baseline, which did not rely on an epidemic model. In other words, the neural framework won the head-to-head contest but did not win the broader forecasting contest.

That is the central result—and it is easy to misread. The real-data finding does not show that PINNs are generally superior, just as the synthetic result does not show that mechanistic models are always better. Instead, it shows that rankings depend on the setting. A carefully specified model can be tested against a synthetic world that obeys its assumptions; real surveillance data contain noise, delays and changing conditions that the model may not capture.

The paper evaluates probabilistic forecasts, not just a single line through the data. That matters because public health decisions need a sense of uncertainty: a forecast of 500 deaths next week is not equivalent to a forecast whose plausible range runs from 100 to 1,000. But the supplied paper excerpt does not provide the numerical score tables or plotted values. For that reason, there are no charts here: turning the qualitative findings into bars or lines would require inventing numbers. The paper’s comparison is best summarized by its reported direction of results, not by fabricated magnitudes.

Why This Changes Things

Epidemic forecasting is not a beauty contest between equations and neural networks. Mechanistic models offer an explicit account of disease progression. Their assumptions can be inspected: how long people remain exposed, how infectiousness changes, and how contacts differ across ages. That structure is useful for asking “what if” questions, such as how a change in contact rates might alter transmission. It can also make the model’s limitations easier to locate.

Neural networks are more flexible pattern learners, but that flexibility can make their outputs harder to interpret and their performance more dependent on the data and training setup. PINNs try to occupy a middle ground. They learn from data while being nudged to obey governing equations. In principle, this can help when observations are sparse, noisy or incomplete, while retaining some connection to the biology.

The study’s real-data result suggests that this hybrid strategy may help when the idealized model does not fully describe the observed epidemic. But the fact that both approaches lost to the baseline is just as important. A simple baseline can be hard to beat, especially at short horizons or when the data’s recent trend carries more predictive information than a fixed mechanistic structure. The baseline is not necessarily a better explanation of disease; it is a reminder that forecasting accuracy and scientific understanding are different goals.

This distinction has practical consequences. A public health agency choosing a forecast system should not ask only whether a model incorporates disease biology. It should ask how the model performs against simple reference forecasts, whether its uncertainty intervals are calibrated, how quickly it can update, and whether it works across different waves and data systems. A sophisticated model that takes too long to retrain or gives falsely narrow uncertainty may be less useful during an emergency than a modest model that updates reliably.

The shared setup also exposes a broader challenge in model evaluation: fairness and realism can pull in opposite directions. Holding assumptions constant makes the comparison cleaner. Yet omitting interventions and mobility makes it less like the circumstances in which governments need forecasts. The study is therefore a benchmark of two frameworks in a minimal setting, not a verdict on the best operational COVID-19 forecasting system.

The finding that PINNs gave sharper estimates in synthetic scenarios should also be handled carefully. Better recovery of a parameter is valuable when the parameter is identifiable from the data and the model is adequately specified. But the real world can confound parameters: different combinations of transmission, behavior and reporting delay may generate similar observed case or death curves. A crisp estimate can conceal that ambiguity rather than resolve it.

What’s Next

The next step is broader, repeated benchmarking. The authors themselves argue for further comparisons. Such tests should span more outbreaks, geographies and surveillance systems, and include realistic shifts in interventions, contact behavior, reporting practices and variants. Forecasts should be judged at multiple horizons, with both point accuracy and uncertainty calibration assessed. An interval that is too wide may be unhelpful; one that is too narrow can mislead decision-makers.

Operational considerations deserve equal attention. The paper’s PINN is trained once across a parameter grid and reused, while parameters are re-estimated as observations arrive. That is an appealing way to avoid retraining the network every week. But real deployment also depends on compute requirements, reproducibility, ease of updating assumptions and how quickly analysts can explain a change in the forecast. The supplied results do not establish that either method is cheaper or faster in a public-health workflow, so those questions remain open.

There is also a question of what information should enter the model. The comparison intentionally relies on incidence data alone, but COVID-19 forecasting can draw on hospital admissions, testing, wastewater, mobility and intervention histories. These sources may improve forecasts, though they bring their own delays, biases and missingness. A useful future test would assess whether PINNs gain more than mechanistic models from combining such streams—or whether simpler methods remain competitive.

For now, the study offers a welcome correction to the idea that adding neural networks automatically improves epidemic forecasts. PINNs beat their mechanistic counterpart on London’s real first wave, but neither beat the baseline; the conventional model did better in the synthetic worlds. That mixed result is not a failure of the comparison. It is exactly what a serious benchmark should reveal: where a method looks promising, where it struggles, and what evidence is still missing before it earns a place in the tools used to prepare for the next outbreak.