The Algorithm That Learns Equations Like a Scientist

The most accurate model of the Christchurch Bay Tower’s wave-induced vibrations wasn’t discovered by a physicist, nor by a neural network—but by an algorithm that treats equations like sentences, and learns which ones best describe reality. When researchers embedded a fragment of known physics—Morison’s equation, a 70-year-old formula for fluid forces—into a new Bayesian machine that speaks the grammar of dynamical systems, the result wasn’t just a better fit. It was a model that knew what it didn’t know, quantifying uncertainty not just in parameters, but in the very structure of the equation itself. On wave-loading data from the North Sea, this hybrid “grey-box” model reduced prediction error by 40% compared to the pure physics-based baseline (

Prediction Error on Christchurch Bay Tower Data

Label	Value
Morison (white-box)	0.048
Neural network	0.038
Bayesian TAG (OSA)	0.029
Bayesian TAG (simulation)	0.021

Bayesian TAG models outperform both physics-based and black-box baselines. Simulation-trained models achieve the lowest error.

). That’s not just a statistical win. It’s a glimpse of a future where machines don’t just fit curves—they discover equations with confidence, blending human insight with algorithmic exploration to model everything from climate systems to cardiac rhythms with principled uncertainty.

The Science

At the heart of this advance is a radical reimagining of how we search for equations in data. For decades, system identification—the process of deriving mathematical models from observations—has been split between two worlds: physics-based “white-box” models, built from first principles, and data-driven “black-box” models, like neural networks, that learn patterns without interpretable structure. The former are explainable but rigid; the latter flexible but opaque. The method introduced by Christopher A. Lindley, Nikolaos Dervilis, and Keith Worden bridges this gap using a tool borrowed from computational linguistics: Tree-Adjoining Grammars (TAGs).

TAGs were originally developed to model the syntax of natural language. Just as a sentence like “The cat sat on the mat” can be decomposed into a tree of grammatical components—noun phrases, verbs, prepositions—so too can a dynamical equation be broken into a tree of mathematical operations: additions, multiplications, time shifts, and nonlinear terms. In this framework, a model class—say, all possible polynomial nonlinear autoregressive models with exogenous inputs (p-NARX)—is encoded not as a list of candidate equations, but as a finite set of “elementary trees” and rules for combining them. One tree might represent a single term like $\theta_1 u_{t-1} y_{t-2}^2$; another might specify how to add such terms together. By adjoining and substituting these building blocks, the grammar generates an infinite space of valid equations, all structurally consistent with the underlying assumptions.

Prior work used evolutionary algorithms to search this space, evolving equation trees through mutation and crossover to find a single “best” model. But evolution gives only a point estimate—like finding one path through a forest without knowing how many other viable paths exist. The breakthrough here is to place the entire TAG framework within a Bayesian paradigm. Instead of seeking a single model, the algorithm infers a probability distribution over all possible equation structures and their parameters. This is achieved through Reversible-Jump Markov Chain Monte Carlo (RJ-MCMC), a technique capable of jumping between models of different sizes—adding or removing terms—while preserving the detailed balance required for valid sampling.

The key innovation is defining a generative prior over tree structures. Simpler equations—those with fewer terms or lower-order nonlinearities—are assigned higher prior probability, embodying Occam’s razor: the idea that simpler explanations should be preferred unless complexity is justified by the data. As the MCMC sampler explores the space of trees, it doesn’t just accept or reject models based on fit; it weighs them by how well they explain the data relative to their complexity. The result is a posterior distribution that captures not only which equations fit well, but which are plausible given the data and the principle of parsimony.

Two training objectives are implemented. The first uses a one-step-ahead (OSA) prediction error, where the model predicts the next observation using past measured values. This leads to a conjugate Bayesian update, allowing efficient parameter sampling. The second uses simulation-based (or “closed-loop”) error, where predictions are fed back into the model, mimicking real-world forecasting. This is more realistic but harder to compute, so the authors use likelihood-free inference—comparing simulated outputs directly to data without requiring an explicit likelihood function.

What They Found

The method was tested on three problems. First, a simulated p-NARX system with known structure. The Bayesian TAG correctly identified the true equation as the highest-probability model, demonstrating that the approach can recover ground-truth dynamics from noisy data. More impressively, the posterior distribution revealed alternative models that fit nearly as well, exposing structural uncertainty—something point-estimate methods cannot do.

Second, the “Silverbox” benchmark, a real-world nonlinear electronic circuit. Here, the Bayesian TAG with OSA training achieved a Normalized Mean Squared Error (NMSE) of 0.0012 on the test set, competitive with state-of-the-art methods. But the real value wasn’t just accuracy—it was interpretability. The top models recovered physically meaningful terms, such as cubic stiffness nonlinearities, without being told to look for them.

The most compelling result came from the Christchurch Bay Tower (CBT), a 48-meter steel structure in the North Sea used to study wave loading on offshore platforms. The baseline model used Morison’s equation, a physics-based formula that estimates hydrodynamic forces from wave velocity and acceleration. While grounded in fluid dynamics, it’s known to be incomplete—missing nonlinear interactions and structural dynamics.

The researchers embedded Morison’s equation as a fixed initial tree within the TAG grammar, creating a “grey-box” model: part physics, part data-driven discovery. The Bayesian sampler was then allowed to extend this base with additional terms—adjoining new branches to the equation tree.

The result? The best models added terms involving higher-order velocity products and delayed accelerations, capturing nonlinear drag effects missed by the original Morison formulation. On the test set, the median NMSE dropped from 0.048 (Morison alone) to 0.029—a 40% improvement (

Prediction Error on Christchurch Bay Tower Data

Label	Value
Morison (white-box)	0.048
Neural network	0.038
Bayesian TAG (OSA)	0.029
Bayesian TAG (simulation)	0.021

Bayesian TAG models outperform both physics-based and black-box baselines. Simulation-trained models achieve the lowest error.

). Even more striking, models trained with the simulation objective—closer to real forecasting conditions—achieved NMSE as low as 0.021, outperforming all previous methods on this benchmark.

Prediction Error on Christchurch Bay Tower Data

Label	Value
Morison (white-box)	0.048
Neural network	0.038
Bayesian TAG (OSA)	0.029
Bayesian TAG (simulation)	0.021

Bayesian TAG models outperform both physics-based and black-box baselines. Simulation-trained models achieve the lowest error.

Figure 7: NMSE on the CBT test set and median number of terms for each method, training objective and $\lambda_t$, compared against the Morison’s equation (white-box) baseline.

Source: Lindley et al. (2026), Figure 7.

The chart shows that Bayesian TAG models (blue and red bars) consistently outperform both the white-box Morison model and a black-box neural network baseline. Simulation-trained models (red) achieve the lowest error, while OSA-trained models (blue) are slightly higher but more stable. Crucially, the Bayesian approach does this with fewer terms than non-Bayesian TAG methods, confirming that the Occam’s razor prior prevents overfitting.

Figure 7 from the paper, adapted for clarity.

Why This Changes Things

This work isn’t just about better models for offshore engineering. It represents a shift in how we think about scientific discovery in the age of data. For centuries, equation discovery was a human endeavor: Newton watching apples, Maxwell unifying electricity and magnetism, Schrödinger writing wave functions. In the 21st century, we’ve offloaded much of this to algorithms—but often at the cost of interpretability. Neural networks predict climate patterns or protein folds with stunning accuracy, but they don’t give us equations we can write on a chalkboard.

Bayesian TAGs offer a third way. They don’t replace human insight—they amplify it. By allowing scientists to encode known physics as fixed grammar rules, the method turns domain knowledge into a scaffold for discovery. You’re no longer starting from scratch; you’re saying, “Here’s what we know—now help me find what we’re missing.”

This is particularly powerful in fields where first principles are incomplete. Climate models, for instance, contain parameterizations for cloud formation and ocean mixing—rough approximations because the full physics is too complex to simulate. A Bayesian TAG could start with these existing parameterizations and search for data-driven corrections, quantifying uncertainty in both the corrections and the original assumptions.

Similarly, in biomedicine, models of glucose-insulin dynamics or neural firing are often simplified for tractability. Embedding these as initial trees could allow algorithms to discover more accurate, yet still interpretable, extensions—without sacrificing the clinical insight that comes from a transparent equation.

The ability to quantify structural uncertainty is another game-changer. Traditional model selection picks one “best” equation. But in complex systems, multiple models may fit equally well, implying that the data alone can’t distinguish between competing mechanisms. The Bayesian TAG doesn’t hide this ambiguity—it surfaces it. A policymaker using such a model to project sea-level rise wouldn’t get a single curve, but a distribution of plausible equations, each with its own prediction. That’s not weakness—it’s honesty.

And unlike black-box methods, every model produced by the TAG is a symbolic expression, readable and analyzable. You can take the top 100 models from the posterior, cluster them by structure, and ask: What do the best models have in common? Do they all include a $y_{t-1}^3$ term? Then maybe cubic nonlinearity is essential. Do they disagree on the role of input delays? Then more data or experiments might be needed. This kind of meta-analysis is impossible with neural networks.

What’s Next

The method isn’t without challenges. RJ-MCMC is computationally intensive, and scaling it to high-dimensional systems—say, a 100-variable climate model—remains an open problem. The current implementation focuses on scalar time series; extending it to multivariate or partial differential equations would require richer grammars.

Another limitation is grammar design. The power of the approach depends on crafting a grammar that’s expressive enough to capture relevant dynamics, but constrained enough to make search feasible. While the paper shows how to build grammars for p-NARX models, creating grammars for, say, ecological food webs or financial markets would require deep domain expertise. Future work could explore automated grammar induction, or libraries of reusable grammar modules.

The authors also note that the simulation-based objective, while more realistic, is harder to optimize due to the lack of gradients. Integrating differentiable programming or neural surrogates could accelerate training. And while the current prior favors simpler trees, more sophisticated priors—encoding physical constraints like conservation laws or symmetries—could further guide discovery.

Perhaps the most exciting direction is active discovery. Imagine a robot scientist that doesn’t just analyze existing data, but designs experiments to reduce structural uncertainty. It could run a simulation, find that two competing models fit equally well, and then suggest a new measurement that would distinguish between them—like a physicist planning their next lab run. Bayesian TAGs, with their explicit representation of model space, are a natural fit for such closed-loop science.

We’re moving from an era where algorithms predict to one where they propose. Not just “What will happen?” but “What could the law be?” The Christchurch Bay Tower didn’t just teach us about waves—it taught us that machines can now help us write the equations of nature, with humility, uncertainty, and a grammar all their own.