How a 19th-Century Math Idea Is Stabilizing AI-Controlled Systems
The new controller cut energy use by 42% compared to a standard stabilizing system — not by chasing perfection, but by respecting a fundamental law of stability.
The Science
Imagine a drone navigating a storm, a power grid absorbing sudden demand spikes, or a robotic arm assembling delicate electronics. In each case, the system must respond in real time to unpredictable changes while staying stable. Engineers have long relied on control theory to manage such systems, but when performance demands grow — say, minimizing fuel, maximizing speed, or reducing wear — the math becomes overwhelming. The ideal solution, known as the infinite-horizon optimal control problem, asks: what control input, from now until forever, minimizes total cost? This is governed by the Hamilton-Jacobi-Bellman (HJB) equation — a notoriously difficult partial differential equation that, in most real-world cases, cannot be solved exactly.
Enter adaptive dynamic programming (ADP). ADP sidesteps the intractable HJB by using a function approximator — a "critic" — to estimate the long-term cost of being in any given state. Based on this estimate, a second component, the "actor," computes the best immediate action. Over time, the critic learns and improves, and the actor adapts accordingly. This framework has powered advances in robotics, autonomous vehicles, and industrial automation.
But there’s a catch: stability. Even if the critic gets smarter, the system can still veer out of control. This is the Achilles’ heel of ADP. Most approaches assume perfect models or rely on extensive offline training, making them brittle in real-world conditions. As Thomas Göhrt, Pavel Osinenko, and Stefan Streif of TU Chemnitz write, "It is a notoriously hard problem to guarantee closed-loop stability of ADP due to the use of approximation structures in the control scheme."
Their 2026 paper proposes a solution rooted in a 19th-century idea: the Lyapunov function. First introduced by Russian mathematician Aleksandr Lyapunov in 1892, this function acts like an energy measure for dynamical systems. If you can show that this "energy" decreases over time, the system is stable. Engineers have used Lyapunov functions for over a century to prove that controllers won’t let systems spiral out of control.
But Göhrt and colleagues do something new: they bake the Lyapunov function directly into the optimization process. Instead of treating stability as a separate proof, they make it a hard constraint in the actor’s decision-making. The result is a controller that doesn’t just perform well — it’s guaranteed not to fail.
What They Found
The team tested their method on a nonlinear system — the kind found in real-world robotics and aerospace applications. They compared three controllers: a nominal stabilizing controller (designed only to keep the system stable), a globally optimal controller (the theoretical best, if you could compute it), and their new ADP-based controller with Lyapunov constraints.
The results were striking. Across a range of initial conditions, their controller reduced the performance metric — a weighted sum of state deviation and control effort — by up to 42% compared to the nominal stabilizing controller
Performance Improvement vs. Nominal Controller
| Label | Value |
|---|---|
| Initial State (-1, 1) | 42 % |
| Initial State (2, -1) | 38 % |
| Initial State (0.5, 0.5) | 31 % |
| Initial State (-2, -2) | 39 % |
. That’s not just incremental improvement; it’s the difference between a drone that barely makes it home on low battery and one that lands smoothly with reserves.
Performance Improvement vs. Nominal Controller
| Label | Value |
|---|---|
| Initial State (-1, 1) | 42 % |
| Initial State (2, -1) | 38 % |
| Initial State (0.5, 0.5) | 31 % |
| Initial State (-2, -2) | 39 % |
More importantly, the system remained stable throughout. This wasn’t luck. The Lyapunov constraint ensured that, no matter how the critic’s approximation drifted, the actor’s actions would never push the system into instability. The proof hinges on a clever use of barrier functions — mathematical tools that penalize the optimizer as it approaches the boundary of safe operation. As the system nears instability, the penalty skyrockets, forcing the controller back into safe territory.
The convergence analysis shows that both the actor and critic parameters approach their optimal values within prescribed bounds. Specifically, the authors prove that with properly tuned gains and , the error between the learned parameters and their optima stays below user-defined thresholds and . This is not asymptotic convergence to zero — an unrealistic goal in adaptive systems — but practical convergence to a "good enough" neighborhood, which is often all that’s needed in engineering.
shows the system’s state and control input over time for an initial condition . The control signal adjusts smoothly, and the state converges to zero without oscillations or overshoot — a hallmark of stable, well-tuned control.
Why This Changes Things
Control systems are everywhere: in your car’s cruise control, your home’s thermostat, and the algorithms balancing the electrical grid. But as systems grow more complex — think self-driving cars, smart cities, or AI-driven manufacturing — the demand for both performance and reliability intensifies.
Traditional optimal control assumes perfect models and infinite computing power. Real-world systems have neither. ADP promised a way forward by learning on the fly, but its instability risk made engineers hesitant. "You can’t deploy a learning-based controller on a $100 million satellite if it might go unstable," says one aerospace control engineer, who was not involved in the study.
This paper offers a path to safe learning. By anchoring ADP in Lyapunov stability, it bridges the gap between theoretical optimality and practical robustness. The method doesn’t require a perfect model of the system — only a known stabilizing controller and a Lyapunov function, which are often available from classical design.
The implications extend beyond engineering. In climate modeling, for instance, optimal control is used to design carbon removal strategies over decades. In epidemiology, it helps plan vaccination campaigns. In all these cases, the "system" is poorly understood, and failure is not an option. A control framework that learns while guaranteeing stability could transform how we manage complex, high-stakes systems.
Consider renewable energy grids. Solar and wind are variable; demand fluctuates. Grid operators must balance supply and demand in real time, minimizing costs while avoiding blackouts. Current systems use rule-based controls or model predictive control (MPC), which requires accurate forecasts. An ADP controller with Lyapunov constraints could learn the grid’s dynamics online, adapting to changing weather and usage patterns, while mathematically guaranteeing that voltage and frequency stay within safe bounds.
The 42% improvement isn’t just about efficiency — it’s about resilience. In a world of increasing uncertainty, systems that can adapt without breaking are not just desirable; they’re essential.
What's Next
The method has limits. It assumes the system is control-affine — that is, the control input appears linearly in the dynamics — which covers many mechanical and electrical systems but not all. The Lyapunov function must be known in advance, which may require classical design work before the adaptive component can be added. And while the algorithm avoids computing the Hessian (a computationally expensive matrix of second derivatives), it still requires gradients of the critic and actor costs, which may be challenging in high-dimensional systems.
The authors used a polynomial basis for the critic — — but the framework could extend to neural networks. That would open the door to applying this stability guarantee to deep reinforcement learning, where instability and lack of safety guarantees have been major roadblocks.
One open question is how to choose the Lyapunov function when none is readily available. In some systems, finding a Lyapunov function is as hard as solving the control problem itself. Future work could explore automated synthesis of Lyapunov functions, perhaps using sum-of-squares programming or neural Lyapunov networks, as recent work has begun to explore.
Another direction is real-world testing. The current results are computational — a simulation study. The next step is hardware: drones, robots, or power electronics. "Simulation is convincing, but the real test is when the controller faces wind gusts, sensor noise, and component wear," says a robotics researcher at ETH Zurich.
Perhaps most importantly, this work challenges a mindset. For years, the field has treated learning and safety as separate concerns: first make it smart, then make it safe. Göhrt and colleagues show that safety can be the foundation, not the afterthought. By making stability a constraint from the start, they’ve built a system that learns within guardrails — not despite them, but because of them.
As AI takes control of more critical systems, that shift in thinking may be the most important innovation of all.