The Smart Way to Poke a Black Box: How Knowing Less Can Mean Doing Less

The Problem of Learning How Machines Think

Imagine you're an engineer handed a black box with wires coming out of it. You know it's governed by some linear rules—you just don't know which ones. Your job is to figure out those rules by poking the box and watching how it responds. But here's the catch: you want to do this as quickly and efficiently as possible, with the fewest pokes, in the noisiest real-world conditions.

This is the central problem of system identification, a field that sits at the intersection of control theory, statistics, and applied mathematics. Every time a chemical plant tunes its reactors, a neuroscientist maps neural circuits, or an aerospace engineer characterizes a satellite's dynamics, they're solving some version of this puzzle. And the key question isn't just how to identify the system—it's what input signal should you use to probe it?

A new paper by Shakouri, Van Waarde, and Camlibel from the University of Groningen tackles this question head-on. Their work, "Experiment Design for Set-membership Identification: From Prior Knowledge to Universal Inputs," published in July 2026, makes significant progress on a deceptively simple problem: how do you design input signals that will work no matter which specific system you're dealing with, as long as it fits within what you already know?

The answer, it turns out, depends critically on what you know in advance—and the paper reveals surprising gaps in our conventional wisdom about what kinds of inputs are necessary.

The Fundamental Puzzle of Experiment Design

When you want to identify a system, you have to think about the data you'll collect. You're applying some input signal $u(t)$ to the system, and you observe the output $y(t)$ (or, in the state-space setting this paper studies, the state $x(t)$). The true system follows the linear dynamics:

$$x(t+1)=Ax(t)+Bu(t)+w(t)$$

where $A$ and $B$ are the unknown matrices you want to learn, and $w(t)$ is noise. Your goal is to use the collected data $(u[0,T-1],x[0,T])$ to estimate $A$ and $B$.

The question is: what input signal should you choose?

The standard answer in control theory is persistently exciting inputs. Intuitively, this means your input signal must be "rich enough" to excite all the system's natural modes. If you think of the system as having $n$ states, a persistently exciting input of order $n+1$ will probe enough of the system's behavior to identify it uniquely. For an $n$-state system with $m$ inputs, this requires collecting at least $nm+n+m$ data samples.

This requirement comes from a celebrated result known as the fundamental lemma of Willems and colleagues, which established that persistently exciting inputs are sufficient for exact identification of linear systems. For nearly two decades, this has been the gold standard.

But here's the problem the new paper identifies: persistency of excitation is sufficient but is it necessary? And more practically: if you know something about your system in advance—say, that certain parameters are bounded, or that the system is stable—can you do better than the general prescription?

What "Prior Knowledge" Changes

The paper introduces a crucial element that has been underutilized in experiment design: prior knowledge. Before you run an experiment, you usually know something about your system. Perhaps you're identifiying a class of chemical reactors that you know are always stable. Perhaps you've narrowed down the system's order to a specific range. Perhaps you know the input matrix $B$ has a particular structure.

This prior knowledge is encoded as a set ${\Sigma }_{pk}$ of all systems consistent with what you know. The true system $(A_{true},B_{true})$ must lie somewhere in this set, but you don't know where.

The key concept the paper develops is that of a universal input. An input signal $u[0,T-1]$ is universal for a given prior knowledge set if, when you apply it to any system in that set, the resulting data is guaranteed to be informative enough for identification.

This is a powerful idea: a universal input works for the whole class of possible systems, not just for one specific instance. If you can find such an input, you've solved the experiment design problem in a robust way.

The Surprising Necessity Result for Single-Input Systems

The paper's first major result (Theorem 7) deals with single-input systems—those with $m=1$. For these systems, the authors show that if your prior knowledge set is open and contained within the controllable systems, then every universal input must be persistently exciting of order $n+1$.

This is a necessity result: it tells you that for single-input systems, you can't escape the persistency of excitation requirement, even if you have rich prior knowledge. The lower bound on data samples remains $T\ge nm+n+m$.

The proof hinges on the structure of single-input systems. When you have only one input, you have less freedom to design clever probing signals. The rank conditions required for identifiability force you into the persistently exciting regime.

This is both good news and bad news. Good news because it confirms that the conventional approach is indeed necessary for this important class of systems. Bad news because persistency of excitation can be impractical in many real-world scenarios—it requires sustained, nontrivial inputs that may violate safety constraints, damage equipment, or be too expensive to implement.

The Multi-Input Surprise

Here's where things get interesting. For multi-input systems, the story changes dramatically.

The authors construct an example (Example 8) that demonstrates the existence of universal inputs that are not persistently exciting. Consider a simple system with $n=1$ state and $m=2$ inputs. The prior knowledge set is:

$${\Sigma }_{pk}=\{(A,[B_1{B}_2])\mid \alpha {\beta }_2+{\beta }_1\ne 0\}$$

This is the set of all single-state systems where the product of the state transition coefficient and the second input gain, plus the first input gain, is nonzero. This is a very mild condition—almost all single-state systems satisfy it.

Now consider an input signal that takes only three distinct values across four time steps: zero, then the second input high, then the first input high, then zero again. This input is clearly not persistently exciting (its Hankel matrix of depth 2 fails to have full row rank).

But the authors show that this input is universal: apply it to any system in ${\Sigma }_{pk}$, with any initial state, and the resulting data enables exact identification.

This is remarkable. The conventional theory would say you need more data, a richer input. But by exploiting prior knowledge—specifically, the structure of what you know about the system—you can get away with less.

Theorem 9: A General Condition for Universal Inputs

The authors don't stop at counterexamples. They develop a systematic method for designing universal inputs based on a new rank condition.

Theorem 9 states that for any prior knowledge set contained within controllable systems, an input $u[0,T-1]$ is universal for identification if:

$$\mathrm{rank}{M}_{A,B}{D}_A{\mathcal{H}}_{n+1}(u[0,T-1])=n+m$$

for all $(A,B)\in {\Sigma }_{pk}$, where $M_{A,B}$ is a matrix built from the system dynamics, $D_A$ encodes the characteristic polynomial of $A$, and ${\mathcal{H}}_{n+1}$ is the $(n+1)$-depth Hankel matrix of the input.

This condition is less demanding than full persistency of excitation because the matrices $M_{A,B}$ and $D_A$ can compensate for rank deficiencies in the Hankel matrix. The key insight is that these compensating factors depend on the prior knowledge set, so different prior knowledge enables different flexibility in input design.

The theorem also comes with a sample complexity bound:

$$T\ge 2n+m$$

Compare this to the persistency of excitation requirement: $T\ge nm+n+m$. For multi-input systems with $m>1$, the new bound can be substantially smaller. When $m=2$ and $n=2$, for instance, you need $T\ge 8$ samples under the conventional approach, but only $T\ge 6$ under Theorem 9.

The authors emphasize that this improvement isn't just theoretical. Fewer required samples mean shorter experiments, less wear on equipment, reduced exposure to potentially dangerous inputs, and lower costs. In settings like clinical trials or large-scale network testing, these savings compound significantly.

Beyond Controllability: The Limits of Prior Knowledge

The paper's analysis extends beyond prior knowledge of controllability. The authors investigate what other types of prior knowledge enable universal experiment design—and, crucially, what types don't.

Their Proposition 6 establishes that for open prior knowledge sets, universal experiment design for exact identification is possible if and only if all systems in the set are controllable. This means that if your prior knowledge includes any system that can't be reached from any initial state using the available inputs, you're out of luck: there won't be any universal input that works for the whole set.

This is an important boundary condition. It tells experiment designers that before they can hope for universal inputs, they need to ensure their prior knowledge is confined to controllable systems. Otherwise, there will always be some system in the set for which any given input fails to yield informative data.

When the System Fights Back: Noisy Data

Real systems are noisy. The paper tackles this reality by extending its analysis to the case where the process noise $w(t)$ is bounded in magnitude: $|w(t)|\le ϵ$.

The goal here isn't exact identification—noise prevents that—but identification up to a desired accuracy. The authors define data as ${\Sigma }_{pk}$-informative for $\rho$-accuracy identification if the radius of the feasible set (the Chebyshev radius) is at most $\rho$. Intuitively, this means all systems consistent with the data and prior knowledge are within $\rho$ of each other, so any estimate you make will have error at most $\rho$.

The noisy setting requires new techniques. The authors show how to design inputs that guarantee $\rho$-accuracy identification regardless of the noise realization (as long as it stays within bounds). This involves extending the rank conditions from the noise-free case to a robust version that accounts for worst-case noise.

The results (Theorems 21, 22, and 24) provide methods for designing universal inputs in the noisy setting. The key finding is that the same principles that enable efficient experiment design in the noise-free case carry over: prior knowledge that constrains the system matrices can reduce the required input richness and sample complexity.

Perhaps most remarkably, the authors show that exact identification in noise is sometimes possible—not just approximately accurate identification. This requires prior knowledge that is uniformly discrete: the set of feasible systems cannot have accumulation points. In practical terms, this means you need enough prior knowledge to rule out arbitrarily similar systems. When such discrete prior knowledge is available, the paper provides an experiment design method (Theorem 30) that achieves exact recovery despite the noise.

Why This Matters: Applications Across Domains

The implications of this work extend across many fields where system identification is essential.

In chemical and biological systems, experiments are expensive and often constrained by safety limits. You can't arbitrarily excite a bioreactor if doing so might cause harmful byproducts. The paper's results suggest that by carefully incorporating what you know about the system's structure—stability, bounded reaction rates, known pathway constraints—you can design shorter, safer experiments that still yield accurate models.

In clinical settings, identifying the dynamics of physiological systems (blood glucose regulation, drug response curves) often requires inputs that can't exceed therapeutic thresholds. A diabetic patient's insulin sensitivity can be characterized without sending the system into dangerous territory, provided you know enough about the underlying physiology to design appropriately constrained inputs.

In large-scale networks—power grids, sensor networks, robotic swarms—running persistently exciting experiments is often infeasible. Coordinating multiple actuators to produce rich excitation across a distributed system is logistically challenging and power-hungry. The paper's framework suggests that if you know something about network topology or coupling structure, you can design more parsimonious experiments.

In aerospace systems, flight tests are enormously expensive. Every minute of data collection costs money and introduces wear on the vehicle. The paper's results support the intuition that you don't need to fully excite every mode if you already know something about the aircraft's aerodynamics from design principles.

The Hands-Off Finding

One particularly striking application highlighted in the paper (Section 7.1) involves hands-off inputs: signals that take zero value over significant intervals of the experiment horizon. Traditional persistency of excitation would seem to rule these out—if the input is zero, how can it excite the system?

But the authors show that hands-off inputs can be universal under appropriate prior knowledge. If you know, for instance, that the system is stable (all eigenvalues inside the unit circle), then periods of zero input allow the system's natural response to reveal information about its dynamics. The state doesn't stay frozen during these periods—it decays toward equilibrium in ways that encode information about the system matrix $A$. By interleaving brief excitation bursts with longer observation periods, you can identify the system with less total input than persistency of excitation would require.

This finding directly addresses practical concerns about input constraints. In many real systems, sustained nonzero inputs are physically impossible or undesirable. The ability to use intermittent, constrained inputs while still achieving accurate identification opens up experimental regimes that were previously inaccessible.

Sample Efficiency: A Quantitative Comparison

The paper's theoretical contributions translate into concrete improvements in sample efficiency. The authors provide benchmark examples (Section 7) that illustrate the gains.

In the multi-input case, the gap between the conventional approach and Theorem 9 can be substantial. For a system with $n$ states and $m$ inputs, the traditional bound is:

$$T_{PE}\ge nm+n+m$$

while the new bound from Theorem 9 is:

$$T_{new}\ge 2n+m$$

For systems where $nm$ dominates (high input dimension or high state dimension), the improvement is significant. A system with $n=10$ states and $m=5$ inputs would require at least 80 samples under the conventional approach, but potentially only 25 under the new framework.

The authors also note that for certain structured prior knowledge sets, even tighter bounds may be achievable. The examples in the paper demonstrate that the $T\ge 2n+m$ bound is tight in some cases—it can be met by actual universal inputs. This suggests the analysis has reached a fundamental limit rather than just an improvement over previous methods.

Limitations and Open Questions

Intellectual honesty requires acknowledging what the paper doesn't do.

First, the results are confined to linear time-invariant systems. Extending the framework to nonlinear systems—where the identification problem is substantially harder and prior knowledge is harder to formalize—remains an open challenge.

Second, the paper assumes perfect knowledge of the system order $n$. In practice, this may not be available, and misspecifying the order can lead to incorrect identification.

Third, the focus is on offline experiment design: the entire input signal is designed before the experiment begins, then applied without modification. Online methods, where the input is adapted based on accumulating data, are mentioned as a related approach but not developed here. For some applications, online design may be more practical.

Fourth, the computation of universal inputs based on Theorem 9 requires solving optimization problems over the prior knowledge set ${\Sigma }_{pk}$. For complex prior knowledge (high-dimensional parameter bounds, intricate algebraic constraints), this optimization may be computationally challenging.

Finally, the paper's results are primarily theoretical. Empirical validation on real physical systems—testing whether the designed inputs perform as predicted in the presence of unmodeled dynamics, nonlinearities, and practical constraints—is left for future work.

What This Opens Up

Despite these limitations, the paper makes several conceptual advances that will shape future work.

By systematizing the role of prior knowledge in experiment design, it provides a framework for understanding when and why efficient identification is possible. The contrast between single-input and multi-input systems reveals that the structure of the input matrix $B$ fundamentally constrains what's achievable—and that exploiting this structure can yield practical benefits.

The concept of uniform discreteness for exact noisy identification is a new theoretical contribution with potential applications in calibration and precision engineering. If you know enough about your system to rule out accumulation points in the parameter space, you can achieve perfect identification even with bounded noise—a surprising result that challenges the conventional wisdom that noise inevitably limits accuracy.

More broadly, the paper demonstrates that the fundamental lemma of Willems et al., while correct and powerful, is not the final word on experiment design. By viewing it as a special case of a more general theory—one that properly accounts for prior knowledge—researchers can find better ways to identify systems in practical settings.

Looking Forward

The path ahead involves several directions.

Algorithmic development is needed to translate the theoretical conditions in Theorem 9 and its noisy counterparts into practical input design algorithms. For structured prior knowledge sets, efficient optimization methods must be developed.

Experimental validation on benchmark systems in chemical process control, robotics, and biomedical engineering would establish whether the theoretical gains translate into real-world benefits.

Extensions to nonlinear systems represent the most ambitious direction. The concepts of data informativity and universal inputs can potentially be generalized beyond linear dynamics, but the mathematical structure is substantially more complex.

The interplay with learning is another frontier. As machine learning increasingly intersects with control theory, understanding what kinds of inputs yield efficient learning of dynamical systems becomes crucial. This paper provides a rigorous foundation for that understanding.

For now, the key takeaway is this: when you set out to identify a system, what you already know matters enormously. The clever use of prior knowledge can transform an intractable experiment into a practical one, turning a theoretically sufficient condition into a practically optimal one. The next time you're handed a black box, the smart move might be to think carefully about what you already know—before you start poking.

References to Key Results

The foundational conditions for data informativity in the noise-free case are established in Proposition 5 (Theorem 3 of the original paper), which shows that for open prior knowledge sets, informativity reduces to a simple rank condition on the data matrices.

The main sufficient condition for universal inputs under general prior knowledge is Theorem 9, which provides the generalized rank condition that enables sample-efficient experiment design.

The noisy identification results are developed in Theorems 21, 22, and 24, with Theorem 30 establishing the surprising possibility of exact identification under noise when prior knowledge is uniformly discrete.

The comparison between single-input and multi-input systems is captured in Theorem 7 (necessity of persistency of excitation for single-input systems) and Example 8 (sufficiency of non-persistently-exciting inputs for multi-input systems).

About the Authors

This work emerged from the Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence at the University of Groningen. Henk van Waarde's research is supported by the Dutch Research Council (NWO) through the Veni program. The collaboration between van Waarde and Camlibel, who leads the Networks and Control group at Groningen, exemplifies the productive intersection of control theory and mathematical optimization that characterizes modern systems theory.