The Math of Machine Boredom: How Sensor Assumptions Kill AI Curiosity—and the Simple Fix That Brings It Back

The Problem of a Bored Agent

Imagine you've built a nearly perfect artificial intelligence. It tracks moving objects, predicts stock prices, navigates autonomous vehicles—whatever task you've given it. You've equipped it with sophisticated sensors and a principled framework for decision-making. And yet, despite all this sophistication, there's something fundamentally wrong: your agent has lost its curiosity.

Not metaphorically. Literally. The AI no longer has any reason to gather information for its own sake. It will complete whatever task you've assigned, but it won't probe, explore, or seek to reduce its own uncertainty. It won't act differently to learn something new. In the language of the field, its "epistemic drive"—from the Greek episteme, meaning knowledge—has vanished.

This isn't a hypothetical edge case or a failure of implementation. It's a mathematical result. In a particular widely-used class of models called linear-Gaussian state-space models, which underpin vast swaths of robotics, control theory, and statistical signal processing, an agent built on the popular "active inference" framework will always lose its epistemic drive. Always. Under any circumstances.

The question Daniel Corva asks in his new paper is deceptively simple: what's the minimum you need to change to bring curiosity back?

His answer is equally surprising. You don't need to overhaul the agent's learning algorithm or abandon the elegant mathematics of Gaussian inference. You just need to acknowledge a fact that should have been obvious all along: that your sensor's accuracy might depend on where the world actually is.

This isn't a minor technical tweak. It's a fundamental reconceptualization of how sensing works—of what observation noise really means. And it bridges two intellectual traditions that have spent decades talking past each other: active inference, born from neuroscience and biology, and dual control theory, developed by engineers trying to build systems that both track and learn.

The Science

What Active Inference Actually Is

To understand why epistemic drive vanishes in linear-Gaussian models—and how to restore it—we need to understand what active inference is in the first place.

Active inference is a unified theory of how agents should act and learn, developed over the past two decades by Karl Friston and colleagues at University College London. Its roots are in the free energy principle, which suggests that all adaptive systems—from single-celled organisms to human brains—are fundamentally trying to minimize surprise about their sensory inputs.

The mathematics gets intricate, but the core intuition is elegant. An agent maintains beliefs about the hidden state of the world. When it senses something, it updates those beliefs. And when it acts, it does so to make its sensations conform to what it expects—to reduce what active inference theorists call "surprise" or, more formally, to minimize the variational free energy of its beliefs.

This sounds straightforward, but it contains a crucial insight: actions have two kinds of value. There are pragmatic values—the rewards or outcomes you get from reaching a goal. And there are epistemic values—the information you gain from resolving uncertainty. A rational agent shouldn't only chase outcomes; it should also act to learn about the world, because that learning will enable better outcomes in the future.

This is where curiosity, exploration, and the drive to understand come from mathematically. In active inference, they're captured in the Expected Free Energy (EFE)—a quantity that, for any potential action, combines both what you expect to gain and what you expect to learn.

The Linear-Gaussian Setting

Now consider the simplest interesting case: linear-Gaussian state-space models. These are the workhorses of modern estimation and control.

A state-space model describes a world where there's a hidden "state"—the true situation—that evolves over time according to some linear dynamics, plus randomness. The agent doesn't observe this state directly. Instead, it gets noisy observations that are linear functions of the state, corrupted by Gaussian noise.

The classic example is a moving object tracked by a sensor. The object's position and velocity form the state. It moves according to simple physics (position changes by velocity, velocity might drift). The sensor measures position (and maybe velocity) with some Gaussian noise. The noise's covariance—the spread of possible measurement errors—is often assumed constant. A radar has a certain precision regardless of where the target is.

The optimal way to track such a system is the Kalman filter, invented by Rudolf Kálmán in 1960. The Kalman filter maintains a Gaussian belief over the hidden state. When a new observation comes in, it updates this belief by combining the prediction from its model with the information from the observation, weighted by something called the Kalman gain.

The Kalman filter is beautiful mathematics. It gives you the theoretically optimal estimate of the hidden state, given the model and the observations. For sixty years, it has been the foundation of navigation systems, financial forecasting, weather prediction, brain-computer interfaces, and much more.

The Surprise: Epistemic Collapse

Here's where things go wrong for active inference.

Recent work—Corva cites multiple papers that have sharpened this result—established a troubling fact: when you embed a Kalman filter into an active inference agent, something strange happens. The epistemic term in the Expected Free Energy becomes constant. It doesn't matter which action the agent chooses; the "information gain" term is always the same number.

Mathematically, this means the agent loses any reason to act differently purely to learn more about the world. Its epistemic drive has collapsed to zero.

Why does this happen? The key lies in how the Kalman filter's gain is computed. In the standard Kalman filter, the gain—which determines how much to weight new observations versus predictions—depends only on the covariance matrix the filter maintains about its uncertainty. And that covariance, in the standard linear-Gaussian setup, evolves independently of the agent's actions. The dynamics of the covariance form what's called a Riccati equation, and its solution is pre-determined. It doesn't depend on which actions you take; only on the model parameters and the passage of time.

This is elegant from one perspective—the mathematics stays clean. But it's devastating from the perspective of curiosity. The agent's belief about its own uncertainty is fixed in advance. No matter what the agent does, it will always have the same beliefs about what it knows and doesn't know. It can't act to reduce its uncertainty more in some states than others.

The agent flattens to a Kalman filter whose gain sequence is fixed in advance, regardless of action. The researchers' verdict was stark: epistemic drive disappears "under any circumstances" within this framework.

The Established Fix: Multiplicative Control

What can you do to restore epistemic drive?

The only established route in the literature involves what control theorists call "control entering the dynamics multiplicatively." This means the agent's actions directly affect how the state evolves—not just by pushing things around, but by changing the system matrix that describes how states transition.

Imagine you're controlling a robot arm. In the standard linear-quadratic regulator setup, your control action adds to the dynamics (you apply a force, the position changes). In multiplicative control, your action might instead change the spring constant or the damping—altering how the underlying physics responds to disturbances.

This approach works. It restores the dual effect: actions now influence both the state and the quality of future estimates. But it's a substantial departure. It requires control to enter in a specific nonlinear way. And it leaves unexplored a critical question: is this the minimal change? What if you don't want to change the dynamics? What if the world you're modeling really is linear, but your sensors aren't perfect in the way the standard model assumes?

What They Found

Corva's paper takes a different path. Rather than modifying how actions affect the dynamics, he modifies how the observations relate to the state. He asks: what happens if the observation noise isn't constant?

State-Dependent Observation Noise

The key innovation is allowing the observation noise covariance—traditionally denoted as a constant matrix R—to depend on the hidden state x. You write this as R(x). A sensor might be more accurate when the target is close, less accurate when it's far. A camera might have different noise characteristics in bright light versus shadow. An economic indicator might be more volatile at certain scales or regimes.

This is not exotic. State-dependent noise is ubiquitous in real sensing systems. A radar's range resolution degrades with distance. A GPS sensor has position estimates that depend on which satellites are visible, which depends on where you are. Medical imaging often has noise characteristics that vary with tissue density or depth.

And yet, in the theoretical literature, this possibility is typically assumed away. The constant covariance assumption is convenient mathematically. Corva argues it's also epistemically catastrophic—it robs the agent of curiosity.

The Minimal Fix

The setup is elegant in its simplicity. The agent runs the standard first-order Gaussian filter of the linear-Gaussian literature—essentially a Kalman filter—but with one critical difference: the observation noise covariance R is evaluated at the predicted mean state, rather than being held constant.

When R depends on x, but the agent evaluates it at its belief about x, something interesting happens. The agent's actions don't change R directly—but they change the predicted mean, and the predicted mean feeds into R. Actions still affect the state, which changes the agent's beliefs about where it thinks the state is, which changes R, which changes the filter's gain.

In other words, actions now influence the quality of future estimates—not just the state estimate itself. This is the Bar-Shalom-Tse dual effect, a concept from dual control theory that describes systems where exploration and exploitation are genuinely coupled. The agent must act not just to reach goals, but to create information.

The Key Mathematical Results

The paper establishes several precise results. The central theorem has two parts.

First, the agent with state-dependent observation noise cannot be reproduced by any fixed linear-Gaussian filter. This matters because it shows the agent isn't equivalent to some clever parameter choice in the standard setup—it's genuinely different. You can't replicate its behavior with a Kalman filter whose gain sequence is pre-computed, because the gain now depends on the action sequence in a way that can't be predicted in advance.

Second, under two technical conditions, epistemic value is no longer constant. The two conditions are:

1. A mild rank condition on the observation map—the sensor needs to actually observe something informative about the state.

2. A non-degeneracy condition on R(x)—the state-dependent covariance can't be pathological. In particular, R(x) can't be constant in regions the agent might actually reach.

For scalar observations (the one-dimensional case, where you're estimating a single number rather than a vector), a weaker result suffices: reachable non-constancy alone is enough. If R varies in regions of state space that the agent can actually get to, epistemic drive reappears.

The mathematics here is precise and rigorous—the paper runs to 44 pages and includes an executable "witness" that demonstrates the incompatibility. If someone claimed to have found a fixed linear-Gaussian filter that replicated the agent's beliefs, the witness code would disprove it automatically.

Corva also releases a software library called cpomdp (for "continuous partially observable Markov decision process") that can detect this incompatibility from the model specification alone and raises a typed IncompatibleLinearizationError. This is notable because it shows the theoretical result has practical implications: if you're building an agent and you want to preserve epistemic drive, your tooling can now warn you when your model assumptions are going to kill curiosity.

Why This Changes Things

It Reframes How We Think About Observation Noise

The deepest conceptual shift in Corva's paper is what it implies about observation noise. In the standard linear-Gaussian framework, observation noise is treated as a fixed property of the sensor—a constant covariance matrix that doesn't vary with where things are or what you're looking at.

This is a convenient fiction. Real sensors don't work this way. A camera has different noise in low light. A range finder has worse resolution at distance. An economic time series has different volatility at different scales.

The standard model absorbs this variation into the assumption of constant covariance, but in doing so, it throws away information that matters. More importantly, it throws away the driver of curiosity. The agent has no reason to go closer to things, or to seek out situations where its sensors perform better, because according to the model, sensor performance doesn't vary with state.

State-dependent observation noise is a minimal, realistic departure that restores the correct incentive structure. The agent now has reasons to seek states where its sensors are better, to get closer to things, to position itself to observe more accurately. It regains the drive to act not just for outcomes, but for information.

It Bridges Two Traditions

Active inference and dual control theory have developed largely in parallel. Active inference comes from neuroscience and biology, emphasizing the free energy principle and the unified treatment of perception and action. Dual control theory comes from engineering, emphasizing the coupling between estimation and control in systems where actions affect what you learn.

Corva's result creates a precise bridge between these traditions. The Bar-Shalom-Tse dual effect—the phenomenon where actions influence the quality of future estimates—is a central concern in dual control. Corva shows that state-dependent observation noise creates a minimal, constructive instance of this effect in the active inference framework.

This matters because it allows insights to flow both ways. Active inference provides a principled account of what epistemic value means and how it should be calculated. Dual control provides decades of analysis of how actions affect information quality. Corva shows how to combine them in a way that preserves mathematical tractability.

It Explains an Intuition That Was Missing

Researchers working with active inference agents in linear-Gaussian settings had noticed that they seemed to lack epistemic drive, but the explanation was murky. Was this a bug in the implementation? A feature of the mathematics? Something that would resolve with better approximation methods?

Corva's result provides a clean theoretical explanation. It's not a bug or an approximation issue. It's a mathematical consequence of the constant-noise assumption. The fix isn't to add heuristics or ad-hoc information bonuses; it's to recognize that real sensors have state-dependent accuracy and to model that accurately.

This is intellectually satisfying. It means the pathology wasn't a failure of the researchers; it was a limitation of the model assumptions. And it means the fix is natural rather than grafted-on.

It Has Practical Implications for Agent Design

The result isn't purely theoretical. When you build an active inference agent, understanding which model assumptions preserve epistemic drive matters for system design.

Corva's cpomdp library formalizes this: given a model specification, it can detect when the assumptions will cause epistemic collapse and raise an error. This prevents the subtle bug of building a supposedly curious agent that has actually had its curiosity mathematically excised.

For practitioners, this means:

• If you're building an agent that needs to explore and learn, you need to model your sensors realistically. Don't assume constant noise covariance.

• If you're using linear-Gaussian models (and you probably are, because they're tractable and well-understood), you need to be explicit about where your observation noise actually comes from.

• If your sensors really do have state-dependent accuracy, you're not stuck with a bored agent—you just need to model that dependence.

What This Means for the World

The abstract framing of "active inference agents" might seem distant from practical concerns, but the applications are immediate and vast.

Autonomous Systems and Curiosity

Consider an autonomous drone surveying a building. Under the standard linear-Gaussian model, the drone's belief about its position (determined by its Kalman filter) evolves independently of which actions it takes, beyond the immediate effect on position. It has no reason to fly closer to walls for better radar returns, no reason to maneuver to get better GPS coverage, no reason to seek vantage points where its sensors work better.

With state-dependent observation noise, it does. The drone now has incentives that mirror what we'd intuitively expect: move to get better data. This isn't just about accuracy; it's about a fundamentally different kind of agent—one that actively manages its information quality, not just its state estimate.

This matters enormously for tasks like search and rescue, where information gathering is as important as movement. It matters for environmental monitoring, where sensor placement affects data quality. It matters for any task where the agent needs to learn about an uncertain environment while operating in it.

Scientific Discovery and Exploratory Behavior

Active inference was developed in part as a theory of biological intelligence—how brains manage uncertainty. The loss of epistemic drive in linear-Gaussian models was therefore not just a technical issue but a puzzle for understanding biological cognition.

If our brains worked like standard Kalman filters, we wouldn't have curiosity. We wouldn't be motivated to explore, to seek out new experiences, to gather information for its own sake. The fact that we do suggests our nervous systems don't assume constant observation noise. They track context-dependent sensory reliability and use that to guide behavior.

Corva's result offers a precise hypothesis about what's different: biological sensory systems likely have state-dependent noise characteristics, and our brains represent and act on that structure. This opens experimental questions for neuroscience—can we find evidence that the brain tracks sensory reliability in a state-dependent way? Does this explain aspects of exploratory behavior?

Machine Learning and Representation Learning

A persistent challenge in machine learning is getting agents to learn good representations—to discover the underlying structure of their inputs without being told what matters. Much current work on unsupervised learning, self-supervised learning, and representation learning grapples with this.

Epistemic drive—the drive to learn—is exactly what these systems often lack. They optimize a reconstruction loss or a contrastive objective, but they don't actively seek out situations where learning is possible.

State-dependent observation noise provides a principled way to inject this drive. If an agent's sensor model includes dependencies on latent states, and if those latent states can be influenced by action, the agent will naturally seek out situations where its representations can be refined.

This connects to curiosity-driven learning, intrinsic motivation, and exploration in reinforcement learning—areas that have struggled to formalize what "interesting" or "worth exploring" means. Corva's framework provides a precise answer: seek states where observation noise reveals information about the latent structure you're trying to learn.

Economics and Decision-Making Under Uncertainty

The dual control problem was originally formulated in the context of economic planning and adaptive control of processes where learning and control interact. State-dependent observation noise provides a natural model for many economic situations.

Consider a central bank setting interest rates. Its observations of inflation, employment, and growth are noisy, but the noise characteristics depend on the underlying economic state. During recessions, some indicators become more volatile or delayed. The bank's actions affect the economy, which affects what it observes, which affects what it can learn.

Active inference with state-dependent noise provides a principled framework for such problems—one that captures both the strategic dimension (achieving targets) and the epistemic dimension (learning about a complex, changing system). This could inform better models of adaptive economic policy.

What's Next

The Rank Condition and Observability

The paper's central result requires a mild rank condition on the observation map. In intuitive terms, this means the sensor has to actually observe something informative about the state—specifically, the observation noise covariance has to vary in a way that's correlated with what the sensor measures.

This condition rules out certain degenerate cases. If R(x) varies but varies in directions orthogonal to what the sensor observes, the agent can't link its actions to information gain. The variation in noise characteristics is invisible to it.

This opens questions about more general cases. What if R varies in more complex ways? What if the agent's observations are non-Gaussian or multimodal? The rank condition is mild, but it's still a condition. Real-world applications will need to verify it holds, or find ways to relax it.

Approximation and the First-Order Filter

The paper assumes the agent runs a "first-order Gaussian filter"—essentially, a Kalman filter with R evaluated at the predicted mean. This is standard in the linear-Gaussian literature, but it raises questions about approximation.

Real agents won't have exact knowledge of the world model. They won't know R(x) perfectly. They'll need to estimate it, and the estimation will introduce additional uncertainty. How do approximation errors affect epistemic drive? Does a small mismatch between true and modeled state-dependence destroy curiosity?

These questions matter for practical deployment. A theory that requires exact knowledge of R(x) is interesting mathematically but limited practically. Understanding the robustness of epistemic drive to modeling errors is an important direction.

Non-Gaussian and Nonlinear Extensions

The linear-Gaussian setting is mathematically tractable, but the real world is nonlinear and sometimes non-Gaussian. State-dependent observation noise in nonlinear systems is more complex to analyze and harder to filter.

The paper gestures toward these extensions but doesn't pursue them. The central result—that state-dependent noise restores the dual effect—is likely robust, but the specific mathematical characterization will change. Extended Kalman filters, unscented filters, and particle filters all have different properties. How epistemic drive manifests under different filtering approximations is an open question.

The Epistemology of Observation Noise

Perhaps the deepest open question is conceptual. If state-dependent observation noise restores epistemic drive, what does this tell us about the nature of observation noise itself?

Standard treatments treat observation noise as a property of the sensor—something intrinsic to the measurement process. State-dependent noise suggests a different view: noise is partly a function of the relationship between sensor and world. It's not just that the sensor is noisy; it's that the noise characteristics depend on what state the world is in.

This reframing has implications for how we think about inference, observation, and the relationship between agent and environment. It's a small shift in one mathematical assumption, but it ripples outward into philosophy of mind, epistemology, and cognitive science.

Implementation and Scale

Corva releases cpomdp as a practical tool, but there's work ahead to scale these ideas to real-world systems. Modern autonomous systems track dozens or hundreds of state variables with complex sensor suites. How state-dependence in observation noise propagates through these systems, and how it affects computational tractability, are practical questions that will shape adoption.

The Kalman filter scales poorly to high-dimensional state spaces if the covariance updates aren't structured. State-dependent noise might introduce additional structure that helps or hurts. This is a question for numerical analysis and systems engineering.

A New View of Curiosity

At its heart, Corva's paper is about what curiosity is and how to preserve it in artificial agents. The mathematical details can obscure the core insight, so let's return to it.

An agent built on the linear-Gaussian framework, with constant observation noise, has no reason to explore for its own sake. It will exploit what it knows, but never act to learn what it doesn't. It has been stripped, mathematically, of the drive to understand.

State-dependent observation noise restores this drive—not as an add-on or a heuristic, but as a natural consequence of modeling sensors as they actually are. A sensor that works differently in different situations creates a world where seeking certain situations is inherently valuable. The agent learns not just that moving closer gives better range estimates, but that moving closer gives information—and information has value.

This isn't just a technical fix. It's a reminder that the mathematics we assume shapes the behavior our agents can have. When we assume constant observation noise, we assume away curiosity. When we model sensors accurately, curiosity emerges.

For researchers, this opens new questions about what other assumptions strip away epistemic drive, and what other minimal changes restore it. For engineers, it provides practical guidance for building agents that genuinely seek to understand. For scientists studying biological intelligence, it offers a precise hypothesis about why our own curiosity exists.

The Bar-Shalom-Tse dual effect—the coupling of action and information quality—has been known in control theory for decades. Corva's contribution is to show exactly where it lives in the active inference framework, and how to activate it with minimal assumptions.

The result is a precise, observation-side characterization of curiosity in a Gaussian agent. It bridges dual control and active inference. And it suggests that the next time you're building an agent and wondering why it lacks curiosity, you might check your noise assumptions. The sensor might know something the model doesn't.