The Hidden Instability of AI-Driven Grid Planning

In a simulated electricity market, a single generator’s temporary lapse in strategic bidding—lasting just 2,000 episodes—caused transmission planners to underestimate optimal grid capacity by 38%, locking in billions in avoidable congestion costs. This isn’t a failure of engineering. It’s a failure of learning.

Electricity markets are no longer centrally controlled systems. They’re complex ecosystems where profit-driven generators bid strategically, often withholding supply to inflate prices. Meanwhile, transmission planners must decide where to build new power lines—multi-billion-dollar investments that shape grid reliability and efficiency for decades. The catch? These decisions are interdependent: new transmission reduces a generator’s ability to manipulate prices, but generators adapt their strategies in response to infrastructure changes. Traditional planning treats these dynamics as static. But in reality, they evolve.

A new study by Kanazawa, Hoshino, and Furutani (2026) reveals how machine learning, when applied naively to this co-evolutionary problem, can destabilize the entire planning process. Using a co-optimization framework where both generator bidding strategies and transmission capacities are learned simultaneously via deep reinforcement learning (DRL), the researchers uncovered a hidden fragility: transient policy degradation in learning agents can cascade into catastrophic underinvestment in grid infrastructure.

But they didn’t stop at diagnosis. The team introduced a replay-based stabilization method that filters out misleading historical data, restoring robustness. Yet even this fix exposes a deeper truth: stability in complex socio-technical systems requires trade-offs. Larger memory buffers improve resilience but introduce oscillations—unless learning rates are carefully tuned. This isn’t just about algorithms. It’s about how we design learning systems for real-world infrastructure, where errors aren’t just statistical noise—they’re economic and environmental costs borne by society.

The Science

The study models a deregulated electricity market using a multi-agent reinforcement learning (MADRL) framework. Three generators—Gen1, Gen2, and Gen3—compete in a day-ahead market modeled on the IEEE 30-bus system, a standard testbed in power systems research

. Gen1, positioned near high-demand zones, holds significant market power. Its strategic bidding—raising prices above marginal cost—creates congestion on key transmission lines, particularly the 4–12 corridor, which has an initial capacity of 10 MW and an expansion cost of $100,000/MW/year.

Each generator is an autonomous agent using the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. At each time step, agents observe system demand, their previous bid, and current transmission capacities. They then choose a bid price—up to twice their marginal cost ($50/MWh)—aiming to maximize profit:

$$r_i(t)=({\lambda }_i^{\mathrm{cleared}}(t)-{\lambda }_i^{\mathrm{cost}})P_i^{\mathrm{cleared}}(t)$$

The market clears via DC optimal power flow (DC-OPF), dispatching generation while respecting transmission limits. Over time, agents learn bidding strategies that exploit congestion for profit.

Simultaneously, a transmission planner learns optimal capacity upgrades. The planner’s objective is to minimize total system cost: the sum of operational costs (driven by bidding behavior) and investment costs. The planner samples transmission capacities from a stochastic policy—modeled as a Gaussian distribution—and updates its parameters using policy gradients based on observed market outcomes:

$${\nabla }_{\mu }\mathbb{E}[G_{\mathrm{total}}]=\mathbb{E}\left[{\nabla }_{\mu }\ln p_{\mu }(\omega )\left(G_{\mathrm{total}}-{\bar{G}}_{\mathrm{total}}\right)\right]$$

This creates a coupled learning loop: generator policies affect congestion and prices, which shape the economic case for transmission, which in turn alters the strategic incentives for generators.

The innovation lies in how the planner learns. Instead of using only recent episodes (as in prior work), the authors introduce a replay-based update with nearest-neighbor filtering. Historical episodes—each containing a transmission capacity setting and resulting system cost—are stored in a replay memory. Before updating, the algorithm performs a spatial filter: for each sample, it finds its $k$ nearest neighbors in capacity space and retains only the one with the highest cost. This removes "outlier" samples where low costs were achieved not through efficient infrastructure, but through temporarily suboptimal bidding (e.g., a generator accidentally bidding low).

What They Found

The results expose two distinct failure modes—and a delicate balance between them.

First, without replay filtering, the co-optimization process is highly unstable. In one simulation, Gen1’s bidding policy—after stabilizing near $90/MWh—suddenly degraded around episode 5,000, dropping to $60/MWh for several thousand episodes. This temporary lapse created artificially low system costs, even with minimal transmission capacity. The planner, seeing these low-cost outcomes, inferred that new transmission was unnecessary and began reducing the target capacity for line 4–12. By episode 10,000, the planned capacity had fallen to just 12 MW—far below the economically optimal 20 MW. This is not a minor deviation. In real-world terms, such underinvestment could lock in tens of millions in annual congestion costs across a regional grid.

Transmission Capacity at End of Training

Label	Value
Conventional update	12
Replay-based update	19.5

Source: Kanazawa et al. (2026), Fig. 3 and Fig. 5

The root cause? Inconsistent data. When plotted, the relationship between transmission capacity and system cost should be monotonically decreasing: more capacity, lower costs. But in failure cases, the scatter of data points dips far below this trend, forming a cloud of misleadingly efficient outcomes

. These are the fingerprints of transient policy degradation—moments when learning goes awry.

The replay-based method with $k=5$ nearest-neighbor filtering eliminates this vulnerability. Even when Gen1’s bid dropped temporarily between episodes 11,000 and 13,000, the planner maintained a capacity trajectory close to optimal

. The filtered data closely followed the expected cost-capacity curve, confirming that the algorithm successfully discarded misleading samples

.

But stability came at a price. The filtered trajectories revealed a new phenomenon: persistent oscillations. Instead of converging smoothly, the transmission capacity repeatedly overshot and undershot the equilibrium, cycling between 18 MW and 22 MW even after bidding strategies stabilized.

The cause? Temporal lag. A large replay memory (e.g., 7,000 episodes) retains outdated information. When the planner evaluates whether to expand, it considers not just recent outcomes, but historical ones from when the grid was weaker. This creates inertia: the planner keeps expanding because old data suggests it’s still beneficial, even after the optimal point has been passed. Only when enough high-cost data accumulates does it reverse course—leading to overshoot and correction.

Oscillation Amplitude vs. Replay Memory Size

Label	Value
W=3000	3.2
W=5000	4.1
W=7000	5.8

Source: Kanazawa et al. (2026), Fig. 6

The trade-off is stark. Larger replay memories improve robustness against transient failures but amplify oscillations. The solution? Reduce the learning rate. At a high learning rate ($\eta =0.01$), oscillations grow with memory size. But at $\eta =0.001$, even a memory of 7,000 episodes produces stable convergence. The cost? Slower learning. The system takes longer to reach the optimal capacity, increasing the time during which suboptimal infrastructure remains in place.

Oscillation Amplitude vs. Learning Rate

Label	Value
η=0.01	5.8
η=0.005	2.9
η=0.001	0.7

Source: Kanazawa et al. (2026), Fig. 6

Why This Changes Things

This study is not about a better algorithm. It’s about a better understanding of how learning systems behave in the real world—where agents are imperfect, data is noisy, and decisions have long-term consequences.

Today, many countries are grappling with how to modernize aging grids to accommodate renewables, prevent blackouts, and reduce emissions. In the U.S., the Department of Energy estimates that transmission investment must double by 2030 to meet clean energy goals. But planning is stuck in a 20th-century paradigm: static models, siloed analysis, and long approval cycles. When regulators approve a new line, they assume generators will bid competitively. But in reality, they don’t.

Kanazawa et al. show that ignoring strategic behavior leads to flawed planning. But so does naively applying machine learning. The promise of AI in infrastructure is not just automation—it’s adaptability. A learning-based planner could, in theory, anticipate how market participants will respond to new lines and adjust accordingly. But this paper reveals the hidden instability of such systems.

The implications extend beyond electricity. Any co-optimization problem—urban planning with adaptive traffic, carbon markets with strategic firms, vaccine distribution with hesitancy feedback—faces similar risks. When learning agents interact with design variables, transient errors can corrupt the entire process. The replay-filtering method offers a general principle: not all data is equally trustworthy. In socio-technical systems, we must design learning rules that are robust to temporary irrationality.

Moreover, the oscillation phenomenon highlights a universal challenge in adaptive systems: the tension between memory and responsiveness. In climate policy, for example, carbon price adjustments based on long historical averages may lag behind current emissions trends. In financial regulation, stress tests using outdated scenarios may miss emerging risks. The trade-off between memory size and learning rate is a fundamental design choice—one that must be made explicit, not accidental.

What’s Next

The study opens several critical paths forward.

First, scalability. The IEEE 30-bus system is a toy compared to real grids like ERCOT or the European interconnected network. Can these methods scale? The authors note that computational overhead is low—less than 0.4% of total training time even with 7,000 episodes in memory. But real systems involve thousands of generators, complex constraints, and discrete investment decisions (you can’t build half a power line). Future work must extend the framework to handle combinatorial expansion choices and mixed-integer planning.

Second, realism. The current model assumes continuous capacity upgrades and perfect information. In reality, transmission projects take years, face political hurdles, and require discrete approvals. Integrating these frictions into the learning loop could reveal new dynamics—such as strategic delay by generators anticipating future capacity.

Third, fairness and market power. The paper focuses on efficiency, but what about equity? Gen1’s ability to manipulate prices reflects real-world concentration in generation markets. A planner that merely optimizes total cost may inadvertently entrench market power by underinvesting in lines that would increase competition. Future models could incorporate social welfare or competitive fairness as explicit objectives.

Finally, human-in-the-loop design. These algorithms won’t run in isolation. Regulators, utilities, and policymakers will interpret and act on their outputs. How do we design interfaces that reveal not just the recommended capacity, but the uncertainty, the trade-offs, and the assumptions? A planner that oscillates isn’t broken—it’s signaling the limits of its information. We need systems that communicate their own epistemic boundaries.

The most profound insight from this work is that stability in complex systems isn’t just a technical problem. It’s a design philosophy. We can build learning systems that are fragile but fast, or robust but slow. The choice depends on what we value—and who bears the cost of failure. In the case of the power grid, the answer is clear: everyone.