The AI That Couldn't Learn: Why Frontier Models Know Things Small Ones Can't

The Invisible Energy Hog in Every Building

Walk into any office building on a summer afternoon and feel the air conditioning. That cool, dry breeze didn't design itself. Somewhere in the building's control system—a gray cabinet somewhere in a mechanical room—a set of rules written years ago is telling dampers how much air to send where, adjusting fan speeds, and cracking open valves for chilled water. Most of the time, it works. Most of the time is not good enough.

Buildings consume roughly 40% of global energy, and heating, ventilation, and air conditioning (HVAC) accounts for about half of that. In the United States alone, commercial buildings waste an estimated $30 billion annually in energy costs due to inefficient HVAC operation. The problem isn't the equipment—it's the brain directing it. Most buildings still run on control logic tuned once, years ago, and left to cope with everything from January blizzards to July heat waves, from empty floors on holidays to packed conference rooms on weekdays. The systems don't know what they don't know.

A team of researchers at the University of Tokyo and Tokyo University of Science wanted to know whether artificial intelligence could do better. Not just in theory—in a real physics simulation of an actual four-zone building, running three summer days with realistic weather, occupancy, and all the messy physical couplings between a chilled-water plant, a supply fan, ductwork, and four separate rooms. They tested whether a frontier AI model could control a building without any building-specific training, and whether that knowledge could be transferred to a smaller model that could run locally, without internet connections or subscription fees.

The results, published on arXiv in July 2026, tell a story that should reshape how we think about AI for physical systems. They found something unexpected: the capability exists. A frontier reasoning model—essentially the kind of AI that can reason through complex problems—achieved a 6.2% reduction in HVAC electricity use while maintaining thermal comfort, with no building-specific training at all. It just read a description of the system and started controlling it. But that capability, it turns out, cannot simply be distilled into a smaller model. After 200 steps of reinforcement fine-tuning with the best techniques available, the open-weight model actually used more electricity than a simple rule-based baseline—and its understanding of how the building physics works remained worse than naive guessing.

This is not a story about AI failing. It's a story about a fundamental gap between two kinds of knowledge: knowing what good control looks like, and understanding why.

The VAV Control Problem

Before diving into what the researchers found, it's worth understanding what they're actually trying to solve—because it's harder than it sounds.

A variable-air-volume (VAV) system is the workhorse of commercial HVAC. One central air handler mixes outdoor air with return air from the building, cools the mixture, and sends it through a network of ducts. At each zone—a room, a section of an open floor, a conference area—a terminal damper opens or closes to regulate how much conditioned air enters that space. The building doesn't cool rooms directly. It cools airstreams. The rooms respond with a delay.

This creates a problem that sounds paradoxical: to reduce energy use, you want to send as little air as possible. But the outdoor air you bring in serves a purpose beyond cooling—it dilutes carbon dioxide exhaled by occupants, removes humidity, and keeps indoor air fresh. Open the dampers too much and you waste energy conditioning air you didn't need. Keep them too tight and the CO₂ levels climb until people start feeling drowsy, headaches set in, and someone opens a window anyway, defeating the entire purpose.

Now add the coupling. When you increase airflow to a warm zone, you cool it—but you also raise the duct pressure, which makes the fan work harder, which changes the flow available to every other zone, which changes the load on the cooling coil. The same sensible cooling load can be met with more air at a moderate supply temperature or less air at a colder temperature. The first option costs more in fan energy; the second costs more in chiller and pump energy. A good VAV controller coordinates all of this in real time, adjusting seven separate actuators—the four zone dampers, an outdoor-air damper, a chilled-water valve, and the supply fan speed—every five minutes, as weather and occupancy shift throughout the day.

Conventional systems use proportional-integral (PI) feedback loops, which react to both the current error and accumulated past errors. They also use higher-level reset rules: when many local loops request more capacity, the system adjusts a shared setpoint such as duct pressure. These controllers are reliable and interpretable, but they're tuned for specific conditions, and the interactions between loops make tuning notoriously labor-intensive. ASHRAE Guideline 36 standardizes high-performance sequences for energy-efficient operation, and it serves as the strong rule-based baseline in this study.

The alternative approaches are model predictive control (MPC) and reinforcement learning (RL). MPC solves an optimization problem over predicted building responses and applies the first command—powerful but requiring a building-specific model, state estimation, an optimizer, and integration with the building automation system. RL learns a policy through interaction with an environment. Both have been studied extensively, but three barriers limit routine deployment: the knowledge they produce is building-specific, neural policies are opaque at their interface, and the experts who could implement them are scarce.

The Reasoning Model Test

This is where the story takes an interesting turn. The researchers didn't start with RL or MPC. They started with a question that sounds almost naive: can a large language model—a system trained on text from across the internet, with no specific knowledge of this building—control a VAV system?

Not just any language model. A "reasoning model"—an LLM post-trained to allocate additional inference-time computation to multi-step problem solving before returning a final answer. The idea is that extended deliberation allows the model to compare interacting constraints, anticipate delayed effects, and revise a candidate command before producing the final action. They used GPT-5 as their capability reference, testing what a frontier model could achieve without any building-specific weight updates.

The interface was deliberately simple. Every five minutes, the system serialized the building management measurements into text: current zone temperatures and CO₂ concentrations, their recent changes, outdoor conditions and trends, the previous action, and actuator bounds. The model received a system prompt explaining the zones, comfort band, CO₂ limit, and main physical trade-offs—including the cubic dependence of fan and pump power on speed (cut the speed in half, and power drops to an eighth), the energy cost of outdoor air, and the heuristic of keeping at least one zone damper fully open to maintain duct pressure. The model then generated a JSON object with a step-by-step thought process and seven numeric action values, one for each actuator. A parser validated and clipped the values before they reached the emulator.

The results were striking. Over three summer days (July 28-30, 2025, in Tokyo, using interpolated weather observations from the Japan Meteorological Agency), GPT-5 reduced HVAC electricity use by 6.2% compared to the Guideline 36 baseline. It maintained temperatures within the comfort band, kept CO₂ below the violation threshold, and did all of this without ever training on this specific building or seeing any historical data from it. It simply read a description of the system and figured out how to control it.

This matters enormously for scalability. Current approaches to smart building control require site-specific modeling or extensive training. Every building is essentially a custom project. But if a general-purpose AI can achieve competitive performance from a text description alone, that changes the economics of deployment entirely. The same model could, in principle, read a description of any VAV system and generate reasonable control actions.

But—and this is the critical caveat—GPT-5 reduced the ventilation margin. It pushed outdoor air intake closer to the minimum required for adequate dilution. This worked in simulation, but real buildings have uncertainties: occupancy might spike, someone might light a fire in the breakroom, the CO₂ sensors might drift. Operating near the edge leaves less safety buffer. The model optimized for the simulated conditions, but the margin of safety matters in practice.

The Transfer Problem

Having established that a frontier model can control VAV systems, the researchers turned to the harder question: can that capability be transferred to a smaller, locally deployable model?

The motivation is practical. GPT-5 is a hosted API. It requires an internet connection, sends building data to external servers, and costs money per query. For many building operators, this is a dealbreaker. They want control that stays on-premises, audit logs that never leave the building, and a system that works even if the internet goes down. An "open-weight" model—where the parameters are available for local inference—addresses these concerns.

The candidate was gpt-oss-20b, a 20-billion-parameter reasoning model. The goal was to train it, using reinforcement fine-tuning (RFT), to behave more like the TD3 policy that had been trained specifically in the emulator. If successful, the result would be a locally deployable controller that captured the building-specific knowledge of the TD3 policy while maintaining the text interface and reasoning capabilities of the LLM.

The transfer approach was rollout-verified RFT. The idea works like this: first, a "teacher" policy (TD3 in this case) is trained in the emulator. States are saved from the teacher's trajectories. For each saved state, the student model proposes several candidate actions. Each candidate is applied for one five-minute interval, after which the teacher controls the emulator for the rest of the day. The total return from that rollout—essentially, how well the day went—is used as the reward signal. The student model is then updated to make higher-reward actions more likely in the future. This is not imitation learning; the student isn't learning to copy the teacher's actions. It's learning which outcomes its own actions lead to.

The researchers used a technique called Dr. GRPO (Group Relative Policy Optimization) for the actual updates. For each state, the model generates multiple candidate responses. The returns are converted to group-relative rewards: each action's reward is compared to the mean of all actions for that state. A policy gradient algorithm then updates the model to favor higher-reward action patterns.

There was a methodological twist. Before running the full RFT, the researchers audited a key component: the learned critic. In RL, a critic estimates the value of state-action pairs—the cumulative future reward you'd expect from taking a given action in a given state. A good critic could potentially serve as a lower-cost verifier, scoring actions without needing the expensive rollout procedure. The researchers tested whether the TD3 critic, trained during its normal training process, could rank the student model's actions accurately.

What they found was unsettling.

The Critic That Looked Great But Wasn't

The audit compared the critic's estimates against direct rollout returns at 10 saved TD3 states. For each state, the base gpt-oss-20b model generated 16 different action candidates. Each candidate was applied, the system rolled forward with TD3 in control for the rest of the day, and the total return was computed. The critic's estimates were then compared to these ground-truth returns.

Across time—meaning, if you looked at the correlation between critic estimates and actual returns as the simulation progressed through a day—the critic looked nearly perfect. A Pearson correlation of r=0.99 means the critic tracked the trajectory-level returns almost flawlessly. For practical purposes, it looked like the critic understood exactly how good each action was.

But RFT doesn't need across-time correlation. It needs within-state ranking. When you're at a specific moment, with specific measurements on the table, you need to know which of several candidate actions is best right now. And here, the critic failed.

Across the 10 states tested, the critic's within-state ranking was unreliable. The macro-averaged Spearman correlation—a measure of whether high estimates corresponded to high returns—was only 0.71. Pairwise accuracy was 79.8%—if you picked two actions at random, the critic picked the better one about 80% of the time, which sounds decent until you consider that random guessing would get you 50%. But the most damning metric: when the critic was asked to select the single best action for each state, it chose the rollout-best action in only 5 of 10 states. Half the time, it picked a suboptimal action.

The researchers had found a failure hidden by near-perfect correlation. The critic worked well enough to track how the day was going, but it couldn't distinguish between actions within a single moment. And for group-relative RFT, that's exactly what matters: you're ranking actions relative to each other, not tracking trends over time.

The diagnostic revealed why. Many of the actions generated by the LLM fell outside the distribution of actions the critic had seen during its training. The critic was trained on TD3's behavior, which explored the action space in a particular way. When asked to evaluate LLM-generated actions—actions that TD3 might never have considered—the critic was extrapolating into territory it hadn't learned. Actions that looked numerically similar to the critic could have very different physical consequences, and the critic couldn't tell them apart.

This finding has implications beyond this specific study. Anyone building RL systems that use learned critics as verifiers should be aware: correlation across time does not guarantee ranking within state. The failure mode exists and can be hidden by aggregate metrics.

Given the critic's failure, the researchers fell back to direct rollouts as the verifier—the ground truth, computed by actually applying each action and following TD3 to the end of the day. They then proceeded with the full RFT experiment.

200 Steps of No Improvement

The reinforcement fine-tuning ran for 200 steps. Each step, the model generated multiple action candidates for several state prompts, rollouts scored each candidate, and the model was updated to prefer higher-reward actions. The LoRA adaptation parameters were updated while keeping the base model fixed, so the trained adapter could be loaded or removed without changing the foundation.

The result: no sustained improvement. After 200 RFT steps, the fine-tuned open-weight model used more electricity than the baseline—not less. Its five-minute transition predictions, tested separately, remained worse than persistence (the naive baseline of predicting nothing changes), both before and after training. The model had learned nothing useful about how to control the building.

This is the central paradox of the paper. The rollout verifier provided exact scores for every sampled action. It wasn't noisy or approximate—it was the ground truth, computed by simulating each action to completion. And yet, this precise feedback produced no learning.

The researchers analyzed why. The rollout scores rank sampled actions—they tell you which action was best among the ones you tried. But they don't tell you which direction to move. They don't reveal next-state effects or an improvement direction. If you sample seven actions and one has the highest rollout return, you know to prefer that one—but you don't know why it was better, or how to generate an action that's better still.

The missing piece was transition knowledge: understanding how each action changes the next state. When you adjust a zone damper, the zone temperature and CO₂ respond over the next five minutes. If the model doesn't understand these local effects, it can't construct better actions—it can only shuffle among the same candidates it already samples, none of which are particularly good.

To test this, the researchers ran a separate experiment: a five-minute transition test. Given the current state and a proposed action, can the model predict what the next state will be? GPT-5 predicted transitions far better than gpt-oss-20b. For three of the four tested zone pairs, GPT-5 recovered the local transition effects accurately. The smaller model could not, either before or after RFT.

The implication is stark: exact rollout scores are necessary but not sufficient for value-based RFT when the student model lacks the local physical understanding needed to generalize. You can tell the model which actions were best in the past, but if it doesn't understand why, it can't improve beyond what it already samples.

The researchers' diagnosis is direct: transition-focused supervised fine-tuning before value-based RFT. First, train the model to predict how the building responds to actions. Once it understands the local physics—the transition dynamics—then use rollout-verified RFT to optimize over those predicted consequences.

Why This Changes Things

This study matters for three distinct audiences, and the implications ripple outward from the laboratory into how we think about AI for physical systems.

For the building controls industry, the findings validate a cautious path forward. GPT-5's 6.2% reduction shows that frontier reasoning models can achieve competitive HVAC control from a text description alone. This is genuinely new. It suggests that the knowledge embedded in large language models—patterns learned from engineering documents, control manuals, physics textbooks, and countless technical discussions—contains useful priors for physical system control. A model that can reason about the tradeoffs between fan power and cooling load, or understand the relationship between occupancy and ventilation requirements, brings something that neither PI loops nor building-specific RL can offer: generalization from experience across thousands of buildings, without needing data from the specific one.

But the transfer failure with the open-weight model should temper enthusiasm for immediate deployment. Running GPT-5 as a building controller requires a hosted API, introduces latency, sends operational data off-site, and creates dependency on an external service. The dream of a locally deployable, open-weight model that captures this capability is compelling—but it remains out of reach with current techniques. The gap between frontier model performance and what can be distilled into smaller models is not a calibration problem that more training steps will solve. It's a fundamental knowledge gap: the smaller model lacks the physical understanding needed to use the reward signal.

For AI researchers, the study exposes a subtle but critical distinction. Group-relative RFT works magnificently in domains like mathematics and code generation, where the reward is exact and the space of candidate outputs is discrete and enumerable. "Correct" and "incorrect" are unambiguous. The ranking tells you which answer is better, and the structural similarity between correct answers allows generalization. But continuous physical control operates differently. The reward is the consequence of physical dynamics, delayed by minutes or hours, distributed across comfort and energy metrics, and dependent on how the world responds to actions. The exact rollout score ranks what you tried—but it doesn't tell you what you didn't try, or why, or how to get there.

The critic audit is a methodological contribution that should influence how RL research is conducted. Near-perfect across-time correlation hiding within-state ranking failure is a pattern that could appear in many learned critic setups. Anyone using a critic to rank actions should verify both metrics—and should be especially suspicious when the critic is evaluated on the training distribution but asked to score actions from a different distribution (in this case, LLM-generated actions). The 5-out-of-10 accuracy is a small sample, but it reveals a systematic problem that aggregate metrics would hide.

For climate and energy policy, the findings suggest that AI for building control is not a solved problem, despite promising results from individual studies. The global potential is enormous: even a 10% improvement in commercial building HVAC energy use would reduce worldwide commercial building emissions by roughly 4%—a meaningful contribution to climate targets. But the path from "GPT-5 achieves 6.2% reduction in simulation" to "millions of buildings running on AI controllers" is longer and more uncertain than the headline number suggests.

What's Next

The researchers point toward a clear next step: supervised fine-tuning on transitions before reinforcement fine-tuning on values. The idea is to first teach the model how the building responds to actions—the local physics of zone temperature and CO₂ dynamics, the relationship between damper positions and airflow, the thermal mass effects that cause delayed responses. Once the model can predict consequences accurately, the rollout-verified RFT can operate over a richer space of anticipated outcomes, allowing the model to construct and evaluate actions it hasn't explicitly sampled.

This sequential approach mirrors how humans learn physical skills. You don't learn to drive by being told which actions are best; you learn how the car responds to steering and acceleration, and then you learn which responses you want. The transition model provides the physical intuition; the value function provides the goals.

Several open questions remain. The transition test was limited to four zone pairs—generalizing the findings requires broader testing. The study used a physics-based emulator, which captures the deterministic dynamics but omits real-world complications like sensor noise, actuator hysteresis, and model-plant mismatch. Real building deployment would need robustness to these factors. The authors note that the same recipe—rollout-verified RFT with a transition-focused pre-training phase—could apply to other HVAC control problems like central plant sequencing, TES scheduling, or district-level coordination, but these extensions remain untested.

Perhaps most importantly, the study leaves open the question of why GPT-5 predicted transitions better. Is it the larger parameter count, the different training data, the reasoning capability that allows it to simulate consequences before committing to an action? Understanding the mechanism would help identify which model families are most promising for physical control applications, and what training objectives would best develop those capabilities.

The paper closes with a direct statement: "The unchanged transition errors motivate transition-focused supervised fine-tuning before value-based RFT." This is not a conclusion of failure. It's a diagnosis—a precise identification of the bottleneck that, once addressed, could unlock the transfer pathway the researchers were testing.

The Broader Significance

Buildings are where most people spend most of their lives. The temperature, air quality, and energy consumption of the built environment affects health, productivity, and climate. A world where AI can optimize these systems—not just in theory, not just in expensive custom deployments, but in every building, reading standard descriptions, adapting to local conditions—is a world with meaningfully lower emissions, lower costs, and more comfortable occupants.

We are not there yet. This study is a snapshot of where the technology stands: capable enough to be exciting, incomplete enough to be honest about the distance. GPT-5 can control a building from text—but it lives in a data center, not a mechanical room. The smaller models that could live locally can't yet learn what the frontier model knows intuitively.

The researchers have mapped the terrain. The path forward is becoming visible: first understand the physics, then optimize the goals. The 6.2% reduction achieved by a frontier model without building-specific training is not a ceiling. It's a benchmark—proof that the destination exists, even if the route is still being plotted.