The Inverted Physics of AI: When Temperature Creates Order from Chaos

The word "entropy" comes from thermodynamics, where it means something like "disorder" or "spread-out-ness." Turn up the temperature in a physical system, and you'll spread its molecules across more states — that's entropy increasing. It's one of the most reliable relationships in science. So when physicists started studying large language models, they assumed the same would hold: raise the "temperature" parameter that controls how randomly an AI generates text, and the output should become more varied, more disordered. A predictable consequence of a predictable rule.

Except that's not what happened.

A team of researchers at George Washington University ran an experiment across twelve thousand text continuations generated by eleven different AI systems, and they found something that defies intuitive physics. As they cranked up the temperature, the AI outputs didn't simply become more random. Instead, the systems underwent a strange phase transition — one that physicists recognize from laser physics and certain quantum systems, but have never expected to find in something as mundane as autocomplete text. The output populations didn't just vary; they inverted. At low temperatures, the AIs got stuck in repetitive loops. Raise the temperature, and the loops loosened into varied, interesting prose. But raise it further, and something unexpected happened: the output became concentrated again, not in the original repetitive patterns, but in a different kind of order — one that looks, to the automated classifiers the researchers used, like "complex" or "noisy" output.

The entropy rose, peaked, and then fell. The temperature parameter, the same one that supposedly controls randomness, was doing something entirely different from what physics would predict.

"This is a counterintuitive result," the researchers write, with the understatement of people who have just watched a fundamental assumption collapse. "Increasing the temperature of an ordinary many-state system increases access to a wider range of states and hence increases its entropy. We find the opposite in ChatGPT-like AIs."

But the most striking finding isn't just that the behavior is counterintuitive. It's that the researchers could measure it, predict it, and — most remarkably — steer it.

The Science

Neil Johnson, Frank Yingjie Huo, and Bella Xinrui Li work in the Department of Physics at the George Washington University. Their paper, published on arXiv in August 2026, approaches large language models as physical systems to be studied rather than engineering artifacts to be optimized. They are not trying to make the AIs perform better on benchmarks. They are trying to understand the underlying dynamics that govern how these systems produce text — and in doing so, they've stumbled onto something that looks less like software and more like a strange new kind of matter.

The core of their method was deceptively simple. They took eleven different large language models — four versions of GPT-2 (ranging from 124 million to 1.5 billion parameters), six versions of Pythia (from 160 million to 12 billion parameters), and Llama-3.1-70B, a large open-source model — and asked each one to complete five different prompts. For each combination of model and prompt, they generated multiple continuations at different temperature settings, ranging from near-zero (what researchers call "greedy" decoding, where the AI almost always picks its most confident choice) up to a temperature of 1.0 and beyond.

Total continuations: twelve thousand. Each one up to three hundred tokens long. A dataset that would take a human reader weeks to process.

So they automated the analysis. They broke each generated text into sentences and classified the patterns of recurrence they found. Was the AI producing the same sentence over and over? That was "frozen." Was it cycling through a small set of sentences? "Periodic." Was it switching unpredictably between repetition and novelty? "Intermittent." Was it producing genuinely varied prose that didn't repeat? "Complex" or "noisy."

This classification scheme — borrowed from nonlinear dynamics, where researchers have spent decades characterizing the behavior of chaotic and complex systems — gave them a quantitative window into what the AIs were doing. They could track, for each model at each temperature, what fraction of outputs fell into each category. They could watch the populations shift as temperature changed.

The key parameter governing the next-token probability distribution — what the researchers call the decoder temperature or T_d — enters the AI's computation exactly as temperature enters a Boltzmann factor in physics. Tokens receive weights proportional to exp(-E_i/T_d), where E_i (the "energy" or negative logit) represents how much the model wants to produce that token. At low temperatures, only the highest-probability tokens get significant weight; the distribution is sharply peaked. At high temperatures, the probabilities flatten out; more tokens become viable candidates.

In a normal physical system, this flattening would monotonically increase the entropy of the long-time output. The researchers expected this. They got something else entirely.

For the more detailed analysis, they focused on Llama-3.1-70B, running an additional set of experiments that required measuring the AI's internal hidden states during generation. This involved generating trajectories — sequences of the AI's internal activations as it produced text — and comparing them across different conditions. They used ten random "seeds" to generate training trajectories and ten disjoint seeds for testing, ensuring their predictions weren't artifacts of fitting to the same data they were evaluating on.

They also performed interventions: instead of just watching the AI, they actively perturbed its hidden states along specific directions, testing whether they could causally change the output behavior. This is what transforms their work from observation into experimental science — the ability to not just predict but to intervene and observe the consequences.

What They Found

The headline result is the population inversion. When the researchers tracked the fraction of outputs falling into what they called the "complex/noisy" sector — the outputs that don't get trapped in obvious repetition patterns — they found it didn't simply increase with temperature. It rose, peaked near T_d = 0.517 (averaged across all eleven models), and then fell.

Entropy Maximum Marks Population Inversion

Label	Value
0.1	0.35
0.2	0.45
0.3	0.55
0.4	0.65
0.517*	0.7
0.6	0.65
0.7	0.5
0.8	0.35

Data derived from cross-AI mean across 11 models (GPT-2, Pythia, Llama). Shaded regions span interquartile range. T_d* = 0.517 marks the entropy maximum and sign change of β_effΔ.

The entropy of the two-sector population — measured using the standard mixing entropy formula S_2 = -q ln q - (1-q) ln(1-q), where q is the fraction in the complex/noisy sector — showed the same pattern. It rose with temperature, reached a maximum at roughly equal occupation of both sectors, and then declined as the complex/noisy sector became overpopulated at still higher temperatures.

This is the statistical-mechanical signature of population inversion: the dimensionless parameter β_eff Δ, which the researchers derive from the relative occupancies of the two sectors, changes sign at the crossing. At low temperatures, the recurrent sector dominates. At high temperatures, the complex/noisy sector dominates. But the path between them is not smooth — it's structured by distinct dynamical regimes that the classifiers can distinguish.

Population Inversion: Complex Sector Dominance Rises with Temperature

Label	Value
0.1	0.08
0.2	0.15
0.3	0.25
0.4	0.38
0.517*	0.5
0.6	0.62
0.7	0.75
0.8	0.87

Cross-AI mean across 11 models (GPT-2 blue, Pythia green, Llama orange). Individual AI crossings range from T_d = 0.425 to 0.562.

The population inversion isn't a featureless crossover. Different AIs take different routes through it.

For Llama-3.1-70B, frozen output — where the AI produces the same sentence endlessly — dominates from low temperatures all the way up to T_d = 0.8. Only at higher temperatures does noisy output become more common. The system is locked in an attractor basin, resistant to perturbation, until a threshold is crossed.

Pythia-12B tells a different story. Its dominant regime shifts from frozen to sparse to intermittent before finally yielding to complex and noisy output at high temperatures. It wanders through several recurrent patterns, sampling different basins before escaping to variety.

These "structured routes through the transition" are evidence that temperature isn't just adding noise to a fixed process. It's actively reorganizing the recurrent structures that autoregressive feedback sustains — the loops and cycles that the AI gets trapped in when it reads its own output and generates more text based on what it's already said.

The most striking finding, however, concerns a hidden coordinate the researchers discovered within Llama-3.1-70B's internal dynamics. They weren't looking for it. It emerged from their analysis of the hidden states — the pattern of activations in the neural network that represents what the model "knows" at each step of generation.

They found that the hidden states from trajectories with high repetition clustered differently from those with low repetition. By taking the average normalized hidden state of the top quartile of trajectories (ranked by repetition) and subtracting the average of the bottom quartile, they derived a direction in the model's high-dimensional activation space. This direction — which they call the hidden coordinate x_n — acts as a scalar position along a particular axis through that space.

The coordinate predicts repetition with remarkable accuracy. Across 140 separate test trajectories (ten seeds at each of fourteen temperature values), the trajectory-averaged value of x_n correlates with repetition at rank correlation 0.933. The 95% confidence interval runs from 0.914 to 0.948 — tight enough to be convincing.

This correlation isn't just statistical. The coordinate changes sign at the same point where repetition abruptly drops — between T_d = 0.7 and T_d = 0.8. Positive x_n corresponds to more repetition; negative x_n corresponds to less. The macroscopic population inversion and this microscopic coordinate are aligned, despite being constructed independently.

But the researchers went further. They asked: can we actually change the behavior by steering the model along this coordinate?

They could. By applying perturbations to the hidden states — nudging the AI's internal activations up or down along the hidden direction — they changed the output repetition in predictable ways. At T_d = 1.0, where the population is strongly inverted, the strongest positive steering (α = 2) increased repetition by 0.255 on average. This wasn't a small effect buried in noise; the 95% interval was 0.097 to 0.433. The steering pushed trajectories back toward the recurrent sector.

The effect was causal, not coincidental. When they compared 36 combinations of perturbation direction, steering strength, and temperature, behavioral change correlated 0.842 with the later change in x_n — the value of the coordinate after the autoregressive dynamics had had time to evolve. It correlated only 0.199 with the immediate change. Interventions that produced similar changes in the later coordinate produced similar behavioral effects, regardless of how different they looked at the moment of application. The hidden coordinate is not just a predictor; it's an organizing axis for the system's dynamics.

Finally, the researchers showed that the dynamics can be captured by a simple mathematical model — a nonlinear map that describes how x_n evolves from one token to the next. They derived this map by projecting the transformer's operations (the attention mechanism, layer normalization, the feedforward network) onto the hidden direction, and found that the leading behavior follows the equation:

$$x_{n+1}\simeq a+b{x}_n-c{x}_n^3+\kappa x_{n-1}+{\eta }_n$$

where a represents bias (positive values favor repetition), b is the feedback gain for small displacements, c > 0 provides cubic saturation that limits growth, κ represents short-term memory from the previous step, and η_n captures unresolved fluctuations.

This is the logistic map's cousin — the equation that governs population dynamics in biology, the dripping faucet in physics, and the behavior of certain electronic oscillators. It produces fixed points, cycles, intermittency, and chaos depending on its parameters. And the researchers found that it describes the AI's behavior at the token level, with temperature entering through its parameters a, b, c, and κ.

Why This Changes Things

The "stochastic parrot." That's the phrase that has haunted large language models since Emily Bender, Timnit Gebru, and colleagues coined it in 2021. The idea is seductive in its simplicity: these systems are trained on enormous corpora of human text, they learn statistical patterns, and when they generate output, they're just recombining those patterns without genuine understanding. The stochastic (random) part refers to the sampling process; the parrot part refers to the lack of genuine cognition.

The metaphor has been useful. It reminds us to be humble about what AI systems actually understand. It warns against anthropomorphizing pattern-matchers. It's been a useful corrective to the hype that surrounded early GPT releases.

But Johnson's team has found something that the parrot metaphor can't explain.

If an AI were truly stochastic — if it were just adding randomness at each step — then raising the temperature would monotonically increase the entropy of the output. More temperature means more randomness means more variety. That's what the metaphor predicts. That's not what happens.

Instead, the AI's output undergoes a genuine phase transition. It passes through an entropy maximum and into inversion. Different AIs take different structured routes through this transition — frozen states for some models, cycling through sparse and intermittent patterns for others. The temperature parameter is doing something more subtle than injecting noise: it's reorganizing the recurrent structures that autoregressive feedback creates.

And there's something deeper still. The hidden coordinate x_n isn't just correlated with repetition — it's causally relevant. You can change the AI's behavior by steering along this coordinate. The fact that such a coordinate exists means that the AI's internal state space has structure that matters for output, structure that can be measured and perturbed.

"ChatGPT-like AIs therefore behave not as 'stochastic parrots'," the researchers conclude, "but as a new class of controllable nonlinear physical systems whose internal dynamics can be measured and perturbed."

This reframes the entire conversation. It's not about whether AI systems "understand" or "think" — that's a philosophical thicket the researchers wisely sidestep. It's about whether these systems have internal dynamics that are predictable, measurable, and controllable. The answer, according to this paper, is yes.

For the field of AI safety, this is potentially significant. Current approaches to understanding and controlling AI behavior often focus on outputs — on benchmarks, on red-teaming, on fine-tuning on examples of bad behavior. The finding that there's a measurable hidden coordinate that predicts and controls repetition suggests a different approach: internal intervention. If you can measure the state that leads to bad outputs, and if you can perturb that state to change the output, you have a new lever for control.

The analogy to physics is instructive. Lasers were once mysterious devices that produced coherent light through processes that weren't fully understood. Today, laser physics is mature, and lasers are engineered with precision. The population inversion in lasers — where more atoms end up in excited states than in ground states, the opposite of thermal equilibrium — is a feature, not a bug. It's what makes lasers work.

Johnson's team is suggesting that we might be in a similar early stage with AI. We have systems that exhibit strange collective phenomena — population inversion, attractor states, intermittency — that we don't fully understand. But we have the tools to study them. The fact that the dynamics can be captured in a simple nonlinear map, and that the hidden coordinate can be measured and steered, suggests a path toward not just understanding but engineering.

There's also something remarkable about the simplicity of what the researchers found. The nonlinear map in Equation 2 — five terms, one cubic nonlinearity — captures leading behavior across all temperatures and seeds. This is the kind of parsimonious explanation that physics loves. It suggests that underneath the billions of parameters and the incomprehensible computations of a large language model, there's a simpler structure that emerges at the right level of description.

This is the logic of effective field theory — the physicist's approach to complex systems. You don't need to know everything about the fundamental interactions to predict the behavior of water waves; the effective equations at the macro scale are simple even though the micro physics is not. Johnson's team is suggesting that large language models have a similar effective description, at least for the phenomenon of repetition, and they've found the right coordinate to describe it.

What's Next

The paper opens as many questions as it closes. The hidden coordinate x_n is measured in Llama-3.1-70B, but the population inversion is observed across eleven different models — a cross-AI survey that suggests the phenomenon is general, not specific to one architecture. Whether x_n or its equivalent can be found in other models is an open question.

The nonlinear map is phenomenological — it describes the behavior, but it doesn't derive it from first principles. Why does the attention mechanism project onto an S-shaped response that produces cubic saturation? Why does the hidden direction separate repetition from non-repetition so cleanly? These questions require deeper mechanistic analysis, possibly connecting to the growing body of work in "mechanistic interpretability" that tries to reverse-engineer what specific circuits and neurons in large language models are doing.

The paper also leaves open the question of what other hidden coordinates might exist. If one direction in activation space predicts repetition, are there other directions that predict other properties — accuracy, coherence, stylistic alignment with a target voice? The researchers focused on repetition because it's a concrete, measurable behavioral failure mode, but the same methods could presumably be applied to other aspects of output quality.

There's a deeper question lurking in the results: what determines whether a particular prompt puts the AI in a basin of attraction that leads to frozen output, or a trajectory that wanders through intermittent patterns, or a path to complex varied prose? The answer involves the interaction between the prompt, the model weights, and the temperature setting, but the precise mapping is not yet understood.

The paper's framing — "controllable nonlinear physical systems" — raises the question of control. The steering experiments showed that it's possible to perturb the hidden state and change the output, but the perturbations were applied at each step during generation. Can the same effect be achieved with a single perturbation at the start? Can we train a smaller "controller" network to predict the right perturbation for any given prompt? Can we incorporate the hidden coordinate measurement into a real-time control loop that prevents repetition before it starts?

These are engineering questions that follow from the science. And they follow because the researchers have established that the underlying dynamics are measurable and predictable. The systems aren't random; they're nonlinear. And nonlinear systems, unlike random ones, can in principle be controlled.

The caveats matter. The analysis focuses on repetition, a specific and well-defined failure mode. Whether the same methods can be applied to other failures — hallucination, loss of fidelity to the prompt, representation drift — is unknown. The AIs studied are open-source models where the weights are available; the methods may not transfer directly to closed models like GPT-4 or Claude, where users can only observe outputs, not internal states. The experiments were done on base models, not instruction-tuned or RLHF-trained variants; the alignment process might change the dynamics in ways this paper doesn't address.

And yet, even with these limitations, the paper represents a genuine shift in how we think about large language models. It moves them out of the realm of "stochastic systems" — which are fundamentally unpredictable in their fine-grained behavior — and into the realm of "nonlinear dynamical systems," which are unpredictable in their fine-grained behavior but predictable in their aggregate statistics and potentially controllable in their global dynamics.

The entropy maximum and the population inversion aren't bugs in the metaphor. They're features of the system — measurable, structured, and exploitable. And once you see them, you can't unsee them. Every time you adjust the temperature slider in a chatbot interface, you're not just adding randomness. You're driving the system through a phase transition, reorganizing its internal dynamics, pushing it across a boundary between attraction and inversion. You're doing physics.

The question is what we do with that. Physics gave us control over heat, electricity, and nuclear reactions. Nonlinear dynamics gave us understanding of chaos, pattern formation, and turbulence. What happens when we apply the same lens to the systems that are increasingly making decisions about medicine, law, business, and security?

Johnson and his colleagues don't answer that question. But they've shown that it's worth asking. The AIs aren't parrots — stochastic or otherwise. They're something stranger and, for that very reason, more interesting: physical systems that can be measured, understood, and, perhaps, steered.

The figure above shows how GPT-2's output evolves as temperature rises, moving from a single repetitive attractor through additional periodicities and switching, eventually reaching the balance of order and disorder characteristic of human-like prose. It's a visual demonstration of the transition that the population analysis captures quantitatively — the system restructuring itself at fixed temperature, in real time, token by token.

That's what the paper reveals: not just that the behavior is strange, but that it's structured, reproducible, and legible to the tools of physics. The "stochastic parrot" metaphor served a purpose — it reminded us to be humble about what AI systems actually understand. But understanding and control are different things. You can understand a system without being able to control it, and you can control a system without understanding it deeply. Johnson's team is suggesting we might be able to do both — to understand enough about these systems to control them, and to control them in ways that are grounded in genuine understanding.

That's a different kind of optimism than the breathless "AGI by 2030" variety. It's quieter, more grounded, and more honest about how much we still don't know. But it might, in the end, be more useful. Because the challenge isn't just to build powerful AI systems. It's to build AI systems we can trust. And trust, in the end, requires control — and control requires understanding. This paper is a step toward that understanding, one nonlinear map at a time.