Bigger AI Models Aren’t Smarter — They’re Just More Expensive

When a 70-billion-parameter AI model faces a math problem, it doesn’t just fail — it fails expensively. On average, it burns through hundreds of tokens generating long, meandering chains of thought, only to arrive at the wrong answer. And here’s the shock: that same model isn’t meaningfully more efficient than its 1.5-billion-parameter cousin. In fact, across four types of reasoning tasks — from basic arithmetic to algorithmic logic — larger models solve harder problems only slightly better, and they do so with no real improvement in token efficiency. According to a 2026 study by Laber et al. (Laber et al., 2026), scaling up model size yields diminishing returns in capability and virtually no gains in efficiency. This isn’t just a technical footnote. It’s a warning sign for the entire trajectory of AI development: brute-force scaling may be hitting a wall.

The Science

The researchers tested five versions of the DeepSeek-R1-Distill family of large reasoning models (LRMs), ranging from 1.5 billion to 70 billion parameters. These models were evaluated on four synthetic reasoning tasks: addition (summing a list of integers), brackets (checking if parentheses are properly nested), index (retrieving an element from a list), and parity (determining if the number of 1s in a binary string is even). Each task has a tunable difficulty — controlled by instance size $n$ — allowing the team to systematically probe how performance changes as problems grow harder.

For each task and size $n$, they generated 100 random instances and prompted each model to solve them using Chain-of-Thought (CoT) reasoning. They recorded two key metrics: (1) whether the answer was correct, and (2) the number of output tokens the model used to reach that answer. This setup allowed them to separate capability — the ability to solve a problem — from efficiency — the ability to solve it with minimal computational cost.

To model capability, they assumed that the probability of solving a problem correctly decays exponentially with instance size $n$:

$$\mathcal{P}(\text{correct}\mid n)=p^{(\infty )}+(p^{(0)}-p^{(\infty )})\exp \left(-\frac{n}{\nu }\right)$$

Here, $\nu$ is the decay scale — a measure of how far a model can “reach” before performance collapses. A larger $\nu$ means the model can handle bigger, harder problems. They then asked: how does $\nu$ scale with model size $N$? They fit a power law:

$$\nu (N)\propto N^{{\beta }^{(\nu )}}$$

For efficiency, they analyzed the average output length $\ell$ of correct responses. They found it follows a power law with instance size:

$$\ell (n)\propto n^{\alpha }$$

The key question: do the prefactor $A$ and exponent $\alpha$ of this power law improve with model size? If larger models were more efficient, we’d expect $A$ to shrink (fewer tokens per problem) or $\alpha$ to decrease (slower growth in token use). But the data told a different story.

The analysis used hierarchical Bayesian models to infer these scaling relationships, accounting for uncertainty and variation across tasks and temperatures ($\tau \in {0.4, 0.6, 0.8}$). This statistical rigor allowed them to draw robust conclusions even when some model-task combinations performed poorly.

What They Found

Capability improves — but with sharply diminishing returns.

The probability of solving a problem correctly drops exponentially as the problem gets larger. For example, in the addition task, a 1.5B model might solve 90% of 10-number sums but only 10% of 100-number sums. The decay scale $\nu$ — the “half-life” of performance — increases with model size, but not linearly. Instead, it follows a power law with exponent $\beta^{(\nu)} \approx 0.69$. That means doubling the model size increases $\nu$ by only a factor of $2^{0.69} \approx 1.61$.

In practical terms: going from 1.5B to 70B parameters — a 47-fold increase — boosts the characteristic problem size a model can handle from about 26 to 364. That’s impressive, but the marginal gain per parameter plummets. The biggest leaps happen at the lower end; each additional billion parameters adds less and less capability.

Capability Scaling: Decay Scale vs Model Size (Addition Task, τ=0.6)

Label	Value
1.5B	26.4
7B	89.1
14B	142.3
32B	231.7
70B	363.5

Efficiency barely improves at all.

Now consider token use. When models do solve a problem correctly, their output length grows as a power law with instance size: $\ell \propto n^{\alpha}$. For the index task, $\alpha \approx 1.1$, meaning output length grows slightly faster than linearly with input size.

Critically, this relationship does not meaningfully change with model size. The prefactor $A_{N,\tau}$ — the baseline token cost — shows no systematic trend. The scaling exponent $\alpha_{N,\tau}$ is equally flat. The posterior mean of the efficiency-scaling exponent $\beta^{(A)}$ is nearly zero: $\mathbb{E}[\beta^{(A)} \mid \mathcal{D}] \approx -0.03$, with a 95% credible interval straddling zero.

This means a 70B model solving a 100-element indexing problem uses about as many tokens per unit of difficulty as a 1.5B model. It’s not thinking more concisely. It’s not discovering shorter algorithms. It’s just thinking longer — and often incorrectly.

The authors tested this across all four tasks and three temperatures. The result held: capability scales sublinearly, efficiency doesn’t scale at all.

shows the stark contrast between the two scaling regimes.

Why This Changes Things

For years, the AI industry has operated on a simple bet: scale up, and performance will follow. This “scaling hypothesis” has been remarkably successful. Larger models do perform better on benchmarks, generate more coherent text, and pass more reasoning tests. But this study reveals a critical flaw in that logic: better doesn’t mean smarter — it just means more expensive.

Consider the economic implications. Inference — running a model to answer a user query — is where AI costs hit the bottom line. Every token a model generates costs money in compute and energy. If larger models don’t become more efficient, then deploying them at scale becomes a losing game. A 70B model might solve 20% more math problems than a 7B model, but if it uses 5x the tokens to do so, the cost-per-correct-answer could be higher. That’s the opposite of progress.

This also challenges the idea that scaling alone will lead to “reasoning” in any meaningful sense. The models aren’t learning efficient algorithms. They’re not compressing knowledge. They’re not generalizing in a way that reduces cognitive load. Instead, they’re brute-forcing solutions with longer, more redundant chains of thought — a strategy that burns resources without gaining insight.

Take the brackets task, which requires checking proper nesting of parentheses. This is a classic computer science problem solvable in linear time with a simple stack. Yet the models don’t converge on this efficient strategy. Larger models don’t produce shorter, stack-like reasoning; they produce longer, more verbose attempts. The lack of improvement in $\alpha$ suggests they’re not learning better algorithms — just more of the same.

This has profound consequences for AI safety and alignment. If models scale in capability only through increased verbosity, then their reasoning becomes less interpretable, not more. A 70B model’s 500-token chain of thought is harder to audit than a 100-token one. And if that chain is only marginally more likely to be correct, the trade-off is deeply questionable.

It also reframes the debate over “emergent abilities.” Some have argued that sudden jumps in performance at large scales indicate qualitatively new behaviors. But this study suggests a more mundane explanation: what looks like emergence may just be the slow, sublinear crawl of exponential decay. A model with $\nu = 364$ can solve problems that a $\nu = 26$ model cannot — not because it “understands” them, but because its performance decay is slower. The boundary of solvability shifts outward, but the underlying mechanism remains the same.

What’s Next

This work doesn’t mean scaling is dead. It means naive scaling — just making models bigger — has limits. The path forward lies in intelligent scaling: improving architecture, training methods, and reasoning strategies that enhance both capability and efficiency.

One promising direction is test-time compute optimization. Instead of relying on longer chains of thought, models could use search, recursion, or dynamic programming — strategies that are algorithmically efficient. Techniques like best-of-$N$ sampling or Monte Carlo tree search allocate compute more wisely, potentially breaking the linear token-cost trap.

Another is post-training efficiency tuning. Just as models are fine-tuned for alignment, they could be fine-tuned for conciseness. Reinforcement learning with a reward for correctness and brevity might force models to discover shorter reasoning paths. Early work in “reward shaping” for efficiency shows promise (Ouyang et al., 2022), but it’s underexplored.

There’s also a role for benchmark design. Most reasoning benchmarks measure correctness, not efficiency. If we want efficient AI, we need to reward it. Imagine a leaderboard that ranks models by “correct answers per thousand tokens.” That would incentivize the kind of progress this study shows is currently missing.

Finally, this work underscores the need for diversity in AI development. The DeepSeek-R1-Distill family represents one architectural path. Other models — sparse, modular, or neurosymbolic — might scale more efficiently. The fact that this family shows no efficiency gains doesn’t mean all models must. But it does mean we can’t assume scaling will solve everything.

The big picture? We’re at a crossroads. We can keep building bigger, costlier models that inch forward in capability while burning ever more resources. Or we can pivot toward smarter, leaner AI — systems that don’t just answer questions, but answer them well. The first path leads to expensive mediocrity. The second could lead to truly intelligent machines. This study is a compass. It tells us which way we’re headed — and which way we should go.