Meridia Insight Tech for Good Frontiers

79.3% of Tech's Big Leaps Happen in a Single Year — And What Comes After Doesn't Matter

79.3% of a tech capability's improvement arrives in the first year of a leap — and commercial signals can't predict the next breakthrough.

79.3% of a tech leap's gain happens in year one — and competition data adds zero predictive power.

79.3%. That’s the share of total capability improvement that arrives in the very first year of a technological leap — and never gets meaningfully surpassed in the next two. Across 204 electronic-component capabilities, from memory speeds to power efficiency, the most consequential advances aren’t the start of a steady climb. They’re sudden, concentrated bursts, followed by silence. And here’s the real shock: everything we thought might predict the next breakthrough — how many companies are competing, how fast product lines are evolving, how crowded the market is — adds zero predictive power beyond simply knowing when the last leap happened.

This isn’t how innovation is supposed to work. We imagine progress as a relay race: one company takes the baton, improves the design, passes it on. Or as an arms race: rivals push each other forward through competition. But a sweeping analysis of 1.74 million commercial electronic components reveals a different reality — one where the frontier advances in sudden, isolated jumps, and the surrounding ecosystem of products, manufacturers, and market dynamics offers no usable signal about when or where the next one will land.

The Science

Kenzi M’Hamdi and Dawoon Jeong, sociologists at Brown and the University of Chicago, set out to test long-standing theories about how technology evolves. Drawing from evolutionary economics, they framed technological change as a process of variation, selection, and retention — much like biological evolution. New designs emerge (variation), some succeed in the market (selection), and successful features get copied or built upon (retention). A key question is whether the commercial landscape — the diversity of products, the intensity of competition, the pace of product-line turnover — can help predict the next major leap at the performance frontier.

To find out, they built a dataset of 1,739,040 product records from DigiKey and Mouser, two of the largest electronics distributors. These aren’t lab prototypes or patent claims — they’re real components being sold to engineers and manufacturers. From this, they extracted 204 distinct technical capabilities, like maximum clock speed in microcontrollers or leakage current in transistors, each with a clear direction of improvement (higher or lower is better).

For each capability, they defined the “frontier” as the 99th percentile of performance in any given year — the best commercially available option. They then measured how this frontier advanced over time, identifying “substantial advances” as jumps of at least 26% (equivalent to a 0.10 log10-unit increase). They also tracked the broader population: how many products were near the frontier, how the distribution of specs evolved, and how manufacturer behavior changed.

The study had two main phases. First, a deep dive into 29 carefully selected “anchor” capabilities to characterize the tempo of change. Second, a forecasting experiment across all 204 capabilities: could models that included commercial signals — like manufacturer concentration, product-line growth, or how far behind the frontier most products were — predict the next big leap better than models using only the history of the frontier itself?

The models used L2-penalized logistic regression, a standard machine-learning approach for binary outcomes (advance or no advance). The baseline model used four predictors based solely on the frontier’s past: recent growth, time since last advance, current maturity (how high the frontier already is), and change in the annual 99th percentile. The enriched model added 20 more predictors from the commercial ecosystem: competition metrics, product population dynamics, and signals from related capabilities in the same device category.

Crucially, the evaluation was forward-looking and out-of-sample: models trained on data up to year Y predicted advances in year Y+1, using only information available at the time. This mimics real-world forecasting, avoiding the trap of overfitting to historical patterns.

What They Found

The data revealed a striking pattern of punctuated equilibrium — long periods of stability interrupted by sudden, concentrated advances. In the 29 anchor capabilities, 77.8% of substantial advance episodes lasted just a single year. That is, in nearly eight out of ten cases, a big leap was followed by no further improvement in the next year — and often for many years after.

Even more telling: when they looked at three-year windows starting with an advance, the initial jump accounted for an average of 79.3% of all the improvement seen in that window. In other words, if a capability improved by 50% over three years, 39.6% of that gain happened in year one. The rest was noise.

Single-Year Advance Episodes Dominate

Single-Year Advance Episodes Dominate
LabelValue
Threshold: 0.05 log100.72 %
Threshold: 0.10 log100.778 %
Threshold: 0.15 log100.81 %

This concentration wasn’t just about timing — it was mirrored in the market. The median share of products within 75% of the frontier was just 1.46%. For most engineers designing real products, the bleeding edge might as well not exist. The vast majority of components sold are comfortably behind the peak, clustered around older, stable specifications.

Figure 2: Frontier–population separation. a: product-record shares near their own year’s cumulative-p99 benchmark across 29 anchor capabilities; dashed line: median 1.46%. b: the six largest retained end ratios of direction-adjusted cumulative p95 to the smoothed median, ranked by end ratio. Teal denotes each series’ start and orange its end; years differ across series. Two confirmed incompatible current fields are withheld. The panels use different frontier summaries
Figure 2: Frontier–population separation. a: product-record shares near their own year’s cumulative-p99 benchmark across 29 anchor capabilities; dashed line: median 1.46%. b: the six largest retained end ratios of direction-adjusted cumulative p95 to the smoothed median, ranked by end ratio. Teal denotes each series’ start and orange its end; years differ across series. Two confirmed incompatible current fields are withheld. The panels use different frontier summaries Source: Kenzi M'Hamdi, Dawoon Jeong

And then came the forecasting result that upended expectations. Despite the rich commercial data, the model with 20 extra predictors did not outperform the simple frontier-history model. In fact, it was slightly worse. The average AUC (a measure of predictive accuracy from 0.5 to 1.0) was nearly identical — 0.695 for the simple model, 0.692 for the enriched one — and the difference was not statistically significant. In most individual years, the enriched model performed worse at ranking which capabilities were more likely to advance.

Forecasting Performance: No Gain from Commercial Signals

Forecasting Performance: No Gain from Commercial Signals
LabelValue
20050.02
20060.01
20070.03
20080
20090.02
20100.01
20110
20120.01

Even more troubling: the enriched model became overconfident. At high predicted probabilities (above 60%), it consistently overestimated the chance of an advance. It saw signals where there were none, mistaking commercial activity for technological momentum.

Figure 4: Adding product indicators lowers forecast accuracy in most years and yields overconfident higher-probability predictions. a: annual AUC with twenty added product-population indicators minus AUC with four frontier-history predictors only; negative bars indicate poorer ranking. The dashed line is the mean; its 95% interval resamples annual folds. b: within each model-specific probability band, open circles show mean predicted chance and filled diamonds show observed advance frequency. A diamond to the left of its circle indicates overprediction. Counts are capability-years; gray rows have fewer than 20. Both panels use the same 19 test years (2005–2023), comprising 2,393 predictions per model
Figure 4: Adding product indicators lowers forecast accuracy in most years and yields overconfident higher-probability predictions. a: annual AUC with twenty added product-population indicators minus AUC with four frontier-history predictors only; negative bars indicate poorer ranking. The dashed line is the mean; its 95% interval resamples annual folds. b: within each model-specific probability band, open circles show mean predicted chance and filled diamonds show observed advance frequency. A diamond to the left of its circle indicates overprediction. Counts are capability-years; gray rows have fewer than 20. Both panels use the same 19 test years (2005–2023), comprising 2,393 predictions per model Source: Kenzi M'Hamdi, Dawoon Jeong

Why This Changes Things

These findings challenge two deeply held beliefs about innovation.

First, the myth of cumulative, incremental progress. We like to think of Moore’s Law as a smooth curve, but this study shows it’s really a staircase — and most of the rise happens in a single step. The same is likely true in batteries, solar cells, gene sequencing, and beyond. A 2023 battery breakthrough that doubles energy density isn’t the start of a decade of steady improvement; it’s probably the only leap we’ll get for years. The rest will be minor tweaks, packaging changes, and cost reductions — important, but not transformative.

This has profound implications for R&D strategy. If big advances are sudden and unpredictable, then pouring resources into “next-generation” roadmaps based on linear extrapolation may be futile. Instead, organizations might need to prioritize optionality — maintaining multiple parallel research threads, staying alert to discontinuous jumps, and being ready to pivot when a new frontier opens.

Second, the illusion of market signals. Economists and strategists have long believed that competition drives innovation. More rivals, faster product cycles, broader adoption — these should, in theory, create pressure and opportunity for breakthroughs. But this study finds no evidence of that. The commercial ecosystem, for all its complexity, is decoupled from the process that generates frontier advances.

That doesn’t mean markets are irrelevant. They determine which advances get adopted, scaled, and monetized. But they don’t seem to shape which advances happen in the first place. The spark appears elsewhere — in fundamental research, in serendipitous discovery, in the recombination of distant ideas.

This decoupling suggests a paradox: the most economically valuable innovations are the least predictable. And the signals we rely on to anticipate them — market share, product velocity, competitive intensity — are noise.

It also reframes the role of dominant firms. If competition doesn’t drive leaps, then monopolies or oligopolies aren’t necessarily innovation killers. In some cases, they may even be better positioned to make big bets, absorb failures, and integrate radical advances — precisely because they’re insulated from short-term market pressures.

What's Next

The authors don’t claim to have solved the mystery of where breakthroughs come from. Instead, they’ve ruled out a major class of explanations. The next step is to look beyond commercial products — to the conditions that make a technical opportunity economically relevant in a specific application.

They propose a new measurement agenda: using language models to extract demand signals from historical documents — engineering journals, product manuals, customer reviews — and linking them to capability trajectories. Did a surge in electric vehicle patents precede advances in battery energy density? Did data-center cooling challenges drive innovations in low-leakage transistors? These are the kinds of application-specific pressures that might truly shape the direction of progress.

This approach could finally bridge the gap between “technology push” and “demand pull.” The push comes from scientific and engineering possibility; the pull comes not from aggregate market size, but from the specific, often hidden, constraints and aspirations of real users.

For policymakers, the lesson is humility. You can’t engineer a breakthrough by tweaking market competition. But you can create the conditions for them: fund basic research, support high-risk exploration, and build ecosystems where unexpected connections can form.

For companies, it’s a call to balance exploitation with exploration. Don’t assume your market intelligence will reveal the next leap. Stay close to the frontier, but don’t confuse commercial activity with technological momentum.

And for all of us, it’s a reminder that progress isn’t a machine. It’s a series of surprises — and the best way to prepare for the next one is to expect the unexpected.

Comments (0)

No comments yet. Be the first to share your thoughts.