The hidden field notes inside every satellite pixel
Annual satellite summaries can reveal fires, crops and disturbances—but two popular products remember different parts of the landscape.
85% of held-out disturbances were dated to the expected window by features hidden inside satellite embeddings.
A satellite pixel can remember a fire, a harvest, even a landslide—and researchers have found a way to read the memory. In tests spanning six continents, a set of features extracted from annual satellite data dated 85% of held-out disturbances to the expected time window. The result matters because governments, conservation groups and scientists increasingly use these compact data products to monitor land and make decisions about forests, farms and carbon.
The data in question are called Earth embeddings: learned numerical summaries of a year of satellite observations, created for every 10-meter patch of land. They are meant to stand in for the images themselves. But until now, users could know that an embedding helped a model answer a question without knowing what information it actually held. Robin Young and Srinivasan Keshav’s study offers a way to look inside—and finds that the summaries can preserve surprisingly specific records of what happened on the ground.
The Science
Satellites take repeated measurements of the planet, but the raw imagery is vast and often difficult to work with. Geospatial foundation models compress those observations into embeddings: lists of numbers, one for each pixel and year. Two products, Tessera and AlphaEarth, make annual embeddings available at global scale. Researchers and agencies can use them as input to map land cover, study agriculture or track ecological change.
The catch is that an embedding is not a labeled photograph. A vector may contain information about a burn scar, but its many dimensions can mix that signal with other properties of the landscape. A model trained on the vector might still recognize the scar, yet give no direct clue which components carry the information—or whether it is present in a consistent, interpretable form.
Young and Keshav used a sparse autoencoder, a method that breaks a complex representation into a larger collection of components called features. Each pixel is reconstructed from a small number of these features, each with a non-negative activation level. The goal is to make features more specialized than the original embedding dimensions: one might respond to a burned surface, another to a crop, and another to a landform. The method was developed in work on interpreting language models; here it is applied to satellite data.
A feature is not automatically meaningful just because it has a name or a distinctive map. The authors tested whether activations aligned with independent reference maps and with the timing of real events. For example, a fire-related feature should become more active on burned pixels after the fire, remain quiet before it, and eventually fade as the landscape recovers. The study tested features against 3,940 mapped events and crop units, across 16 settings on six continents.
A striking first example comes from California’s 2021 Dixie Fire. The burn scar appears in the embedding, but its signal is distributed across many of the original dimensions. The sparse decomposition identifies a family of 32 features whose average activation traces the scar. On that tile, the family distinguished burned from unburned pixels with an area-under-the-curve score of 0.914; 96% of pixels above the family’s median activation were mapped as moderately or severely burned. The authors then applied the same family, without retuning, to three other Sierra Nevada fires, where its contrast also peaked in the first year after the fire.
Disturbance signals survive in annual embeddings
Held-out validation rate and random-feature null median for 14 dated settings. Values are percentages, transcribed from the paper’s validation table.
| Label | Value |
|---|---|
| Western US fire | 88.9 % |
| SE US wildfire | 79.2 % |
| SE US prescribed fire | 79.8 % |
| Boreal fire | 93.9 % |
| SE Australia fire | 62.4 % |
| Europe harvest | 83.2 % |
| Europe wind/bark beetle | 84.1 % |
| Europe fire | 66.9 % |
What They Found
Across the full set of disturbance tests, feature families were usually able to identify the expected event window on data they had not been selected from. In the 14 dated settings, held-out validation ranged from 62.4% for southeastern Australian fires to 100% for rainfall-triggered landslides in Italy and China. The rates exceeded the median performance of random feature sets of the same size in every dated setting. For European forest harvest, for instance, the feature family dated 83.2% of 826 held-out tile-years to the expected window, compared with a random-feature median of 30.1%.
These scores are not all measuring exactly the same thing. Some reference maps record the year an event happened; others record when an annual satellite product first detected it. The expected windows reflect that distinction. The key result is not that embeddings replace specialist monitoring systems, but that a general-purpose representation retains event-linked information without having been trained specifically to detect each event.
The features also captured a useful ecological clock. Their activation tended to peak in the calendar year containing the first growing season after a disturbance. That timing depends on both the date of the event and how quickly the vegetation responds. In western US conifer forests, only 2% of wildfires peaked in the event year: trees generally do not regrow during the remaining months after a burn. In the southeastern US, where understory plants can return within weeks, 80% of prescribed fires peaked in the event year. A satellite summary can therefore reflect not just the calendar date but the landscape’s biological response.
When the fire signal peaks depends on regrowth
Share of units whose feature-family contrast peaked in the event year for selected US fire settings. Values reported in the paper.
| Label | Value |
|---|---|
| Western US wildfire | 2 % |
| SE US prescribed fire | 80 % |
That record can distinguish severity, but not necessarily cause. The southeastern wildfire feature responded more strongly to wildfires than to prescribed burns, enough that the researchers could separate the two in a matched comparison. The strongest activation deciles consisted of 88–94% wildfire, compared with a pooled wildfire share of 57%. But windthrow, bark beetle damage and harvest can all remove tree canopy abruptly. When they leave similar surface conditions, the embedding’s features largely overlap. The data record what happened to the surface more clearly than why it happened.
One of the most consequential results is that two widely used embedding products do not encode the same things, despite drawing on largely the same satellite observations. Tessera detected small southeastern fires that recovered quickly: the burned-versus-unburned signal after the event was 1.56 times the pre-event noise level. AlphaEarth’s corresponding ratio was 1.09, close to no change, and the same feature-screening method found no fire feature there. For crops, the balance reversed: corn and soy were separated by 3.19 within-class standard deviations in AlphaEarth, compared with 1.43 in Tessera.
The study proposes one explanation, while treating it as a hypothesis rather than a settled fact. Tessera is trained to represent a given year consistently across different samples of that year’s observations, which may make it more sensitive to events that happen within that year. AlphaEarth represents places as a continuous function of time, potentially emphasizing persistent conditions and smoothing short-lived changes. The contrast suggests that an embedding’s usefulness depends on the question: a product well suited to crop classification may be less suited to detecting a brief disturbance.
AlphaEarth retains more disturbance signal two years later
Ratio of disturbance signal retained by AlphaEarth compared with Tessera two years after the signal peak, for the same events.
| Label | Value |
|---|---|
| Western US megafires | 2.8 × |
| SE US wildfires | 3.5 × |
| SE US prescribed fires | 1.7 × |
The paper’s crop example shows what interpretability adds beyond a good prediction score. In an Iowa tile, one raw embedding dimension mixed two different ends of a spectrum: its lowest values were mostly corn, while its highest values were mostly developed land. A decomposed feature was more direct: 91% of pixels where it activated were corn, although corn covered 45% of the tile overall. Across fields recorded as switching from corn to soy, the corn feature’s active share fell from 82% to 5%, while the soy feature’s rose from 5% to 83%. The features followed the crop in the field rather than simply identifying a fixed property of the location.
Why This Changes Things
Land monitoring increasingly feeds decisions about carbon accounting, forest protection, agricultural production and biodiversity. Those decisions depend on knowing not only whether a model performs well, but what its inputs can and cannot tell us. If an embedding contains a repeatable signal of canopy loss or crop rotation, analysts may be able to use one compact data product to ask many questions—without starting from raw satellite imagery each time.
Interpretability also makes a practical difference when choosing a product. A researcher studying rapid, small fires might favor Tessera based on these results; someone separating corn from soy might find AlphaEarth more informative. That is not a universal ranking. It is evidence that representations with similar inputs can preserve different aspects of the world, much as two maps made from the same terrain can emphasize different features.
The cross-region tests offer another reason for cautious interest. Some learned concepts transferred across continents: a winter-wheat feature identified wheat in Europe about as well as in the US Plains, with an area-under-the-curve score of 0.80 in both regions. A California fire family also responded to European fires, while remaining distinct from European harvest and windthrow. These examples suggest that at least some features capture surface conditions rather than merely memorizing local geography.
That is valuable for global monitoring, where reference labels are uneven and expensive to assemble. But it does not mean an embedding can replace field observations, carefully validated change-detection systems or local knowledge. The paper itself shows why: an annual summary cannot resolve exactly when within the year a change happened, and distinct causes can become indistinguishable when they leave the same physical footprint.
What’s Next
Annual composites impose a hard limit on timing. The authors compared 17,014 Amazon clearing plots dated to the day by radar alerts with the annual embeddings. Whether a clearing occurred earlier or later within the year did not change which annual record captured it more reliably. In other words, the embedding may reveal the first growing season in which a changed surface appears, but it cannot recover day-level timing that the annual input has already blurred.
Nor did the researchers find a reliable early-warning signal. When they searched for features that became active before a mapped disturbance, the average pre-event effect was small and its uncertainty included zero. The study also covers a defined set of event types and regions, not every landscape or ecological process. Its strongest findings concern changes that are independently mapped and leave a visible surface signature.
There is still work to do before these features become routine tools. Researchers will need to test stability across model versions, years, sensors and additional ecosystems; establish how thresholds transfer between regions; and examine whether the feature families remain useful under changing climate and land-management practices. The proposed explanation for differences between Tessera and AlphaEarth also needs direct tests that isolate training objectives from other design choices.
Still, the core advance is important: a compressed satellite vector need not remain an inscrutable list of numbers. It can be decomposed into signals that correspond to fires, crops and other changes, then checked against records independent of the model. That opens a path toward Earth data products that are not only powerful, but legible enough for scientists and decision-makers to know what they are seeing—and where the blind spots remain.
The embeddings record what a disturbance did to the surface, not what caused it.
Sign in to join the conversation.
Comments (0)
No comments yet. Be the first to share your thoughts.