1 Introduction↩︎

Earth Observation (EO) processing pipelines do not fail because imagery stops arriving. They fail because the images that arrive cannot be used. Clouds, haze, and missing valid pixels can hide the surface for days or weeks, making the observation quality the main challenge rather than data availability. In operational settings, that changes the question. The issue is not only what the surface will look like next, but whether the next acquisition will be usable at all and, if not, how long the wait will be.

We treat that as an observability forecasting problem. The aim is not pixel reconstruction, surface-index regression, or planning. It is to estimate when the scene becomes observable again after cloud-obstructed acquisitions.

EarthNet2021 was introduced as a benchmark for forecasting high-resolution Earth surface observations from coarser meteorological drivers [1]. Most work on the dataset therefore focuses on video prediction or surface forecasting. We use the same dataset for a different question: can a latent world model forecast whether future observations will be usable, whether any usable observation will return within a short horizon, and when that return is most likely to happen?

LeWorldModel (LeWM) is a recent world-model architecture based on a Joint-Embedding Predictive Architecture (JEPA) [2]. Instead of reconstructing pixels, it learns a latent state, predicts future latent states from past latents and covariates, and regularizes that space toward a stable Gaussian geometry. LeWM is a sensible model for cloud-aware EO monitoring because observability is fundamentally a temporal state problem, not a pixel synthesis problem. What matters operationally is not whether the model can draw the next image, but whether it can track the hidden evolution of scene visibility under changing weather conditions and recent observation history. EarthNet2021 provides exactly the ingredients for that setting: irregularly useful observations, cloud-driven partial observability, and exogenous meteorological covariates that shape what can be seen next. A JEPA world model is well matched to this structure. It learns a predictive state that must carry forward the information needed to anticipate future observation quality, while avoiding the burden of reconstructing every spatial detail of a cloudy frame. That is an important distinction. For this task, exact pixels are often ambiguous or operationally irrelevant, whereas the latent transition from obscured to usable is the signal that matters. The appeal of LeWM, then, is not just that it is a modern latent predictor; it is that its training objective is closer to the decision problem posed by cloud-aware monitoring.

The research question addressed in this paper is straightforward: once LeWM is adapted carefully to EarthNet2021 and evaluated under a locked protocol, does it provide a useful signal for cloud-aware monitoring?

Our contributions are practical and empirical:

• We reformulate the EarthNet2021 use case from generic surface forecasting to observability forecasting under cloud occlusion, with explicit targets for next-step usability, any usable future observation within horizon, and first usable future horizon.

• We provide a reproducible LeWM adaptation pipeline for this setting, including cloud-aware conversion from raw minicubes to episodic HDF5, explicit cloud-mask handling, and dataset-specific normalization.

• We define a locked evaluation protocol that separates model training, linear-probe fitting, validation-time calibration, and final reporting across EarthNet2021 valsplit, IID, OOD, and extreme splits.

• We show that LeWM is most convincing not as a pixel forecaster but as an observability model: it improves over persistence in latent forecasting, strongly predicts whether future observations will be usable, estimates when usable observations return, and provides useful temporal anomaly-ranking signals.

• We compare LeWM against both persistence and a frozen LightGBM baseline. The comparison reveals a clear task split: LeWM is strongest on continuous observation-quality regression and on exact recovery timing on most splits, while LightGBM is strongest on the simpler binary any-usable-within-horizon task and on OOD more broadly.

In short, the value of LeWM on EarthNet2021 is not that it becomes a spatial forecaster or a planning system. The value is that it can act as a latent observability forecaster for cloud-aware EO monitoring.

2 Related Work↩︎

2.0.0.1 Earth surface forecasting with EarthNet2021 and related EO forecasters.

EarthNet2021 was introduced as a benchmark for forecasting high-resolution Earth surface observations conditioned on mesoscale weather drivers [1], [3]. In that literature, the main objective is future surface prediction, either as image forecasting or as prediction of downstream surface indicators from recent satellite context and future forcing. Earthformer broadened that agenda by showing that transformer-based spatiotemporal models can capture geophysical dynamics at scale [4]. EO-WM pushes further toward probabilistic EO forecasting with weather-aware conditioning and diagnostics for weather-response fidelity [5]. We keep the same EarthNet2021 setting, but change the question. Our target is not future surface reconstruction. It is whether the future observation itself will be usable and when a usable view will return.

2.0.0.2 Clouds, missing data, and cloud-gap reconstruction.

Clouds and invalid pixels are a routine obstacle in EO monitoring, so much of the related literature treats the problem as gap filling, imputation, or cloud removal. SEN12MS-CR-TS formalized multimodal, multitemporal cloud removal with optical and SAR context [6]. UnCRtainTS adds calibrated uncertainty to cloud removal in optical satellite time series [7]. Other work uses partial convolutions or foundation-model-based priors for cloud-gap imputation [8], [9]. These methods are directly relevant as context, but they solve a different task. Their goal is to reconstruct the hidden surface. Ours is to forecast observation usability itself: whether the next acquisitions are useful enough for monitoring, and how long recovery will take.

2.0.0.3 EO self-supervised learning and foundation models.

A parallel line of work focuses on EO-specific representation learning rather than explicit future forecasting. SatMAE adapts masked autoencoding to temporal and multispectral satellite imagery [10], while Scale-MAE emphasizes multiscale geospatial representation learning [11]. Larger geospatial foundation-model efforts such as Prithvi argue that EO data benefit from domain-specific pretraining and broad transfer across downstream tasks [12]. Recent work further suggests that EO representation learning benefits from explicit geo-temporal conditioning and does not necessarily require very large models: a lightweight metadata-aware Mixture-of-Experts masked autoencoder combines imagery with latitude/longitude and cyclic seasonal/daily encodings and achieves competitive frozen-encoder transfer with only 2.5M parameters [13]. Together, these studies support the broader premise that EO representation learning should not simply inherit assumptions from natural-image pretraining.

2.0.0.4 Foundation models versus latent world models.

Even so, EO foundation models and latent world models are not interchangeable. Most EO foundation-model papers evaluate transfer to static or weakly temporal downstream tasks such as classification, segmentation, or regression after fine-tuning or linear probing. A world model asks for something more structured: a state representation that evolves coherently under time and exogenous forcing. That difference is central in our setting. Forecasting cloud-aware observability is not just a question of whether a representation is useful downstream. It is a question of whether the representation preserves the temporal state needed to predict visibility, usability, and recovery timing.

2.0.0.5 Latent predictive models, JEPA, and world models.

Outside EO, a long line of work studies predictive latent dynamics from high-dimensional observations. World Models, PlaNet, and Dreamer showed that compact latent states can support multi-step prediction and planning without reconstructing every pixel at every step [14]–[16]. JEPA-style learning shifts the emphasis from pixel reconstruction to representation prediction [17]. V-JEPA 2 extends that view to video understanding, prediction, and planning [18]. LeWorldModel brings these ideas together in an end-to-end JEPA world model with latent prediction and Gaussian regularization [2]. That makes it a natural candidate for EO settings where distinct pixel realizations can still correspond to the same monitoring outcome. In our case, different future images can all lead to the same answer: the observation remains unusable because cloud cover stays high.

2.0.0.6 Position of this work.

This paper sits at the intersection of these lines of work. From EarthNet2021 and EO forecasting, it inherits multispectral image sequences driven by exogenous weather. From cloud-gap work, it inherits the practical fact that cloud cover often sets the real limit on EO monitoring. From EO representation learning, it inherits the view that EO data benefit from domain-specific pretraining. From JEPA and world-model work, it inherits the idea that predictive state can be learned in latent space without explicit future-image synthesis. What is still missing in prior work is an explicit treatment of observability forecasting: whether the next observation will be usable, whether any usable observation is likely to appear within a short horizon, and when the first usable future observation is likely to return. That is the gap addressed here.

3 Application Setting↩︎

We frame the target use case as follows. An EO monitoring pipeline receives a sequence of multispectral observations and associated exogenous weather drivers. At each time step, the pipeline may need to decide among actions such as:

• whether the next acquisition is likely to be usable enough for surface analysis;

• whether to defer a monitoring decision because the next few observations will likely remain cloud-obstructed;

• when a useful observation is likely to become available again;

• whether a newly observed transition is temporally inconsistent with the recent context and should be flagged for quality control.

These are forecasting and inference problems, not control problems. The weather variables in EarthNet2021 are exogenous covariates, not actions that a planner can optimize. This distinction matters. It is valid to test forecast sensitivity to perturbed weather inputs, but it is not valid to claim policy optimization or action planning from this dataset alone.

4 Data and Preparation↩︎

4.1 Raw EarthNet2021 structure↩︎

EarthNet2021 training minicubes contain high-resolution Sentinel-2 image sequences and mesoscale meteorological variables [1]. In the raw training split, each cube contains:

• highresdynamic of shape \((128,128,7,30)\);

• mesodynamic of shape \((80,80,5,150)\);

• static variables that are not used in the present LeWM adaptation.

For the high-resolution dynamic tensor, the relevant channels are:

• channels \(0\ldots3\): blue, green, red, near-infrared (NIR);

• channel 6: binary cloud mask (cldmask), interpreted as 1 = cloudy and 0 = clear.

Channels related to cloud probability and scene class are not used by the final model. For the external paired context/target splits, the stored schema is already simplified: each dynamic cube has shape \((128,128,5,T)\) with the binary cloud mask in channel 4.

4.2 Cloud-aware conversion to LeWM episodes↩︎

We convert each minicube into one episodic sequence in the LeWM HDF5 format. The final observation tensor keeps five channels, \[x_t \in \mathbb{R}^{128\times128\times5} = [\text{blue},\text{green},\text{red},\text{nir},\text{cloud\_mask}],\] while the covariate vector is \[u_t = [\Delta t,\sin(\mathrm{doy}),\cos(\mathrm{doy}),\mathrm{rain},\mathrm{pressure},\mathrm{temp\_mean},\mathrm{temp\_min},\mathrm{temp\_max}] \in \mathbb{R}^{8}.\]

A central design decision is to preserve all time steps while making cloud occlusion explicit. For each frame we:

1. clip the four spectral bands to \([0,1]\) following the EarthNet2021 convention;

2. set cloudy or non-finite spectral pixels to zero;

3. append the binary cloud mask as an additional channel instead of dropping cloudy frames.

This choice preserves the temporal alignment between satellite imagery and weather covariates. It also lets the model distinguish true low reflectance from missing surface information due to clouds. In the final training configuration we keep all frames, so each converted training episode remains 30 steps long.

4.3 Time features and weather aggregation↩︎

EarthNet2021 weather is stored at higher temporal resolution than the image sequence. For every retained transition from frame \(t\) to frame \(t+1\), we compute:

• the day gap \(\Delta t\) between the two image timestamps;

• cyclical day-of-year features \(\sin(2\pi\,\mathrm{doy}/365.25)\) and \(\cos(2\pi\,\mathrm{doy}/365.25)\);

• interval weather statistics from mesodynamic.

Precipitation is aggregated by summation over the transition interval. Pressure and temperature variables are aggregated by averaging over the same interval. In effect, the action/covariate vector describes the exogenous forcing between successive image observations.

4.4 Training HDF5 and normalization↩︎

The final training HDF5 contains 717,120 steps and 23,904 episodes. Every episode has length 30 and every observation uses the native \(128\times128\) spatial resolution. The evaluation splits are converted with the same representation, with the extreme split yielding 60-step episodes (see Table 1).

We estimate per-channel pixel normalization statistics directly from the converted EarthNet2021 training data, using only clear spectral pixels that are not all-zero after masking. This avoids importing image statistics from unrelated natural-image datasets. The cloud mask channel is left binary with mean 0 and standard deviation 1 (see Table 2).

5 Model↩︎

5.1 EarthNet2021 adaptation of LeWorldModel↩︎

We keep the original LeWM training path and adapt only the input interface and dataset pipeline [2]. The instantiated model has 18.0M trainable parameters and consists of:

• a \(1\times1\) learned channel adapter from the 5 EarthNet2021 channels to the 3-channel ViT encoder input expected by the backbone;

• a ViT-tiny encoder with patch size 16 and image size 128;

• an MLP projector that maps the encoder class token to a 192-dimensional latent state;

• an MLP action encoder that maps the 8-dimensional covariate vector to the same latent width;

• a 6-layer conditional autoregressive transformer predictor with 16 heads.

Only the encoder class token is used for the forecasting path of the final model. Patch tokens are exposed in code for probing, but they are not part of the forecasting loss used in the reported checkpoint.

5.2 State and covariate notation↩︎

Let \(x_t\) be the multispectral-cloud observation and let \(u_t\) be the covariate vector. The encoder and action encoder produce \[z_t = E_\theta(x_t) \in \mathbb{R}^{192}, \qquad a_t = A_\phi(u_t) \in \mathbb{R}^{192}.\] For a training window of length 11 with history size 10, the model observes \((z_1,\ldots,z_{10})\) and \((a_1,\ldots,a_{10})\) and predicts the next latent state at every position: \[(\hat{z}_2,\ldots,\hat{z}_{11}) = P_\psi(z_1,\ldots,z_{10}, a_1,\ldots,a_{10}).\] This is important: the final training loss is teacher-forced over the whole context window, not only at the terminal step.

5.3 Training objective↩︎

The final objective is \[\mathcal{L} = \mathcal{L}_{\mathrm{pred}} + \lambda_{\mathrm{sig}}\,\mathcal{R}_{\mathrm{sig}}(z_{1:11}),\] with \[\mathcal{L}_{\mathrm{pred}} = \frac{1}{10D}\sum_{t=1}^{10}\lVert \hat{z}_{t+1}-z_{t+1}\rVert_2^2.\] \(\mathcal{R}_{\mathrm{sig}}\) is the LeWM isotropic-Gaussian regularizer based on random projections.

6 Training Setup↩︎

The reported checkpoint is trained from scratch with AdamW on a single NVIDIA A100 20GB MIG instance. The final run uses batch size 32, learning rate \(3\times10^{-4}\), weight decay \(10^{-3}\), bf16 mixed precision, and 24 epochs. Gradient clipping is set to 1.0 and \(\lambda_{\mathrm{sig}}=0.05\). The used training configuration are available in Table 3.

Figure 1, plotted from the final training log, shows that training is stable and converges smoothly. Both the total loss and the latent prediction loss decrease throughout training, while the validation regularization term remains controlled. The final validation prediction loss is 0.0164 and the final validation SIGReg term is 1.83.

7 Evaluation Protocol↩︎

7.1 Window definitions↩︎

With history size 10 and one future target step, every 30-step episode yields 20 standard next-step evaluation windows. For six-step rollout evaluation, every 30-step episode yields 15 windows, because six additional future steps must be available after the context. The extreme split contains 60-step episodes and therefore yields 50 standard windows and 45 rollout windows per episode.

The full training HDF5 is internally split 90/10 into train and validation windows. This produces 430,273 train windows and 47,807 validation windows for the standard next-step task. The script-based benchmark traverses all available windows on the external evaluation splits, while the notebook diagnostics use smaller fixed subsamples for tractability.

7.2 Locked downstream protocol↩︎

We use one protocol across all reported splits:

1. train LeWM once on the EarthNet2021 training HDF5;

2. fit all linear probes on training windows only;

3. tune minor calibration choices on valsplit only;

4. freeze those probes and thresholds for IID, OOD, and extreme.

This matters because refitting a downstream head separately on each test split would blur the distinction between true generalization and split-specific adaptation.

In the results that follow, the observability benchmark uses the frozen full-bundle outputs, whereas latent forecasting, latent geometry, anomaly ranking, and weather sensitivity come from those fixed notebook diagnostic windows.

The paper reports two evaluation modes: i) the full evaluation frozen-bundle on all available valsplit, IID, OOD, and extreme windows to benchmark observability experiments, as well as ii) smaller fixed budgets for latent forecasting, latent geometry, anomaly ranking, and weather sensitivity. This separation keeps the reported benchmark exhaustive while keeping the exploratory figures tractable.

7.3 Baselines↩︎

Two baselines are used to compare their results with the results of LeWM:

• Persistence. For latent forecasting, predict every future latent by copying the last observed latent state. For the observation-quality tasks, persistence instead uses the last observed quality score, i.e. the minimum of clear fraction and valid fraction in the final context frame.

• LightGBM observability baseline. A frozen tabular baseline fit once on the train-side windows of the converted EarthNet2021 HDF5. It uses only explicit quality histories and future weather/calendar covariates, not learned image latents. This baseline is deliberately strong for binary observability decisions and therefore helps separate what truly requires a latent world model from what can be solved well with structured tabular signals.

7.4 Tasks and their monitoring objective↩︎

We evaluate five task families. The first two are supporting diagnostics: they verify that the latent dynamics model has learned non-trivial temporal structure beyond persistence. The last three are the application-facing tasks that motivate the paper: anomaly ranking for temporal quality assurance, next-step observation usability, and forecasting when usable observations return.

7.4.0.1 1. Next-step latent forecasting.

Objective: determine whether the model learned non-trivial one-step dynamics at all. We compare the predicted next latent against the true next latent and against persistence.

7.4.0.2 2. Six-step autoregressive rollout.

Objective: test whether the latent dynamics remain useful when the model is rolled forward recursively. Future weather covariates from the dataset are provided during rollout, so this is forecasting under known exogenous forcing rather than unconditional image generation.

7.4.0.3 3. Ranking-based anomaly detection.

Objective: flag temporally inconsistent futures for quality control. We synthesize three anomaly types: future-frame shuffle, future-action shuffle, and future visual shift. The anomaly score is the one-step latent prediction error. We report AUROC, average precision, and score ratio.

7.4.0.4 4. Next-step observation quality.

Objective: estimate whether the next acquisition will be usable enough for surface analysis. We fit linear probes from the predicted next latent to next-step clear fraction, next-step cloud fraction, and a binary usable_next target. A step is usable when both clear fraction and valid fraction are at least 0.8.

7.4.0.5 5. First usable future observation.

Objective: estimate when monitoring can resume after cloudy or low-validity observations. Using six-step rollout latents, we predict both whether any usable observation occurs within the horizon and the exact horizon of the first usable step.

We additionally run a weather-sensitivity analysis by perturbing the last observed weather covariates (\(\pm 5\) mm rain, \(\pm 2^{\circ}\)C temperature). Because EarthNet2021 actions are exogenous, we interpret this as sensitivity analysis, not planning.

8 Results↩︎

8.1 Latent forecasting as supporting evidence↩︎

Before interpreting the observability tasks, we verified on fixed sampled diagnostic windows that the latent world model actually learns predictive temporal structure rather than simply copying the last state, and this was the case on every split. Figure 2 shows that LeWM improves over persistence for both next-step prediction and six-step rollout. The effect is strongest on valsplit, degrades on IID, weakens further on OOD, and partially recovers on extreme.

This ordering is informative. The extreme split is harder in cloud frequency and sequence length, but its latent forecasts remain better than OOD on several metrics. The main limitation is therefore not simply longer horizons or more clouds; it is distribution shift in the latent dynamics.

To make the aggregate improvements more concrete, Figure 3 shows the full held-out next-step error distributions on valsplit. The LeWM predictions are concentrated near zero latent error and cosine similarity one, whereas persistence produces a broad high-error tail. Figure 4 shows the detailed horizon-by-horizon rollout curves on the extreme split. Even in this much cloudier and longer-horizon regime, LeWM remains better than persistence at every horizon.

8.2 Latent geometry↩︎

The Gaussian regularizer forces an isotropic regime on the latent dynamics, especially on train-like data and degrades gracefully under shift. On the fixed sampled notebook diagnostics summarized in Table 4, valsplit and IID remain near the intended isotropic regime, with median per-dimension standard deviation close to one and modest off-diagonal covariance. OOD shifts the latent mean away from zero. Extreme compresses the latent variance considerably, which is consistent with its much lower clear-fraction statistics.

Figure 5 complements Table 4 with more diagnostics on the same fixed sampled valsplit subset. The per-dimension means remain centered around zero, most per-dimension standard deviations stay near one, and the covariance matrix remains largely diagonal. This is the qualitative signature we expect from the SIGReg objective when the latent space is healthy.

8.3 Anomaly ranking for temporal QA↩︎

Figure 6 reports ranking-based anomaly metrics from the fixed sampled notebook diagnostics. Frame-shuffle anomalies are easiest to detect and remain strong on all splits, with AUROC between 0.75 and 0.87. Action-shuffle anomalies are weaker but still meaningfully separable. Visual-shift anomalies are strong on valsplit and clearly weaker on OOD and extreme. For monitoring, this means the latent predictor can serve as a temporal-consistency checker. It is not a semantic anomaly detector in the broad sense; rather, it is a detector for futures that do not fit the recent temporal context under the learned dynamics.

8.4 Next-step observation quality↩︎

The next operational question is whether the model can estimate if the next acquisition will be useful. We evaluate this both as continuous quality regression and as binary usable-next classification, and we compare LeWM against both persistence and the frozen LightGBM baseline.

For continuous next-step clear/cloud regression, LeWM is best on valsplit, IID, and extreme, with RMSE 0.177, 0.206, and 0.272 versus 0.247, 0.247, and 0.344 for LightGBM and about 0.60 for persistence. OOD is the only split where LightGBM is better on this task (RMSE 0.331 versus 0.361 for LeWM), which reinforces the conclusion that distribution shift remains the main weakness of the latent model.

For binary usable_next, LeWM and LightGBM are close on train-like splits, but the pattern remains task-dependent. LeWM wins on valsplit (balanced accuracy 0.887 versus 0.875) and extreme (0.844 versus 0.799), is effectively tied on IID (0.878 versus 0.878), and loses on OOD (0.769 versus 0.793). Figure 7 and Table 5 summarize this trade-off. The key point is that LeWM clearly beats persistence everywhere, but the stronger tabular baseline remains competitive whenever explicit quality histories already contain much of the signal.

8.5 Forecasting when usable observations return↩︎

This is the strongest application result of the paper. Given the current context, we ask whether any usable observation will occur within the next six steps and, if so, at what horizon the first usable step will appear (see Table 5 and Figure 8 from the full frozen-bundle benchmark).

For the binary any-usable-within-six question, LightGBM is the strongest baseline on valsplit, IID, and extreme. Its balanced accuracy is 0.916, 0.916, and 0.865 on those splits, compared with 0.873, 0.858, and 0.806 for LeWM. OOD is mixed: LeWM has slightly better balanced accuracy (0.775 versus 0.740), while LightGBM has the better AUROC (0.877 versus 0.862). This task therefore behaves like a strong structured-tabular decision problem.

The exact-timing version is where the latent model becomes more convincing. LeWM beats both baselines on valsplit, IID, and extreme, with exact first-usable-horizon accuracies 0.806, 0.787, and 0.705, compared with 0.756, 0.762, and 0.633 for LightGBM and 0.348, 0.341, and 0.120 for persistence. It also substantially reduces mean absolute timing error: 0.372, 0.415, and 0.627 steps for LeWM versus 0.572, 0.563, and 0.885 for LightGBM and 2.45, 2.47, and 3.44 for persistence. OOD is again the one split where LightGBM is better on timing (exact accuracy 0.636 versus 0.602).

The extreme split is especially informative. It has the lowest clear fraction and the broadest recovery distribution, yet LeWM still recovers a large timing advantage over persistence and a meaningful margin over LightGBM. Figure 9 from the same full frozen-bundle evaluation shows why this split is harder: its recovery targets are shifted toward later horizons and more no-recovery cases within the six-step window. This is the clearest evidence that the latent model adds value when the target is not just “will recovery happen?” but “when will recovery happen exactly?”

8.6 Weather sensitivity↩︎

On fixed sampled notebook windows, perturbing the immediate weather covariates yields coherent but small changes in the predicted next-step usability. Across valsplit, IID, and extreme, drier conditions slightly increase predicted clear fraction and usable score, while wetter conditions decrease them. Temperature perturbations produce smaller effects. OOD sensitivity is near zero. This is a reassuring sanity check: the model responds in the expected direction to exogenous weather forcing, but the effect size is modest and should not be overstated.

9 Discussion↩︎

9.0.0.1 Latent forecasting matters, but it is not the main claim.

The observability results only matter if the model actually learns non-trivial temporal structure. On that point the sampled latent diagnostics are clear: LeWM beats persistence on next-step latent prediction and on six-step rollout across all reported splits. That is the necessary foundation for the cloud-aware observability application but it is not the application itself.

9.0.0.2 LeWM and LightGBM solve different parts of the problem well.

The comparison with LightGBM is useful because it shows where the latent model earns its keep. LightGBM is hard to beat on the simpler binary question, “will any usable observation appear somewhere in the next six steps?” LeWM, in contrast, is better on continuous clear/cloud regression and on exact first-usable timing on valsplit, IID, and extreme. That suggests the latent representation is most useful when the downstream task depends on a richer temporal state than explicit quality histories and weather covariates can provide on their own.

9.0.0.3 OOD is still the weak point.

OOD is the one split where LeWM falls behind more consistently. The geometry diagnostics suggest that this is not just a readout problem. The latent distribution itself shifts away from the train-like regime. For this reason, the cleanest claim is not broad robustness across all deployment conditions; it is strong performance on train-like and difficult-but-related conditions, with a clear drop under stronger distribution shift.

9.0.0.4 The extreme split is difficult, but it is also informative.

Extreme is much cloudier and has longer episodes, yet LeWM still holds a clear advantage over persistence and remains stronger than LightGBM on exact recovery timing. That matters because it shows the model is not only exploiting easy near-IID structure. It can still track recovery dynamics in a substantially harsher monitoring regime.

9.0.0.5 This is a forecasting result, not a planning result.

EarthNet2021 does not provide controllable actions. The covariates are weather drivers. Our perturbation experiments are therefore sensitivity analyses: they show how the model reacts when the forcing changes, but they do not demonstrate policy optimization or counterfactual control.

9.0.0.6 The current model is global, not spatial.

LeWM predicts a global future latent derived from the encoder class token. It does not produce future patch tokens or future image maps. That boundary matters. The contribution of this paper is a cloud-aware temporal monitoring model, not a spatial video forecasting system.

9.0.0.7 Availability of future meteorological covariates.

All reported experiments condition on the future meteorological covariates provided by EarthNet2021. LeWM does not forecast these covariates; results are conditional observability forecasts under known exogenous drivers.

10 Conclusion↩︎

We presented a full EarthNet2021 application of LeWorldModel built around a change in problem framing: from surface forecasting to observability forecasting. The contribution is not a new world-model architecture. It is a careful adaptation, training, and evaluation study that asks a practical EO question: will the next observation be usable, and if not, when is a usable view likely to return?

The experiments support a specific conclusion. In sampled latent diagnostics, LeWorldModel forecasts latent futures far better than persistence on every reported split, which confirms that it learns useful temporal structure. On the full frozen-bundle observability benchmark, those latent forecasts support downstream tasks that matter for cloud-aware monitoring. Compared with persistence, LeWorldModel gives much better next-step usability estimates and much better timing estimates for the return of a usable observation. Compared with a strong frozen LightGBM baseline, it is better on continuous observation-quality regression and on exact recovery timing on valsplit, IID, and extreme, while LightGBM remains stronger on the simpler binary any-usable-within-six task and on OOD more broadly.

Future work should target robustness across geography, season and weather regime. Additionally, the current model gives point predictions which may not be sufficient for operational EO monitoring. Calibrated uncertainty on usable next and first-usable timing would be more useful than a single deterministic forecast.

Appendix A follows the same order as the results section and collects the full supporting tables and diagnostic figures behind each main-text claim.

11 Supplementary Results↩︎

This appendix follows the same order as the results section in the main paper. Section 11.1 records the checkpoint used throughout the paper and complements Figure 1. Section 11.2 expands the latent-forecasting results discussed around Figures 2, 3, 4, and 5, together with Table 4. Section 11.3 supports Figure 6. Section 11.4 expands the next-step observation-quality benchmark summarized in Figure 7 and Table 5. Section 11.5 expands the recovery tasks summarized in Figures 8 and 9 and in Table 5. Section 11.6 gives the full weather-sensitivity results behind the short main-text discussion.

We keep the same split order throughout: valsplit, IID, OOD, and extreme.

11.1 Training checkpoint used in all experiments↩︎

All downstream evaluations in the paper use the same 24-epoch LeWM checkpoint. Table 6 records the final training and validation metrics for that run and complements the learning-curve summary in Figure 1.

11.2 Latent forecasting details↩︎

This subsection expands the latent-dynamics results that motivate the use of LeWM as a forecasting model rather than only as a feature extractor. It collects split-wise next-step metrics, rollout metrics broken down by forecast horizon, and latent-geometry figures. All results in this subsection come from fixed sampled diagnostic windows in the executed split notebooks, not from the frozen full-bundle benchmark.

11.2.1 Next-step latent forecasting↩︎

Table 7 gives the split-wise next-step metrics behind the main-text latent-forecasting discussion. Figure 10 shows the corresponding distributions for LeWM and persistence.

11.2.2 Six-step rollout by forecast horizon↩︎

Tables 8 to 11 unpack the six-step rollout into per-horizon metrics on the same fixed sampled notebook windows. Figure 11 summarizes the same comparison visually across all splits.

11.2.3 Latent-geometry figures↩︎

Figure 12 complements the scalar geometry summary reported in Table 4. It shows what the aggregate numbers look like in distribution form, especially the widening gap between train-like splits and OOD or extreme conditions.

11.3 Temporal-QA anomaly ranking↩︎

Table 12 expands Figure 6. It reports the anomaly-separation metrics for each synthetic anomaly type and split on the same fixed sampled notebook windows.

11.4 Next-step observation quality↩︎

This subsection expands the benchmark summarized in Figure 7 and Table 5. The first table covers continuous clear and cloud regression. The second covers binary usable_next classification.

11.5 Recovery of usable observations within the rollout horizon↩︎

This subsection expands the recovery-focused results summarized in Figures 8 and 9 and in Table 5. We first show the target support by split, then the binary any-usable benchmark, and finally the exact timing benchmark and confusion matrices for the first usable future observation.

11.5.1 Target support by split↩︎

Table 15 gives the raw class counts behind the normalized support plot in Figure 9 and the normalized confusion matrices in Figure 13. It makes the recovery problem concrete by showing how much probability mass sits on immediate recovery, delayed recovery, and no recovery within the six-step horizon.

11.5.2 Any-usable-within-six classification↩︎

Table 16 gives the full frozen-bundle metrics behind the top row of Figure 8 and the corresponding columns of Table 5. It shows directly where LeWM, LightGBM, and persistence separate on the simpler binary recovery question.

11.5.3 Timing of the first usable future observation↩︎

Table 17 gives the exact-timing metrics behind the bottom row of Figure 8 and the timing columns of Table 5. This is the task where the latent model is most consistently useful on train-like and difficult-but-related splits.

Figure 13 complements Table 17 by showing where the timing errors occur after row normalization. Table 15 should be read alongside it, because the normalized display improves comparability across splits and horizons while the table preserves the raw support counts. The main pattern is that persistence collapses toward very early recovery or no recovery, while LeWM preserves much more of the true horizon structure.

11.6 Weather sensitivity details↩︎

This subsection expands the short weather-sensitivity discussion in the main text. All results here come from the fixed sampled notebook windows rather than the frozen full-bundle benchmark. The table gives the split-wise means and deltas, while Figure 14 shows the same effects graphically.

References↩︎