This work offers a scalable pathway for next-generation Earth system models to learn directly from sparse, incomplete real-world observations and derives an optimization framework based on the expectation-maximization (EM) algorithm that enable learning directly from sparse and noisy observations.
Abstract
Data-driven methods have revolutionized ocean modeling, yet current approaches rely heavily on complete reanalysis datasets, imposing computational constraints and limiting model performance to that of the training data. Here, we present a generative state-space model and an optimization framework that enable learning directly from sparse and noisy observations. The model is essentially a hidden Markov model with a continuous state space, where oceanic physical quantities are treated as hidden states and measurements as observations, enabling a unified representation of ocean fields and observational data. Both the initial-state and state-transition modules are implemented as neural networks to capture the complexity and temporal evolution of ocean states, while the emission module is formulated as a masked Gaussian distribution. To train the model from sparse observations, we derive an optimization framework based on the expectation-maximization (EM) algorithm. The framework alternately reconstructs high-fidelity ocean fields via Langevin dynamics and optimizes deep neural networks to capture temporal evolution. Theoretical analysis shows that the framework maximizes the likelihood of observations under the generative model. For efficiency, we assume that ocean-state evolution follows a stationary, ergodic, and Markovian stochastic process and adopt only length-two state sequences during optimization. Experiments on CMIP6 simulation data and FY-3D satellite data demonstrate high-fidelity reconstruction and accurate prediction, showing that sparse observations can directly improve the model's representation of ocean-state dynamics. This work offers a scalable pathway for next-generation Earth system models to learn directly from sparse, incomplete real-world observations.
OceanBench is a benchmark designed to evaluate and accelerate global short-range data-driven ocean forecasting, constructed from a curated dataset comprising first-guess trajectories, nowcasts, and atmospheric forcings from operational physical ocean models, typically unavailable in public datasets due to assimilation cycles.
Anass El, Quentin Gaudel, Juan Emmanuel Johnson et al.· Advances in Neural Informati...· 7 citations· ⚡2
4DVarGen is proposed, a 4DVar-inspired generative framework for reconstructing sea surface variable fields at eddy-resolving scales from sparse remote-sensing observations that establishes a mathematical equivalence between 4DVar and an observation-guided denoising process.
Junpeng Huang, Wuxin Wang, Xiaoyong Li et al.· 0 citations
Abstract. Reconstructing spatial climate variability from proxy records requires forward models “emulators” that capture the dynamical structure of the climate system while remaining computationally efficient. Traditional emulators based on Empirical Orthogonal Functions (EOFs) and Linear Inverse Models (LIMs) face inherent limitations due to linearity and variance-based dimensionality reduction. Here we develop and evaluate a hierarchy of CMIP-class climate model emulators, for annual surface air temperature field emulation, that integrate autoencoder-based dimensionality reduction with nonlinear prediction architectures, including Reservoir Computing (RC) and Recurrent Neural Networks (RNNs). Using a comprehensive experimental protocol applied to the IPSL-CM6A-LR model and 52 CMIP6 models, we show that the combination of a prediction-oriented autoencoder (AE) latent representation with RC dynamics retaining memory and nonlinear state evolution (AERCn) yields, the most robust configuration when training data are plentiful. This improves the representation of El Niño Southern Oscillation and Atlantic Multidecadal Variability, while preserving spatial reconstruction quality and robustness across distinct CMIP6 model structures. When training data are scarce, a multimodel pre-trained AERNN provides a data-efficient and competitive alternative. These properties make the proposed architectures particularly well suited for integration into Particle Filters and Ensemble Kalman Filter PDA frameworks. Our results highlight the importance of predictability-oriented dimensionality reduction and nonlinear dynamical memory for emulator design. They provide a scalable proof of concept toward multivariate climate-field emulation for improved reconstructions of climate variability over the Common Era.
Auguste Gaudin, M. Khodri· Geoscientific Model Developm...· 0 citations
Regional high-resolution ocean environmental forecasting combines spatial numerical modeling with temporal prediction, and is essential for monitoring the ecological security of specific ocean regions. In recent years, deep learning methods are generally more computationally efficient than traditional numerical models and enable fast, accurate forecasting. However, as data resolution increases, the training and computational costs of existing approaches increase substantially. To address this issue, we introduce Slow-OCast, a transfer-learning based model designed for high-resolution ocean environmental forecasting. Specifically, Slow-OCast incorporates the slow-varying motion characteristics of the ocean and comprises two insightful modules. The Fluid Motion Separator that injects low-frequency background dynamics into the fine-tuning process of a foundation model, functioning as a "magnifier" to encode physical priors of ocean dynamics. The Hydrokinetic Energy Path Integrator that provides an implicit representation of flow-field evolution, serving as a "compass" to guide accurate change prediction. We evaluate Slow-OCast on two high-resolution Mediterranean datasets, and results demonstrate Slow-OCast consistently outperforms all baseline methods across forecasting tasks with different lead times.
Qixiu Li, Xiang Zhu, Xiaoyong Li et al.· Proceedings of the 32nd ACM...· 0 citations
We propose a variational autoencoder framework to directly assess uncertainties in subsurface models produced by single‐ and multiparameter full waveform inversion (FWI). The new method does not require pretraining on labeled data, thus it significantly reduces computational cost and storage requirements. The framework takes the seismic shot gathers as input and returns a set of possible velocity models that fit the input data. The network has three key components: (a) an encoder that maps the input seismic shot gathers to feature distributions in the latent space, (b) a latent vector sampled directly from feature distributions, and (c) a decoder that maps the sampled latent vector to the velocity model space. We incorporated Deep Image Prior principles, leveraging convolutional layers and LeakyReLU activations to regularize the inversion and improve reconstruction. At each epoch, the reconstructed velocity model is passed to a finite difference partial differential equation solver for forward modeling. We then calculate the data misfit between the input seismic data and the simulated data. Additionally, we compute the FWI gradient by cross‐correlating the adjoint and forward wavefield; the FWI gradient is then back‐propagated to guide the update of the neural networks weights and biases. The injection of physics‐based gradients further constrains the inversion and enhancing convergence. We tested the framework on synthetic acoustic and elastic seismic data sets, inverting models of varying complexity and quantifying their associated uncertainties. The results show accurate mean velocity models with meaningful uncertainty estimates, highlighting the potential of the proposed method for practical FWI applications.
A. Elmeliegy, Mrinal K. Sen, A. Dhara et al.· Journal of Geophysical Resea...· 0 citations
Accurate estimation of background error (i.e., forecast error) distribution is critical for effective data assimilation (DA) in numerical weather prediction (NWP). In state-of-the-art operational DA systems, it is common to account for the temporal evolution of background errors by employing hybrid methods, which blend a static climatological covariance with a flow-dependent ensemble-derived component. While effective to some extent, these methods typically assume Gaussian-distributed errors and rely heavily on hand-crafted covariance structures and domain expertise, limiting their ability to capture the complex, non-Gaussian nature of atmospheric dynamics. In this work, we propose LoRA-EnVar, a novel hybrid ensemble variational DA algorithm that integrates low-rank adaptation (LoRA) into a deep generative modeling framework. We first learn a climatological background error distribution using a variational autoencoder (VAE) trained on historical data. To incorporate flow-dependent uncertainty, we introduce LoRA modules that efficiently adapt the learned distribution in response to flow-dependent ensemble perturbations. Our approach supports online finetuning, enabling dynamic updates of the background error distribution without catastrophic forgetting. We validate LoRA-EnVar in high-resolution assimilation settings using the FengWu forecast model and simulated observations from ERA5 reanalysis. Experimental results show that LoRA-EnVar significantly improves assimilation accuracy over models assuming static background error distribution and achieves comparable or better performance than full finetuning while reducing the number of trainable parameters by three orders of magnitude. This demonstrates the potential of parameter-efficient adaptation for scalable, non-Gaussian DA in operational meteorology
Yi Xiao, Hang Fan, Kun Chen et al.· Neural Information Processin...· 3 citations