Temporal transformers for biomass estimation

Abstract

Large-scale forest biomass estimation is critical for carbon accounting and climate change mitigation, yet remains challenging due to the complex temporal dynamics of vegetation phenology and the difficulty of fusing multimodal satellite observations. Recent deep learning approaches based on convolutional neural networks show promise but often fail to effectively capture long-range temporal dependencies in multi-temporal satellite time series, relying instead on simple averaging or recurrent architectures that struggle with seasonal patterns and missing data. In this work, we propose a hybrid CNN-Transformer framework for accurate above-ground biomass estimation from multi-temporal Sentinel-1 and Sentinel-2 imagery. Our method introduces a Transformer-based temporal aggregation module with learnable positional encodings and multi-head self-attention, enabling the model to adaptively weight monthly observations and capture phenological cycles without explicit seasonal supervision. To improve spatial feature representation, we design an enhanced decoder architecture combining Squeeze-and-Excitation blocks for channel-wise attention, Feature Pyramid Networks for multi-scale fusion, and spatial attention gates that focus on biomass-rich forest regions. Extensive experiments on the BioMassters benchmark dataset demonstrate that our approach substantially outperforms CNN-based baselines with simple temporal pooling, achieving remarkable performance across diverse global biomes while maintaining computational efficiency for operational deployment.