Foundation Models Under Class Imbalance in SAR Ship Classification: A Fusion-Based Evaluation

Abstract

Remote sensing foundation models are increasingly used for downstream tasks, but their behavior under extreme class imbalance is not well characterized. We present an empirical study of pretrained encoders for SAR ship classification on two public datasets, FUSARShip and OpenSARShip, which exhibit severe label skew. We benchmark seven encoders spanning optical and multispectral pretraining, mixed SAR and optical pretraining, SAR-specific pretraining, and a general-purpose self-supervised vision backbone, using a frozen-feature evaluation pipeline. We show that, compared to accuracy and weighted F1-score, macro F1-score is a more suitable metric to assess the performance under imbalance. We then evaluate four fusion strategies, including early fusion, stacking, and two late-fusion variants with informed and grid-searched weighting, as well as lite configurations that exclude the two weakest encoders. Latefusion variants provide the best macro F1 on both datasets, improving over the strongest single model by +5.32 macro-F1 points on FUSARShip and +4.49 points on OpenSARShip, while maintaining strong weighted-F1 and accuracy. Our results highlight the importance of macro-F1 based evaluation and show that simple, validation-driven fusion can improve performance in imbalanced SAR ship classification.