Land Cover Classification of Multi-Source Airborne Data using Conventional and Deep-Learning-Based Unsupervised Domain Adaptation

Abstract

Abstract. For an increasing number of applications, land cover maps can be generated from remote sensing imagery using conventional and deep-learning-based semantic segmentation models. Relying on a large pool of training data, the networks struggle with the spatial-temporal-spectral heterogeneity in the complex and diverse remote sensing imageries, leading to a significant number of errors in the model predictions. This paper presents a workflow comprising domain adaptation and classification. In particular, we analyze two domain adaptation techniques: First, a conventional histogram-matching method, which has turned out to be a surprisingly fast and reliable tool in a previous study, and second, a CycleGAN, which we applied both in its standard form and with the perceptual loss, thereby penalizing style inconsistencies on deeper layers. By applying the workflow to three remote sensing datasets and six directions of domain adaptation, we show that there is “no free lunch” in the sense that all domain adaptation methods have their advantages. Depending on the dataset, classification method, and especially on the availability of 3D data, the performance gap can be reduced to up to 1.5% of the mean F1 score, demonstrating the soundness of the proposed method.