Transformer-Fused Hybrid Descriptors for Remote Sensing Scene Classification, Integrating Texture-Morphology Features with EfficientNetV2 Semantic Embeddings on MLRSNet
TL;DR
A novel transformer-fused hybrid feature learning model is developed to effectively combine handcrafted and deep features for accurate Land Use/Land Cover (LULC) scene classification, effective for improving inter-class discriminability for visually similar LULC classes.
Abstract
Classification in remote sensing images is a difficult problem, given high intra-class variability, inter-class similarity, and spatial complexity. In this paper, a novel transformer-fused hybrid feature learning model is developed to effectively combine handcrafted and deep features for accurate Land Use/Land Cover (LULC) scene classification. In this model, texture features are represented using Local Binary Patterns (LBPs) and Grey-Level Co-occurrence Matrix (GLCM) features, while morphological region features provide shape information for scene characterization. Meanwhile, deep semantic features are also learned from EfficientNetV2-B0. To effectively fuse these features, a novel multi-head self-attention fusion mechanism is developed to learn explicit feature dependencies between texture, morphological, and semantic features for a compact yet discriminative feature representation. Experimental evaluation is conducted using the complete MLRSNet dataset comprising all 46 scene classes and 46,000 images, with 1,000 images considered from each class to ensure a balanced and comprehensive experimental setting. The proposed framework achieves an average five-fold accuracy of 99.98%, demonstrating high learning consistency across the complete set of diverse and visually similar remote sensing scenes. Comparative evaluation with established pretrained CNN and transformer-based models under the same experimental setting, together with component-wise ablation analysis, further demonstrates the contribution of the handcrafted descriptors, EfficientNetV2 semantic embeddings, and transformer-guided fusion mechanism. This fusion approach is effective for improving inter-class discriminability for visually similar LULC classes, which is a powerful tool for large-scale LULC mapping, urban growth analysis, environmental surveillance, etc., from remote sensing images.