Toward Lightweight Fine-Tuning of YOLO via Guided Optimization for Cross-Domain Remote Sensing Object Detection

Abstract

Deep learning has advanced remote sensing object detection, yet prevailing detectors are tailored to individual datasets, generalize poorly across domains, and rely on large labeled corpora and heavy computation ill-suited to on-device platforms. This paper presents RMoE-YOLO, a lightweight fine-tuning framework that adapts a pretrained YOLOv8n detector to new remote sensing domains under tight label and compute budgets. Its core component is an object-oriented mixture-of-experts (OMoE) module placed before the detection head, where lightweight expert subnetworks learn object-category-conditioned representations and an image-level gating network selects the most relevant experts for each input. A saliency-aware sparsity term and a phased hierarchical fine-tuning schedule guide the adaptation, balancing retention of pretrained knowledge against adaptation to the target domain. On NWPU-VHR-10, RSOD, and SIMD, together with cross-domain transfer, RMoE-YOLO improves low-label cross-domain detection while remaining efficient. Under a controlled multi-seed protocol on a 5% label single-class adaptation setting, its multi-expert configuration significantly outperforms the YOLOv8n baseline by 1.7 mAP 50 points (p < 0.05), while its single-expert configuration matches the baseline accuracy at only 19.5 active GFLOPs, below YOLOv8n, and a comparable frame rate. RMoE-YOLO offers a practical route to accurate, generalizable, and efficient detection in low-resource remote sensing.