Unified UAV Open-Vocabulary Semantic Segmentation: Benchmark Construction and LLM-Guided Text–Visual Enhancement
TL;DR
This paper constructs a unified UAV OVSS benchmark by reorganizing multiple UAV segmentation datasets into cross-dataset transfer settings with explicit category harmonization and seen/unseen vocabulary analysis and proposes UAV-OVSeg, a Cost Aggregation-style dense matching framework enhanced in two complementary directions.
Abstract
Open-vocabulary semantic segmentation (OVSS) of unmanned aerial vehicle (UAV) imagery aims to recognize arbitrary text-specified categories in aerial scenes, but existing OVSS models often suffer from UAV-domain shifts. To provide a reproducible testbed, this paper constructs a unified UAV OVSS benchmark by reorganizing multiple UAV segmentation datasets into cross-dataset transfer settings with explicit category harmonization and seen/unseen vocabulary analysis. Based on this benchmark, we propose UAV-OVSeg, a Cost Aggregation-style dense matching framework enhanced in two complementary directions: an LLM-guided Category Expansion Module that converts raw category names into structured UAV-aware descriptions, and a DINO-enhanced Geometric Feature Fusion Module that injects local structure into dense visual–text matching. Under SynDrone training, UAV-OVSeg achieves 65.5% mean intersection over union (mIoU) and 78.0% mean accuracy (mACC), improving upon the CAT-Seg baseline by 2.6 mIoU and 3.2 mACC. Under Aeroscapes training, it achieves 64.5% mIoU and 76.8% mACC, improving upon CAT-Seg by 2.6 mIoU and 3.1 mACC. Additional analyses of LLM variants, object scales, prompt sensitivity, boundary quality, and computational cost further verify the effectiveness and reproducibility of the proposed UAV-oriented text–visual enhancement strategy.