RS-ReXSeg: Rejection-Aware Explanatory Language-Guided Segmentation for Open-Query Remote Sensing Images
Abstract
Language-guided remote sensing image segmentation enables pixel-level localization from natural-language queries, but most existing methods implicitly assume that the queried target is present. In open-query interpretation, absent targets and visually similar land-cover objects may cause models to generate incorrect masks, posing risks to disaster assessment, infrastructure monitoring, and urban facility search. To address this issue, we propose RS-ReXSeg, a rejection-aware multimodal large language model framework that jointly performs target segmentation, target-absence rejection, and query-centric explanation. RS-ReXSeg adopts explicit token-level decisions, where [SEG] activates mask decoding for target-present queries and [REJ] suppresses mask generation for target-absent queries. To learn reliable rejection from conventional segmentation annotations, we construct positive, easy-negative, and hard-negative queries, and introduce visual-dominant multivisual–semantic hard-negative query construction with bias-resistant multimode training to improve discrimination under remote-sensing-specific visual confusion. We further construct RS-Rej, a benchmark for jointly evaluating segmentation and rejection. On RS-Rej, RS-ReXSeg achieves 55.6% gIoU and 61.9% cIoU, while achieving 94.1% hard-negative rejection accuracy, compared with 78.5% for GSVA fine-tuned on RS-Rej with random negatives, and maintaining a 99.3% positive-query acceptance rate. On target-present benchmarks, it achieves 73.2% gIoU on EarthReason and 79.58% oIoU on RRSIS-D. Additional evaluation on 75 manually written, scenario-driven free-form queries further assesses robustness to practical query variations. These results indicate that RS-ReXSeg improves target-absence handling while retaining effective target-present segmentation performance.