Skip to content
Open access

UniRS-Seg: Shared Sparse–Dense Query Decoding for Joint Semantic and Referring Remote Sensing Segmentation

Oct 2026 · Remote Sensing · 0 citations

Abstract

Remote sensing semantic and referring segmentation share mask geometry but differ in whether a category vocabulary or a natural-language expression selects the output. UniRS-Seg bridges these interfaces through a shared sparse–dense query compiler and a hierarchical mask decoder, jointly trained on independently annotated datasets. Class names and referring expressions are projected from SigLIP 2 into the same decoder space; dense image–query activations supply spatial support, and sparse query tokens specify the requested region. An adapted SAM ViT-B pathway and parallel convolutional features recover local geometry, while learnable soft mask cross-attention refines both tasks and class-balanced mask-guided self-attention supports semantic parsing. UniRS-Seg achieves 57.30%, 85.31%, and 88.24% mIoU on LoveDA, Vaihingen, and Potsdam, and 78.18% and 70.88% gIoU on RefSegRS and RRSIS-D. These scores exceed the strongest listed baseline means by 0.68, 0.99, 1.18, 0.79, and 1.71 percentage points, respectively. Against separately trained semantic-only and referring-only versions of the shared architecture, joint training improves the four evaluated datasets by 0.45, 0.89, 1.40, and 0.97 points. An additional supervised RISBench evaluation achieves 72.91% gIoU and 74.85% cIoU, with the highest gIoU among the listed methods. On RefSegRS, adding learnable mask cross-attention to combined sparse and dense prompts increases gIoU by 0.86 points. The results support supervised dual-task transfer through a common query-conditioned decoder; the comparison uses published baseline results and does not establish unseen-domain or zero-shot generalization.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.