This work proposes a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation that maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.
Abstract
Robust and reliable perception is essential for autonomous robots operating in real-world environments, particularly in long-term missions where environmental conditions may change significantly over time. Although recent advances in Visual Foundation Models (VFMs) have improved open-vocabulary semantic segmentation, these models can still suffer from domain shift, which can significantly degrade performance if they are not adapted to the current environment. Training-free domain adaptation is a relevant paradigm for adaptation, consisting of adjusting the model online using lightweight adapters. Recent approaches apply this on a per-frame basis, which is impractical for deployments on resource-constrained robotic hardware. To tackle this, we propose a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation. Our method leverages temporal coherence across consecutive frames by monitoring and combining complementary aspects of domain shift (visual change, adapter mismatch, and semantic drift) to trigger adaptation only when needed. We validate our approach on a benchmark including indoor and outdoor environments and using real robotic data. We demonstrate that our approach maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.
SPeaR (Steering Primitive for Realigning Representations) is introduced, which inserts lightweight learnable modules at stage boundaries and optimizes them directly from the test stream, requiring neither source data nor supervised warm-up.
Muhammad Sudipto Siam Dip, Ali Etemad· arXiv.org· 0 citations
Test-time adaptation (TTA) for vision-language models (VLMs) is usually studied in a fully unsupervised regime, where adaptation relies on pseudo-labels, confidence scores, or entropy computed by the model itself. This is a severe restriction on hard shifted samples: the model is asked to repair its own uncertain predi...
Jihwan Bang, Sumyeong Ahn, Hwanjun Song et al.· 0 citations
Tests on five architecturally diverse VLAs suggest that adaptation cost in VLAs is structured enough to measure before fine-tuning begins, and introduces a pipeline that observes, diagnoses, allocates, and adapts.
Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev et al.· arXiv.org· 0 citations
This work proposes FETA, a Feedback-Efficient Test-Time Adaptation method that improves both feedback efficiency and optimization reliability, and introduces a hierarchical sample selection strategy to localize informative samples.
Zi-Xu Wang, Hua-Quan Zhang, Shou-Kai Xu et al.· 0 citations
Open-Vocabulary Audio-Visual Event Localization (OV-AVEL) requires models to localize audio-visual events while predicting explicit categories for events unseen during training. Current baselines adopt a text-centered pairwise modality decision strategy. However, it lacks unified modeling of multimodal semantic consist...
Bo-Han Su, Sheng-Lan Liu, Chi Liu et al.· Proceedings of the 34th ACM...· 0 citations
Three-dimensional feature extraction and multi-task perception are fundamental components of modern autonomous driving systems. Although large image encoders, high-resolution inputs, and long temporal contexts can substantially improve representation quality and overall performance, jointly leveraging these strategies...
Zhong-Yu Xia, Zhi-Wei Lin, Yong-Tao Wang et al.· arXiv.org· 6 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.