DCRM-ViT is proposed, a domain-conditioned residual modulation framework for Vision Transformers that preserves general-vision knowledge while adapting to diverse medical and natural domains and achieves strong cross-domain performance with low overhead.
Blind Image Quality Assessment (BIQA) models trained on one distortion distribution often degrade when exposed to new ones, making sequential adaptation without forgetting a fundamental challenge. While continual learning offers a natural solution, existing methods typically retrain the entire backbone per task, limiting scalability and parameter efficiency. We propose ContEditIQA, a parameter-efficient framework for continual BIQA that selectively edits a pre-trained Vision Transformer (ViT) rather than retraining it. Following a locate-then-edit strategy, a lightweight attention-guided hypernetwork identifies distortion-sensitive Feed-Forward Network (FFN) parameters for each incoming task and restricts updates to those regions, while attention layers remain frozen to preserve globally shared representations. This targeted editing enables robust sequential adaptation without model expansion or memory replay. Experiments across six BIQA benchmarks demonstrate superior knowledge retention and cross-dataset generalization while modifying fewer than 30% of backbone parameters, establishing selective model editing as an effective and scalable paradigm for continual BIQA.
Satish Maurya, Parimala Kancharla· International Conference on...· 0 citations
Vision Transformers pretrained via self-supervised learning have demonstrated strong representation capability in natural image analysis and are increasingly adopted in medical imaging tasks. However, when transferred to medical domains, pretrained encoders often exhibit limited adaptability to ambiguous anatomical boundaries and low-contrast structures as contextual dependencies learned from natural images may not adequately capture uncertainty characteristics inherent in medical data. In this work, we propose a layer-wise uncertainty-guided feature perturbation and refinement framework that operates directly in representation space and can be seamlessly integrated into pretrained transformer encoders. The proposed method explicitly estimates spatial uncertainty from encoder features and performs controlled semantic diffusion in feature space, enabling selective refinement of ambiguous regions while maintaining stable representations elsewhere. The enhanced representations are integrated through residual modulation, enabling progressive adaptation without disrupting pretrained dynamics. The proposed framework is fully plug-and-play and can be inserted into each transformer layer without modifying the original architecture. Extensive experiments on multiple medical image analysis tasks demonstrate consistent performance improvements over strong transformer baselines, validating the effectiveness of uncertainty-guided representation diffusion for medical visual understanding.
Lei Zhang· Journal of Electronic Imagin...· 0 citations
Pre-trained Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for transformer self-attention layers parameterized by 2D matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D matrix disrupts the inherent spatial topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic segmentation, and generative benchmarks.
Sojung An, Junha Lee, Sujeong You et al.· 0 citations
Efficient Unsupervised Domain Adaptation (EUDA) is proposed, a parameter-efficient framework that leverages a frozen DINOv2 backbone as a feature extractor and updates only a lightweight bottleneck and classification head to promote both discriminative learning and cross-domain alignment.
Ali Abedi, Q. M. J. Wu, Ning Zhang et al.· International Journal of Mac...· 9 citations
Vision-language models (VLMs) have significantly advanced open-vocabulary image understanding by learning aligned representations from large-scale image-text datasets. Despite their zero-shot generalization capabilities, adapting these foundation models to specific application domains remains challenging. Full fine-tuning is often infeasible due to computational costs and the risk of overfitting when labeled data are limited. Parameter-efficient fine-tuning (PEFT) approaches promise to address this issue by updating only a small set of parameters while keeping the pre-trained encoders largely frozen. However, existing PEFT strategies often exhibit insufficient spatial awareness, rendering them suboptimal for medical imaging, where subtle visual differences can lead to distinct clinical diagnoses. To address this challenge, in this study we propose a novel PEFT technique that leverages natural-image pretrained DINOv3’s attention maps to enforce spatial alignment.
M. A. Aydın, Furkan Genc¸, Efe ¨Ozdilek et al.· Signal Processing and Commun...· 0 citations
Test-Time Domain Adaptation (TTDA) aims to adapt Deep Neural Networks to distribution shifts using only streaming, unlabeled test data in real time. Current methods for semantic segmentation tasks suffer from critical limitations. Entropy minimization techniques require costly backpropagation, risking catastrophic forgetting and producing noisy segmentation boundaries. Memory-bank methods, while backpropagation-free, exhibit slow adaptation, requiring numerous samples to converge and struggle to handle continuous domain shifts. We introduce TestMate, a novel, real-time, and backpropagation-free TTDA framework that overcomes these issues. TestMate leverages generalization capability of a lightweight Visual Foundation Model to guide the adaptation. We use a zero-shot instance segmentation YOLOv8-seg based model to generate unlabeled mask proposals for objects and their parts at multiple scales in real time. These proposals are fused with the primary model via a heuristic, size-ordered competitive scheme, where small, high-confidence regions dominate and refine predictions in surrounding larger, less certain areas. This paremeter-free mechanism enables immediate adaptation from the first frame, inherently avoids catastrophic forgetting and effectively preserves fine object details and boundaries, even for small objects. TestMate can be used as a standalone, efficient refinement module or seamlessly integrated into existing TTDA methods to significantly boost their performance. We demonstrate state-of-the-art results across two benchmark datasets, proving TestMate's effectiveness in three distinct adaptation tasks: TTDA, Source-Free Domain Adaptation (SFDA), and online-TTDA. Code is available.
Dimitrios Fotiou, V. Mygdalis, Ioannis Pitas· 0 citations
