SFASA is proposed, an approach that leverages multimodal contrastive learning to recover invariant semantic factors and dynamically routes only semantically salient tokens into lightweight low-rank adapters by reusing frozen VLM projection heads as zero-parameter modality mappers.
Ze-Xiang Li, Wen-Xu Wang, Jia-Wen He et al.· Proceedings of the 34th ACM...· 0 citations
Large pre-trained Vision-Language Models (VLMs) excel in zero-shot generalization but face challenges in efficient few-shot adaptation. Existing parameter-efficient methods apply uniform updates across all tokens, overlooking the inherent sparsity of cross-modal interactions and exacerbating overfitting. We propose SFA...
Ze-Xiang Li, Wen-Xu Wang, Jia-Wen He et al.· Proceedings of the 34th ACM...· 0 citations
A Causal-Invariant representation learning framework with Global Redundancy information (CIGR) for OSMTDA, and a graph-based multi-domain causal invariance learning (MCIL) module that generates virtual unknown features from redundant factors, enabling causally invariant representations for known and unknown classes.
Wen-Xu Wang, Ru-Xin Zhou, Zhenbo Li et al.· IEEE Transactions on Pattern...· 0 citations
Behavioral benchmarking reveals \emph{what} biases exist in vision-language models but not \emph{which internal components} are most sensitive to targeted intervention, precluding principled intervention. We argue for mechanism-level evaluation as a necessary complement, demonstrating causal mediation analysis as a dia...
Zhi-Peng Zhao, Wen-Xu Wang, Peishun Liu et al.· 0 citations
Gender bias in large vision-language models (LVLMs) undermines their fairness and reliability, compromising output trustworthiness. Current mitigation methods rely on training-phase adjustments or post-hoc calibration, but face limitations in dynamic visual bias mitigation. These include inability to capture real-time...
Zhi-Peng Zhao, Zhao Wei, Peishun Liu et al.· 0 citations
When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores---using models as instruments for understanding behavior. We address this by asking: can we use large vision-language models (LVLMs) as experimental instruments for studying their own failure dynam...
Zhi-Peng Zhao, Wen-Xu Wang, Peishun Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.