SFASA is proposed, an approach that leverages multimodal contrastive learning to recover invariant semantic factors and dynamically routes only semantically salient tokens into lightweight low-rank adapters by reusing frozen VLM projection heads as zero-parameter modality mappers.
Ze-Xiang Li, Wen-Xu Wang, Jia-Wen He et al.· Proceedings of the 34th ACM...· 0 citations
Large pre-trained Vision-Language Models (VLMs) excel in zero-shot generalization but face challenges in efficient few-shot adaptation. Existing parameter-efficient methods apply uniform updates across all tokens, overlooking the inherent sparsity of cross-modal interactions and exacerbating overfitting. We propose SFA...
Ze-Xiang Li, Wen-Xu Wang, Jia-Wen He et al.· Proceedings of the 34th ACM...· 0 citations
Gender bias in large vision-language models (LVLMs) undermines their fairness and reliability, compromising output trustworthiness. Current mitigation methods rely on training-phase adjustments or post-hoc calibration, but face limitations in dynamic visual bias mitigation. These include inability to capture real-time...
Zhi-Peng Zhao, Zhao Wei, Peishun Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.