When Modalities Compete: Modality-Aware Optimization on Low-Rank Manifolds
Parameter-efficient fine-tuning like low-rank adapters is widely used to adapt multimodal large language models to downstream tasks, yet we identify an unaddressed failure mode called Non-Textual Norm Collapse (NTNC). NTNC causes non-textual modality representations to shrink to near-zero after low-rank projection, lea...