Imbalanced multimodal learning remains a critical challenge, as models often over-rely on dominant modality while under optimizing non-dominant one, ultimately limiting overall performance. Essentially, we observe that such over-reliance is closely associated with discrepancies in feature aggregation capability across...
Xing-Er Li, Xiao-Xue Wei, Qiang-Yan Yan et al.· Proceedings of the 35th ACM...· 0 citations
Multimodal learning often suffers from modality imbalance, leading to an optimization dilemma. A key reason is that different modalities exhibit asynchronous learning dynamics during joint training. A representative line of existing methods seeks to rebalance multimodal learning through cross-modal alignment. However,...
Yu-Han Zheng, Yang Yang· Proceedings of the 35th ACM...· 0 citations
This paper proposes multimodal Max Confidence Regularization (MaxCR) to dynamically intervene in modality semantic confidence, and reveals that this flaw stems from unimodal characteristics rather than multimodal learning, and this confidence discrepancy can be corrected by positive cross-modal intervention.
Long-Fei Huang, Xiang-Yu Wu, Yang Yang· 0 citations
This paper develops a dynamic strategy that jointly optimizes modality fusion and alignment, and develops a learning-based strategy using a bi-level optimization framework and theoretically proves the convergence of the learning algorithm to ensure its reliability.
Yang Yang, Feng-Qiang Wan, Qing-Jun Jiang et al.· IEEE Transactions on Pattern...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.