Aug 2026· Multimedia Systems· Vol 32· 0 citations· 40 references
TL;DR
The framework first introduces learnable sentiment prototypes as semantic anchors to provide explicit sentiment-discriminative guidance for feature completion, and a gradient decoupling strategy is designed to separate the optimization paths of unimodal and multimodal objectives, preventing fusion gradients from interfering with unimodal encoders, thereby synergistically enhancing both discriminative representation learning and multimodal fusion.
A framework for learning adaptive cross-modal interactions for multimodal sentiment analysis that consistently outperforms previous methods and enhances multimodal representation capability for sentiment classification is proposed.
Chuhan Cheng, Hangcheng Wu, Junqiao Wang et al.· International Conference on...· 0 citations
In multimodal sentiment analysis, textual, acoustic, and visual modalities often contain redundant and noisy information. Such information increases model complexity and weakens core sentiment representations, degrading accuracy and robustness. To address this issue, we propose CLIBN, a multimodal sentiment recognition network based on contrastive learning and information bottleneck. First, we design a sentimentintensity-aware contrastive learning strategy. It constructs positive and negative pairs according to sentiment intensity distances and assigns adaptive weights to different pairs, enabling the model to capture fine-grained sentiment differences. Second, we introduce a hierarchical information bottleneck module. It treats text as the primary modality and progressively integrates complementary cues from acoustic and visual modalities, while preserving task-relevant semantics and suppressing redundant information. Experimental results on CMU-MOSI and CMU-MOSEI show that CLIBN achieves superior performance. Specifically, Acc-2 reaches 87.8% and 86.7%, and F1-Score reaches 87.8% and 86.6% on the two datasets, respectively. These results demonstrate the effectiveness of CLIBN for multimodal sentiment representation learning.
Xu Meng, Yi Zhang, Yang Li· 2026 8th International Confe...· 0 citations
Intelligent human–computer interaction, healthcare monitoring, social media analytics, and conversational AI rely on emotion recognition and sentiment analysis. However, multimodal learning systems frequently presume equal dependability between text, audio, and visual modalities, rendering them susceptible to environmental noise, missing information, and modality-specific errors. An Adaptive Reliability-Guided Multimodal Artificial Intelligence (ARGMAI) system for robust multimodal emotion and sentiment classification addresses these problems. In a single end-to-end architecture, reliability-aware preprocessing, adaptive cross-modal attention, dynamic feature fusion, contrastive representation learning, noise suppression, and cross-modal consistency optimisation are combined. Missing modality reconstruction allows the framework to preserve discriminative representations when one or more modalities are lacking. In model is tested using the public CMU-MOSEI and IEMOCAP datasets. Experimental results show that ARGMAI achieves 90.36% accuracy, 90.04% precision, 89.78% recall, 89.91% F1-score, and 96.08% AUC on the CMU-MOSEI dataset and 88.14% accuracy, 87.88% precision, 87.53% recall, 87.70% F1-score, and 88.41% weighted accuracy on the IEMOCAP The approach outperforms existing state-of-the-art multimodal learning algorithms under noisy and missing modality situations and provides consistent convergence and enhanced generalisation for real-world affective computing applications.
Sunil Kumar Piyush, K. Pareek· Journal of Intelligent Decis...· 0 citations
Incomplete multimodal sentiment analysis has garnered significant attention in recent years. Existing approaches typically assume that data is missing at random or are designed specifically for certain missing patterns, ignoring the modality combination inconsistency between training and testing phases. However, in real-world scenarios, the testing phase often encounters modal combinations that were not present during the training phase, which leads to insufficient generalization capabilities and unstable performance. In this paper, we introduce the problem of Incomplete Multimodal Sentiment Analysis with Unseen Modality Combinations (IMSAUMC), aiming to enhance model generalization for unseen modality combinations. To address this challenge, we propose the model named $\textbf{C}$ontrastive $\textbf{M}$ixed $\textbf{P}$rompt $\textbf{L}$earning ($\textsf{CMPL}$) for IMSAUMC. It introduces a label-guided contrastive feature learning mechanism to learn robust and discriminative cross-modal representations. Additionally, we design modality-combination prompts with a soft router to facilitate better learning of various modality combinations. Furthermore, we introduce three prompt contrastive learning strategies, which enable effective learning of prompts corresponding to unseen modality combinations, thereby significantly strengthening the model's generalization capabilities in diverse testing scenarios. Extensive experiments on three widely used datasets demonstrate that $\textsf{CMPL}$ achieves more than a 5% improvement in accuracy compared to state-of-the-art approaches.
Kaixin Xu, NaiJin Liu, Yu-Ri Kang et al.· 0 citations
A Modality Dropout strategy is first introduced at the input stage to alleviate over-reliance on a sin-gle modality and improve robustness and the proposed Hierarchical Global-Local Interaction and Refinement framework for Multimodal Sentiment Analysis (HGLIR) is proposed.
yuanyuan zhou· Poster Volume 0007 The 2026...· 0 citations
: Multimodal sentiment analysis (MSA) has made significant progress in integrating heterogeneous information from text, speech, and vision. However, real-world multimodal data often suffer from modality noise, semantic inconsistency, and incomplete modality information, which can weaken cross-modal fusion and reduce the reliability of sentiment prediction. To address these challenges, this paper proposes RUAL, a robust uncertainty-aware learning framework for multimodal sentiment analysis. Specifically, RUAL first employs a Gathered Multi-Head Attention Pooling (GMHA) module to aggregate intra-modal features and estimate modality uncertainty based on attention entropy. Then, an Uncertainty-Aware Cross-Modal Coupled Layer (UACCL) is introduced to dynamically regulate cross-modal residual fusion according to sample confidence, thereby reducing the negative influence of unreliable modalities on fused representations. In addition, uncertainty-weighted learning and uncertainty-guided self-distillation (UWL and U-SD) are jointly integrated through an optimization strategy to further improve training stability and generalization in complex scenarios. Experimental results on CMU-MOSI, CMU-MOSEI, and MVSA-Single demonstrate that RUAL achieves strong overall performance and maintains stable prediction results under missing-modality and Gaussian-noise conditions, validating the effectiveness and robustness of the proposed framework for multimodal sentiment analysis.
