A Cross-Attention-Based Framework for Multimodal Sentiment Analysis

Abstract

Multimodal Sentiment Analysis (MSA) aims to predict human sentiment by jointly modeling complementary information from textual, acoustic, and visual modalities. However, effectively exploiting heterogeneous multimodal features remains challenging due to semantic inconsistency, temporal misalignment, and noisy modality-specific representations. To address these issues, this paper proposes a Cross-Attention-based multimodal sentiment analysis framework that explicitly models inter-modal interactions through multi-directional cross-modal attention. Specifically, modality-specific features are first projected into a unified latent space via lightweight modality encoders, after which bidirectional cross-attention is employed to capture complementary dependencies among visual, acoustic, and textual modalities. To further enhance representation learning, multi-head attention and positional encoding mechanisms are incorporated to improve cross-modal interaction modeling and temporal structure awareness. Extensive experiments on the CMU-MOSI benchmark demonstrate that the proposed framework consistently outperforms conventional fusion baselines, including concatenation-based, additive, and self-attention-based fusion strategies. Comprehensive ablation studies further verify the effectiveness of each architectural component, while qualitative attention visualization confirms that the model learns interpretable attention patterns and focuses on semantically informative regions during multimodal fusion. These results indicate that the proposed method provides an effective and interpretable solution for multimodal sentiment analysis and offers useful insights for future attention-based multimodal fusion research.