Research on multimodal sentiment analysis based on Mamba aggregation

Abstract

To solve the application efficiency issue of traditional sentiment analysis methods on long-sequence multimodal data, this research presents another multimodal sentiment analysis method based on Mamba aggregation. This method consists of three main components including a parallel feed-forward adapter module, a collaborative residual prompt module, and a Mamba aggregation module. The parallel feed-forward adapter module maintains the integrity of input features by configuring adapters and feed-forward networks in parallel. The collaborative residual prompt module guides the joint learning of multimodal features, and the Mamba aggregation module efficiently models interactions among different modalities. Experiments on the CMU-MOSI and CMU-MOSEI datasets validate its performance against other traditional models. The F1 score and accuracy reach 86.41% and 86.45% respectively, achieving improvements of 1.63% and 1.61% over the best baseline model. Ablation studies further verify the performance enhancement contributed by each module, demonstrating that the proposed method holds distinct advantages in multimodal sentiment analysis tasks.