Multimodal rumor interpretation generation based on multisource semantic graphs
Abstract
As social media gains influence across various sectors of society, multimodal rumors that combine text and images can spread more rapidly and influential. Most existing research focuses solely on determining the authenticity of rumors, while research on the interpretable generation of rumors has significant shortcomings. This paper proposes a multimodal rumor explanation generation model based on a multi-source semantic graph. It employs Faster-RCNN object detection technology to extract the categories and attribute information of key objects in images, converting them into standardized textual metadata. Furthermore, leveraging the ConceptNet common-sense knowledge base, retrieves common-sense knowledge linking anchor concepts in textual and visual metadata. By leveraging a Graph Convolutional Network (GCN) to perform cross-modal and cross-domain semantic reasoning, the model generates logically coherent and interpretable text. Finally, the model’s effectiveness was validated on the public benchmark dataset MORE. The results demonstrate that the proposed model can accurately uncover the misleading core and false logic of multimodal rumors, significantly outperforming existing mainstream models across various evaluation metrics.