Multimodal Learning for Enhancing Context-Aware Recommendation Systems

Abstract

Context-aware recommendation systems are essential in many e-commerce streaming and smart city applications for alleviating information overload such as providing personalized recommendations for a user's changing context and for items with rich attributes. But current multimodal methods lose considerable information in the heterogeneous features, the one-size-fits-all fusion strategies, and the lack of consideration of explicit fine-grained personalization preferences and global collaborative signals leads to suboptimal personalization performance in the presence of data sparsity and modality interference. We propose a new multimodal approach that jointly improves the modalities-specific item attributes through modality-specific graphs models multiple user preferences by performing dynamic context injection, captures hierarchical global signals using attention-based mechanisms and fuses the modalities using a heterogeneous cross-attention approach to make robust prediction. The superiority of the proposed method is also extensively verified on Amazon Baby dataset with the result of Recall@10 = 0.213 and NDCG@20 = 0.185, which is 9.2% and 7.6% better than the best baseline respectively and consistent effects are observed on the ablation studies showing the benefit of the proposed context awareness, hierarchy and attention modules. These results highlight the effectiveness of the framework to improve the accuracy, novelty and explainability of the generated multimodal recommendations, towards scalable recommendations in real-world, sparse and dynamic contexts.