Adaptive Modality Routing for Temporal Knowledge Graph Forecasting with Heterogeneous Mixture-of-Experts

Abstract

Temporal knowledge graph forecasting aims to predict future missing facts from continuously evolving relational data in dynamic information systems. Such forecasting tasks are increasingly important for intelligent applications involving complex interactions among entities, events, and evolving environments. Existing approaches typically employ unified architectures that process all queries through the same representation pipeline, despite substantial variations in structural connectivity, historical availability, and semantic information among different queries. Such a homogeneous modeling strategy may limit the ability to effectively handle diverse forecasting scenarios, including highly connected entities, recurrent temporal patterns, and cold-start cases with insufficient historical evidence. To address this challenge, we propose a Heterogeneous Modality Mixture-of-Experts (H-MoE) framework that adaptively selects specialized reasoning pathways according to query characteristics. Unlike conventional mixture-of-experts architectures with homogeneous subnetworks, our framework incorporates three complementary experts with distinct inductive biases: a time-aware graph neural network for structural relational reasoning, a temporal transformer for historical sequence modeling, and a semantic representation adapter based on pretrained language models for knowledge transfer in sparse scenarios. A query-aware gating network is introduced to dynamically allocate computational resources among different experts according to the available structural, temporal, and semantic evidence. Furthermore, we employ entropy-based routing regularization to encourage confident expert selection while maintaining balanced expert utilization, together with a representation diversity constraint to promote complementary feature learning among heterogeneous reasoning pathways. Experiments on three benchmark temporal knowledge graph datasets indicate that the proposed framework can improve forecasting performance under the evaluation setting used in this study. The largest observed gains occur for queries with limited historical evidence. These findings support heterogeneous expert specialization as a useful design direction, while the scope of the conclusions remains limited to the tested ICEWS benchmarks and implementation.