Integrating BERTopic and Heterogeneous Knowledge Graphs for Temporally-Evaluated Research Collaboration Recommendation

Abstract

The rapid growth of scientific publications makes it increasingly difficult to identify relevant research themes and suitable collaborators, and existing approaches typically address topic modeling, structural link prediction, and knowledge-graph construction separately while relying on random data splits that overstate real-world predictive performance. This study proposes an integrated framework combining BERTopic-based topic modeling with a heterogeneous knowledge graph connecting authors, publications, and topics, evaluated on 51,502 Scopus-indexed Computer Science publications affiliated with Indonesian institutions (2021-2025). BERTopic discovered 500 non-outlier topics $(C_{v}=0.7148$, diversity $=0.655$, outlier rate $=37.21 \%)$, which were integrated into a knowledge graph of 149,275 nodes and 2,299,184 edges. Previously unconnected but topically related researchers were then ranked using a hybrid score combining FastNode2Vec embeddings with Adamic-Adar, Jaccard, and Common Neighbors measures, producing 20 topranked collaboration candidates. Under a temporal evaluation predicting first-time 2025 collaborations from historical data through 2024, leak-free structural baselines achieved a ROC-AUC of only 0.686. An initial embedding-based evaluation reused the production graph embedding and reached an inflated ROC-AUC of approximately 0.999; an architectural audit traced this to temporal leakage, since that embedding was learned from the complete 2021-2025 graph rather than exclusively from pre-2025 data. Correcting this by retraining a separate embedding restricted to the pre-2025 subgraph yields a leakage-controlled ROC-AUC of 0.801 (95% bootstrap CI 0.797-0.806), a statistically robust, non-overlapping improvement over the structural baseline's CI (0.682-0.690), showing that graph embeddings retain a genuine, though far more modest, advantage over purely structural evidence once evaluated without exposure to future data. These results demonstrate the value of combining semantic and structural evidence for collaboration recommendation while underscoring that strictly temporal graph construction and evaluation - not just a temporal train/test split - is required to obtain trustworthy performance estimates.