A Comparative Accuracy Analysis of Node2Vec and FastRP

Abstract

The rapid increase in scientific article publications has created a need for automated methods to support various applications, such as recommendation systems that help researchers discover relevant literature by measuring similarity between scientific publications using graph and text embeddings. This study mainly compares the accuracy of two graph embedding methods, Node2Vec and FastRP, where similarity is measured based on title, abstract, and citation attributes using a hybrid approach. In this approach, relational attributes (citations) are processed with graph embeddings, while textual attributes (titles and abstracts) are processed with text embeddings (S-BERT). The resulting vectors are then combined using a weighting scheme across attributes of title, abstract, and citation, with similarity values calculated using Cosine Similarity. The ordinal ranking results of these similarity scores are evaluated using Kendall Tau and Normalized Discounted Cumulative Gain (NDCG). The evaluation results indicate that Node2Vec achieves higher agreement with the similarity rankings generated by SPECTER than FastRP, as measured using NDCG and Kendall Tau. This study contributes to understanding the impact of graph structures in Node2Vec and FastRP on similarity scores derived from embeddings of title, abstract, and citation attributes. However, the current results are limited to a dataset consisting of only three attributes (title, abstract, and citation) from scientific publications, suggesting that further research must incorporate additional attributes (introduction, methods, results & discussion, and conclusion). The findings of this study are expected to provide practical guidance for researchers and developers of recommendation systems in selecting the most appropriate embedding method for their needs.