A Three-Stage Cross-Lingual Knowledge Transfer Approach Based on the XLM-RoBERTa Model for Detecting Fake News in Ukrainian

Abstract

In recent years, there has been an increase in the amount of fake news in the media, which is why fact-checking systems are gaining popularity, particularly those that use natural language processing (NLP) to quickly identify and flag fake news. One of the main limitations in the development of such systems is the limited number of datasets containing verified information, which are necessary for the effective training of models. The situation is particularly critical for non-English datasets, specifically those in the Ukrainian language. This article proposes a three-stage algorithm for training a model to recognize fake news in the Ukrainian language. At the core of the proposed approach lies the multilingual transformer model XLM-RoBERTa, which solves this problem by utilizing cross-lingual knowledge transfer from English to Ukrainian. This approach means there is no need to search for a large, high-quality dataset in Ukrainian; instead, a significantly smaller dataset in Ukrainian can be used for the final calibration of the model. The model developed as a result of the experiment proved effective in extreme low-resource scenarios, achieving 90.7% accuracy on just 500 training records and outperforming the baseline model by 9.7%.