Effectiveness of Fake News Detection Systems Under in-Domain and Blind Testing Conditions

Abstract

With the rapid growth of social media platforms, the spread of misleading and false information has become a critical challenge. Earlier research investigates Machine Learning Techniques with features such as TF-IDF, BOW and n-grams showing high accuracy on curated datasets with significant decline in performance on new or unseen data indicating poor generalization. Motivated by these, this paper presents a systematic evaluation of the effectiveness of Fake News Detection Systems using four deep learning architectures: GloVe-CNN, GloVe-LSTM, FastText-CNN, and FastText-LSTM. All models are trained on the ISOT dataset using stratified 70:15:15 splits, with a completely independent external dataset (TextDB3) reserved exclusively for blind testing. Results show near-perfect in-domain accuracy 98–99%, which drops substantially to 63–66% under blind testing comparable to traditional machine learning baselines. This indicates that although deep-learning models capture semantic information more effectively, handling data from diverse domains is still a major challenge in real-world fake news detection. These findings highlight the need for more robust and domain-adaptive solutions in future work.