Machine Learning for Fake News Detection: Model Comparison, Interpretability, and Cross-domain Evaluation

Abstract

Using a Kaggle dataset of 44,898 balanced news articles, this study compared four classic machine learning models—logistic regression, multinomial Naive Bayes, random forest, and linear SVM—with a lightweight deep learning model, DistilBERT, for fake news detection. The linear SVM paired with TF-IDF features turned out to be the strongest, reaching a 98.86% accuracy and outperforming DistilBERT, all while needing much less computation and training time. SHAP analysis showed that these models mostly picked up on stylistic cues, like casually mentioning a political figure, rather than doing any actual fact-checking. When tested on topics the models hadn’t seen before, accuracy dropped by 3 to 5 percentage points, which points to real weaknesses in generalization. So while traditional machine learning models can serve as fast, efficient baselines for narrow tasks, they still fall short in practical, everyday use. Moving forward, research could lean more on content-based verification and bringing in deep learning architectures that are more fully featured.