What Does 99% Accuracy Measure? A Reproducible Audit of Shortcut Learning in a Widely Used Fake News Corpus
Text classifiers trained on the ISOT/Kaggle "Fake and Real News"corpus routinely report accuracy and F1 above 0.98, which concludes that within-corpus scores here quantify source and topic separability rather than veracity, and is recommended metadata-only, small-sample, and topic-disjoint baselines as inexpensive diag...