A Large-Scale Vietnamese News Dataset for Text Classification: Construction and Evaluation
TL;DR
This study analyzes classification performance through four critical dimensions: model architecture, temporal data shift, source-origin bias, and training data scale, and reveals distinct learning behaviors across model families: transformer models benefit increasingly from larger training sets, whereas the strongest traditional and deep learning baselines remain competitive throughout the evaluated range.
Abstract
The Vietnamese language presents distinctive natural language processing (NLP) challenges, which are compounded by a critical shortage of standard benchmark datasets. To address this gap, this paper introduces BN-VN3S, a large-scale Vietnamese news dataset containing 946,696 final processed articles derived from 1,042,295 articles initially collected from three major Vietnamese news publishers: VnExpress, VietNamNet, and Dân TrÍ. Spanning an extended publication period from 2019 to 2025 across 10 topical categories, this dataset provides unprecedented diversity. Utilizing this resource, we conduct a comprehensive empirical evaluation of ten different models across three paradigms: Traditional Machine Learning (Naïve Bayes, Logistic Regression, LinearSVC, SGDClassifier), Deep Learning (TextCNN, BiGRU, TextRCNN), and Transformers (mBERT, XLM-R, PhoBERT). Our study analyzes classification performance through four critical dimensions: model architecture, temporal data shift, source-origin bias, and training data scale. Experimental results surprisingly reveal that deep learning and traditional models surpass Transformers in overall performance; BiGRU achieved the highest macro F1-score of 92.42%, closely followed by LinearSVC at 92.33%, whereas PhoBERT reached 87.98% and mBERT lagged at 78.73%. However, under temporal distribution shifts evaluated on 2023–2025 data, Transformers particularly PhoBERT demonstrate superior robustness and maintain the most stable performance. Furthermore, we find that models are highly sensitive to source bias; BiGRU suffered a substantial performance drop of up to 9.71 F1 points during cross-source evaluation, while Naïve Bayes and mBERT were significantly more resilient. Finally, the data-scale analysis reveals distinct learning behaviors across model families: transformer models benefit increasingly from larger training sets, whereas the strongest traditional and deep learning baselines remain competitive throughout the evaluated range. No consistent crossover is observed under the present experimental configuration, suggesting differences in sample efficiency rather than the general superiority of any model family. Taken together, these findings provide actionable insights and practical recommendations for optimizing Vietnamese news classification systems in real-world environments.