Benchmarking Transformer Models for Sentiment Analysis on YouTube User Comments

Abstract

Social media has become one of the most important platforms for public expression due to its ability to provide users with an unprecedented scale for sharing opinions, sentiments, and experiences. Consequently, social media platforms provide valuable insights into public perception and user sentiment across various topics. This study presents a comparative evaluation of five transformer-based models, namely DistilBERT, BERT, BERTweet, RoBERTa, and Twitter-RoBERTa, on a cleaned and preprocessed YouTube Comments Dataset for sentiment analysis tasks. All models were evaluated under identical experimental settings and preprocessing procedures to reduce textual noise and improve data quality. Each transformer model was fine-tuned under common training conditions to assess its effectiveness in handling informal and user-generated social media content. Experimental results show that Twitter-RoBERTa outperforms other approaches and achieves the best overall accuracy: 90.13% (F1-score: 90.01%), demonstrating its ability to learn from contextual nuances and social media-specific language patterns. In addition, McNemar statistical analysis further supports the statistical significance of many performance differences between the models evaluated including those between Twitter-RoBERTa and smaller general-purpose models such as DistilBERT and BERT. These results demonstrate the value of using domain-specific pretraining on noisy social media text to improve sentiment classification performance and that transformer architectures built with fine-tuning in mind can effectively capture platform specific aspects of linguistics. In summary, this work presents a strong benchmark for transformer-based-sentiment-analysis on YouTube comments and valuable information for future research in social media sentiment analysis.