Evaluating the Robustness of Transformer-based Models Against Data Poisoning Attacks in Social Media Sentiment Analysis

Abstract

Transformer-based Pre-trained Language Models (PLMs) has become standard tools for sentiment analysis on social platforms, yet their vulnerability to data poisoning during training remains a critical concern. In this paper, we conduct a baseline resilience study of BERT, RoBERTa, and BERTweet against three increasingly complex adversarial attacks: label flipping, trigger injection, and clean-label poisoning. Using the Sentiment140 corpus, we introduced poisoned data at rates of 5%, 10%, 20% and 30%, tracking performance across multiple metrics including Accuracy, Macro-F1, Attack Success Rate (ASR), and Clean Accuracy (CACC). The experimental data—averaged over three random seeds—revealed that while basic label flipping caused steady performance degradation, our conservative configurations of trigger injection and clean-label poisoning often failed to disrupt the models reliably. Ultimately, the domain-aligned BERTweet model demonstrated the highest baseline resilience to these specific manipulations, highlighting the need for future evaluations using more advanced threat models.