Comparative Analysis of Language Models for Sentiment Classification
TL;DR
Comparing and analysing the performance of several machine learning algorithms on fine-grained sentiment classification problems to examine their suitability and shortcomings for use as models in sentiment analysis suggests large language models perform significantly worse on the 28-class classification task in zero-shot settings, suggesting that they are better suited for generative and open-ended emotional interaction than for standardized classification benchmarks.
Abstract
— By comparing and analysing the performance of several machine learning algorithms on fine-grained sentiment classification problems to examine their suitability and shortcomings for use as models in sentiment analysis. Sentiment analysis remains a prominent research area in Natural Language Processing (NLP). However, systematic comparisons of whether these methods demonstrate superiority in fine-grained classification settings have yet to be conducted. Based on the GoEmotions dataset that includes 28 sentiment classes as our experiment’s reference for evaluating four kinds of models: logistic regression, BiLSTM, BERT, and the large-scale language model Claude (claude-haiku-4-5). The experiments employ TF-IDF feature extraction, sequence encoding, pre-training with fine-tuning, and zero-shot and few-shot prompting strategies. Based on this experiment, the model’s generalisation capabilities improve as architectural depth increases. The accuracy of logistic regression was only 55.11%; BERT obtained the highest F1 score and performed the best overall, and bidirectional pre-trained representations were most valuable among other methods. However, large language models perform significantly worse on the 28-class classification task in zero-shot settings, suggesting that they are better suited for generative and open-ended emotional interaction than for standardized classification benchmarks.