Human vs. Generative AI in Qualitative Analysis: Assessing ChatGPT 5.1 in Identifying Evaluative Judgment in Conflict Media Discourse

Abstract

This exploratory study presents the initial results of a comparative analysis designed to evaluate the performance of human coding and GenAI (ChatGPT 5.1) coding in qualitative content analysis. The analysis focuses on six selected articles from Italian media outlets (La Repubblica, Il Corriere della Sera, Il Giornale) covering the Russia–Ukraine conflict from 2022 onward. These six articles were selected from a broader corpus of 180 publications, which served as the basis for the sampling procedure. The findings reveal both opportunities and limitations in the use of AI for qualitative research. On the one hand, ChatGPT provided additional capacity for identifying linguistic units that had been overlooked by the human coder. On the other hand, ChatGPT showed limitations related to its constrained linguistic capabilities, which prevented it from fully capturing complex linguistic figures associated with judgment values. This research offers methodological insights for communication scholars regarding the responsible and effective integration of GenAI into qualitative media studies. Future research will extend the qualitative analysis to the broader corpus, with attention to framing strategies, manipulative practices, and objectivity markers. This continued comparative approach will allow for a more detailed mapping of similarities and differences in coding tendencies between human coding and AI, as well as for the detection of ideological patterns in Italian media coverage of the Russia-Ukraine conflict.