A Phenomenological Experiment of Human-AI Communication to Study Online Discourse
The paper describes the author’s personal phenomenological experience of interacting with a chat-based AI system to examine its ability to analyse online discussion. Jurgen Habermas’ validity claim concept was applied as the underlying theoretical framework to examine the text of posted messages by revealing the relationship between their language and semantic logic through AI lens. The key theoretical finding is that AI understood well the complex task of coding the messages’ intended meaning by discerning the link between validity claims and respective illocutionary speech acts. The key practical finding is two-fold: (a) AI worded correctly both the claims and speech acts in line with the guiding example presented by the author, and (b) AI significantly improved the accuracy of such wording following another round of specifying prompts. Overall, it can be concluded that AI’s performance in this experiment was high but depended on the quality of the author’s input.