Beyond Manipulation: How Users Perceive Harmful AI Chatbot Interactions

Abstract

AI chatbots are increasingly deployed across all areas of life. Research on conversational dark patterns documents influence-based harms such as biased framing and behavioral steering, while AI companion research highlights relational harms including boundary violations and social discomfort. Yet it remains unclear whether users perceive these as distinct forms of harmful interaction. We report a mixed-methods study (N = 100) in which participants were asked to recall either a positive, inappropriate, or manipulative chatbot interaction. Exploratory factor analysis of an adapted self-report perceived manipulation questionnaire identified two dimensions: an Experiential Dimension (capturing cognitive and affective responses to manipulation, such as feeling deceived, controlled, or taken advantage of) and a Behavioral Dimension (capturing user behaviors perceived as influenced by the chatbot, such as acting beyond original intentions). Manipulative interactions scored higher on the Experiential Dimension than inappropriate interactions, while both conditions did not differ on the Behavioral Dimension. Qualitative findings further showed that manipulative interactions were associated with evaluative contexts and steering behaviors, whereas inappropriate interactions were linked to relational contexts and social discomfort. We discuss implications for trustworthy AI design and for regulatory frameworks such as the EU AI Act.