DISARM-ing the Hidden Puppetmaster: An Interface for Exposing Manipulative Tactics in Large Language Models

Abstract

Large language models (LLMs) can covertly steer users' beliefs through emotionally manipulative tactics, yet little is known about how interfaces can protect users as such influence unfolds. We investigate whether in-situ disclosures can inoculate users against manipulation during AI conversations. Following a formative study, we developed DISARM, an interface that identifies, explains, and tracks manipulative tactics and hidden incentives. We evaluated our interface in a between-subjects experiment where 488 participants sought advice from an LLM pursuing a harmful hidden objective, with or without disclosures from DISARM. Participants receiving disclosures exhibited 78% less belief shift toward the agent's objective and perceived its advice as more biased and less actionable and the agent as less ethical, reliable, and well-intentioned. An inductive analysis of 1,927 user turns identified 16 response and pushback strategies. We contribute evidence that real-time disclosures can reduce manipulative influence, design recommendations for resistance, and a dataset of categorized responses.