Balancing Trust and Deliberation in Human–AI Decision Support: The Effects of Explainable AI and Cognitive Forcing Functions
TL;DR
This work examines how six decision-support mechanisms affect engagement, trust, and collaborative task performance in a diabetes meal-planning scenario and argues for a contextual, balanced pairing of CFF and XAI design that accounts for interactivity, decision frequency, and task complexity.
Abstract
Reliance on AI systems for decision support is expanding into domains where mistakes carry real consequences, which makes it important that users can weigh AI suggestions against their own judgment rather than deferring to them by default. Prior work has found that human-AI teams sometimes underperform AI alone, a pattern usually attributed to automation bias: people continue to follow AI recommendations even when those recommendations are wrong. Most existing systems address this through text-based explanations (XAI), intended to make the AI's reasoning legible to users. Because everyday decision-making leans heavily on fast, intuitive, bias-prone processing, users may skim past these explanations or fail to engage with them in any depth. One proposed remedy is the cognitive forcing function (CFF), a design intervention meant to interrupt automatic responding and prompt more deliberate evaluation. We examine how six such decision-support mechanisms affect engagement, trust, and collaborative task performance in a diabetes meal-planning scenario. In a controlled study with 108 participants (18 per condition), we tested two explanation types, text and visual, alongside four CFFs: AI confidence levels, human feedback, AI-generated questions, and performance visualization. AI confidence levels, text explanations, and performance visualization each significantly improved decision accuracy, and reasoning cues such as text and confidence levels also improved trust. Human feedback and AI-driven questions prompted deeper reflection but appear to have come at a cost to performance, plausibly because of the added cognitive effort and scrutiny involved, which in turn dampened trust. Visual explanations, despite their simplicity, did not meaningfully move trust. These results argue for a contextual, balanced pairing of CFF and XAI design that accounts for interactivity, decision frequency, and task complexity.