Identifying Interaction Dimensions and their Relation to Agent Behavior in Human-AI Teams
TL;DR
This work provides a new framework for AI agent designers to understand user behavior and guidelines for better supporting it, and observed that while users self-report goal prioritization independently of agent behavior, in-game logs reveal that non-collaborative goals are prioritized significantly more often with AI leaders than with followers.
Abstract
Identifying and promoting effective user interaction strategies in human-AI interaction is critical to improving collaboration quality. However, what motivates users or how agent behavior affects them remains unclear. We analyzed self-reported user strategies in a study (N=60) using a collaborative game with three AI partners reflecting leader, follower, and shifting initiative behavior. We identified four interaction dimensions — intention, perception, communication, and coordination — and assessed their relation to agent behavior, team performance, and user preferences. We observed that while users self-report goal prioritization independently of agent behavior, in-game logs reveal that non-collaborative goals are prioritized significantly more often with AI leaders than with followers. Moreover, users tend to construct plans early through observation when the AI leads, yet they are more likely to use strategies to propose goals when the AI follows. Our work provides a new framework for AI agent designers to understand user behavior and guidelines for better supporting it.