Judgement and the Feeling of Agency When Instructing a Computer to Act

Abstract

In cognitive psychology, the sense of agency refers to the feeling of controlling an external event through one's own actions. This paper asks how people experience agency when they give an instruction to a computer, which then acts on their behalf. Research in human-human interaction has found that while people experience agency over the initial actions taken by another in response to their command, they may not experience agency over the ultimate outcome of those actions. We present two experiments. The first draws on prior studies of human-to-human interaction but replaces a human co-actor with a computer. The second study addresses potential limitations in the first, exploring the effect of more meaningful outcomes, and assessing both pre-reflective feelings and post-reflective judgements of agency. We find that participants experienced agency over the initial action taken by the computer in response to their command, but did not experience pre-reflective agency over the final outcome. Interestingly, however, they did judge themselves to have caused the final outcome when it was meaningful and their command specific. This difference between the pre-reflective feeling and post-reflective judgement of agency has important implications for design. If designers want people to feel in control and take responsibility for outcomes that result from instructions given to a computer, they cannot rely on pre-reflective intuition. Instead, we should design systems that encourage people to reflect on their actions and make deliberative judgments regarding responsibility.