"Do You Trust It... or Check It?": Exploring Children's Verification Behaviors in Game-Based Human-AI Collaboration

Abstract

Children are increasingly interacting with AI agents in learning and problem-solving contexts, where AI suggestions may appear fluent, helpful, and socially engaging. However, children may not always be able to judge whether such suggestions are accurate, incomplete, or misleading. This paper presents a game-based human-AI collaboration probe designed to examine how children verify AI suggestions during task-based interaction. The system embeds AI suggestions within a biological knowledge reasoning game and compares three Agent interaction styles: partner-like, tool-like, and dual-state Agent. Instead of treating trust as a static post-task attitude, this study focuses on children's situated verification behaviors, including acceptance, evidence checking, questioning, revision, and continued AI use. A preliminary evaluation with children aged 8-10 combined interaction logs, post-task ratings, and semi-structured interviews. The findings suggest that Agent style shaped children's engagement and response pathways: partner-like interaction tended to support direct questioning, while tool-like interaction was more associated with evidence seeking. Children generally viewed AI as useful but fallible, and their questioning behaviors varied in reasoning depth. The study contributes design insights for supporting children's critical, reflective, and evidence-based use of AI in collaborative tasks.