X-AI Techniques for Human-AI Teams: The Implementation-Design Framework

Abstract

Explainable artificial intelligence (X-AI) techniques aim to make the actions and decisions of autonomous systems understandable to humans interacting with these systems. In human–AI teams, explainability supports individual understanding and coordination, shared mental models, and collective decision-making among humans and AI agents. Research has shown that X-AI enhances trust in autonomous systems, improves human–AI team performance, and supports collaboration across domains including aviation, finance, healthcare, hospitality, and sports. However, X-AI technologies face difficult challenges, including a lack of transparency and interpretability due to complex underlying models, also known as the “black-box” nature of AI systems. These technologies also lack any universally accepted evaluation metrics and have limited generalizability across applications. One deficit in the X-AI literature is that most frameworks focus on individual-level outcomes, with limited attention to team-level processes. The current study conducted a systematic literature review adhering to PRISMA guidelines and the SALSA framework. This study introduces the Implementation-Design (I-D) framework that organizes X-AI approaches along two dimensions: implementation, ranging from visual to interactive approaches, and design, ranging from isolated explanations to workflow-integrated systems. This framework captures lower-level engagement, involving individual users, to higher-level understanding that is necessary for teams and collectives. Findings indicate that visual explanation approaches support user engagement, while interactive workflow approaches promote deeper understanding, appropriate reliance, and distributed cognition within human–AI teams. Implications highlight the need for team-oriented explainability grounded in shared mental models, transactive memory systems, and collaborative X-AI artifacts. Practical guidelines are included to support researchers and practitioners in selecting appropriate X-AI techniques based on their context and level of analysis. The I-D framework is offered as a conceptual organizing model to guide research and practice, and empirical validation is identified as a priority for future work.