Aug 2026· Management Sciences· 0 citations· 38 references
TL;DR
An analytical model is developed that incorporates the defining features of human and machine intelligence, capturing the limited but flexible nature of human cognition with imperfect machine recommendations, and represents how AI-based explanations influence the DM’s belief in the algorithm’s predictive quality.
Abstract
Explainable artificial intelligence (AI) models have been proposed to mitigate overreliance and underreliance on AI, which reduce the effectiveness of human-AI collaborative tools. Yet, empirical evidence is mixed, and the impact of explainable AI on the cognitive effort and fatigue of a decision maker (DM) is often overlooked. This paper offers a theoretical perspective on these issues. We develop an analytical model that incorporates the defining features of human and machine intelligence, capturing the limited but flexible nature of human cognition with imperfect machine recommendations. Crucially, we represent how AI-based explanations influence the DM’s belief in the algorithm’s predictive quality. Our results indicate that explainable AI has varying effects depending on the level of explainability provided. Although low explainability levels have no impact on decision accuracy and reliance behavior, they lessen the cognitive burden on the DM. In contrast, higher explainability levels enhance accuracy by improving overreliance but at the expense of increased underreliance. Further, the relative impact of explainability is higher when the DM is more cognitively constrained, when the decision task is sufficiently complex, or when the stakes are lower. Importantly, higher explainability levels can escalate the DM’s cognitive burden (and hence, overall processing time and fatigue) precisely when explanations are most needed (i.e., when the DM is pressed for time to complete a complex task and doubts the machine’s quality). Our study clarifies how explainability affects decision outcomes and cognitive effort, informing the design of effective human-AI systems across decision environments.
This paper was accepted by Jeannette Song, operations management.
Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2024.07450 .
This work presents a comprehensive literature review of the current state of the subfield of XAI that consist of causality-motivated post-hoc XAI methods, and a causal framework for categorising XAI is introduced, and three types of post-hoc XAI methods are identified: observational methods, internally causal methods and externally causal methods.
Anna Rodum Bjøru, Helge Langseth, Inga Strümke et al.· Machine-mediated learning· 1 citation
Artificial Intelligence (AI) has revolutionized decision-making systems of today, allowing automated data analysis, intelligent prediction, and real-time decision-making in a variety of application areas, including healthcare, finance, transportation, manufacturing, cybersecurity, and public administration. While deep learning and other advanced machine learning techniques have been able to deliver impressive results, numerous AI models can be considered as ‘black-box’ models, meaning that they give very accurate predictions without actually offering understandable explanations for their decisions. This lack of transparency has generated a number of concerns about trust, accountability, fairness, ethical compliance, and regulatory acceptance. Explainable Artificial Intelligence (XAI) is thus becoming an indispensable research field which aims to reconcile the predictive power and human interpretability. By explaining the reasoning behind AI system output, model importance, feature impact, and confidence scores, XAI helps users gain insights into how the system is working. This is done to build trust among stakeholders and promote responsible AI governance and decision-making. This paper offers a detailed overview of the concept of Explainable AI in contemporary decision-making processes, covering its theoretical underpinnings, its development, prominent explainability methods, implementation in practice, hurdles, and prospects. A methodology is advanced to embed explainability in the AI decision-making process, starting from data preprocessing to generating explanations and human evaluation. The paper also delves into the implications of explainability on decision quality, user trust, model reliability, and regulatory compliance. The results highlight the potential of explainability to enhance human comprehension and foster responsible use of AI systems in high-stakes decision-making scenarios.
Mahabala H.N· International Journal of Mod...· 0 citations
XAI provides a powerful framework for responsible AI development, challenges such as the performance-interpretability trade-off, lack of standardized evaluation metrics, and potential for human misinterpretation remain areas of active research.
P. Pradhan, Amol Rajmane, C. patil· Journal of image processing...· 0 citations
Aim/Purpose
This paper examines the role of explainability in AI and ML across diverse scientific fields, including medicine, psychology, geology, education, and public policy.
Background
Artificial Intelligence (AI) and Machine Learning (ML) are transforming various scientific disciplines, offering powerful tools for prediction, analysis, and decision-making. However, the complexity of these models often makes them opaque, limiting their utility in high-stakes domains.
Methodology
We selectively sample pivotal domain-specific and cross-disciplinary literature to isolate, contrast, and critique the socio-technical challenges of Explainable AI across five distinct fields of practice.
Contribution
This paper contributes awareness to the critical role of explainability of AI as it is getting integrated into every domain of life.
Findings
Transparency and explainability of AI have different roles in different fields of research. Despite the different roles, explainability is critical to the trust in the results and to enable further scientific advancement.
Recommendations for Practitioners
Practitioners must reject high-level platitudes and deploy domain-specific operational playbooks – such as embedding visual saliency maps in clinical medicine or physical constraints in geosciences – to anchor AI systems in human-centered accountability.
Recommendations for Researchers
Researchers should empirically investigate human-grounded evaluation metrics and adversarial vulnerabilities to mapping tools, moving past generic post-hoc explanations toward context-sensitive, contestable architectures.
Impact on Society
Researchers should empirically investigate human-grounded evaluation metrics and adversarial vulnerabilities to mapping tools, moving past generic post-hoc explanations toward context-sensitive, contestable architectures.
Future Research It is important to better understand the technical and theoretical aspects that hinder explainability in general and specifically for each research domain and map how these can be overcome.
Yahel Giat, A. Mitelman· InSITE Conference· 0 citations
Explainable artificial intelligence (XAI) has become a central methodology for developing transparent, accountable, and human-centered AI systems. As data-driven models are increasingly deployed in high-stakes and socially consequential settings, explanations are expected not only to illuminate model behavior but also to support validation, error analysis, fairness auditing, regulatory compliance, and effective human–AI collaboration. This Editorial introduces the Special Issue “Explainable Artificial Intelligence Technology and Its Applications” and situates its contributions within the broader trajectory of XAI research. We briefly review major methodological families, including intrinsic interpretability, local surrogate and Shapley-value explanations, gradient- and perturbation-based visual attribution, counterfactual and causal explanations, and human-centered evaluation. We then highlight representative contributions in this Special Issue, which demonstrate how XAI is moving from generic explanation visualizations toward domain-sensitive, data-aware, and operationally reliable methods in network security, computer vision, knowledge graphs, and financial decision support. Finally, we discuss future directions, emphasizing faithful and plausible explanations, causal and multimodal reasoning, real-time and hardware-efficient deployment, trustworthy governance, and the emerging role of XAI in education.
Xuewen Sun, Lan Tian, Weidong Zhou et al.· Applied Sciences· 0 citations
Through the application of Artificial intelligence (AI), there has been the establishment of the technology as a cornerstone in current decision systems in essential fields like health, finance, transport, industrial automation, and the governance of citizens. Although traditional AI and machine learning frameworks, specifically deep learning models, have shown outstanding predictive performance, their nature is not enlightened by default, and as such, they have cast considerable doubt on the issues of trust, accountability, fairness, and regulatory compliance. It is thanks to this limitation that Explainable Artificial Intelligence (XAI) as a paradigm has emerged, aimed at ensuring that AI-driven decisions are easy to understand and interpret by the human stakeholders without major performance reduction. This paper is a thorough and stepwise analysis of how XAI was created in current decision systems. It starts with the historical contextualization of the development of AI, as an expert system driven by rules, to a data-driven black-box model and the increasing demand to explain decision-making processes. The paper provides a critical literature review of the current research on XAI methods classifying them into model-intrinsic and post-hoc methods of explanation, and discussing their relevance to various fields. An elaborate methodology is suggested, that incorporates explainability protocols into the AI choice channel, such as information pre-processing, model order, explanation creation and human-centered assessment. Additionally, the paper evaluates the experimental findings and case-based debates on how XAI enhances transparency, end-user trust, compliance with regulations, and system resilience. Popular explainability methods are also compared and evaluated including SHAP, LIME, saliency maps, and rule extraction. The results indicate that explainable models, in addition to increasing interpretability, can also help to improve debugging, bias detection and ethical AI deployment. The paper ends by recommending the current challenges, areas of open research, and future roles of XAI in the development of responsible and human-centered intelligent decision systems.
Meena Krishnan· International Journal of Mod...· 0 citations