A practical decision framework for four foundational measures - Entropy, KL divergence/cross-entropy, Mutual Information, and Transfer Entropy is provided, organized around three prescriptive questions for each: what question does the measure answer and in which AI context; which estimator is appropriate for the data type and dimensionality; and what is the most dangerous misuse.
Abstract
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning and feature selection, and transfer entropy reveals directed influence in dynamical systems. Despite wide adoption, measure selection is often decoupled from estimator assumptions, failure modes, and safe inferential claims. This survey provides a practical decision framework for four foundational measures - Entropy, KL divergence/cross-entropy, Mutual Information, and Transfer Entropy - organized around three prescriptive questions for each: (i) what question does the measure answer and in which AI context; (ii) which estimator is appropriate for the data type and dimensionality; and (iii) what is the most dangerous misuse. The framework is operationalized in two complementary artifacts: a measure-selection flowchart and a master decision table. We cover both AI/ML and decision-making agent application domains per measure, with standardized Bridge notes linking IT quantities to cognitive and neuroscientific constructs. Two worked examples illustrate the framework on concrete practitioner scenarios spanning representation learning and temporal influence analysis, and a reproducible multi-agent case study across three learning architectures validates the transfer-entropy surrogate-testing guardrail against a null control.
This work examines intrinsic limits of data-driven decision systems from an information-theoretic and interaction-based perspective and describes decision systems, including LLM-integrated agent architectures, as feedback-driven stochastic processes where state-dependent dynamics may induce emergent macroscopic behavior.
A trust-centric framework that integrates ensemble-based uncertainty estimation, maximum mean discrepancy (MMD) shift detection, a dependability signal, and safety-aware decision logic to prioritize reliable actions over raw predictive confidence is proposed.
Yusuf Surajo, S. Basri, A. Balogun et al.· Proceedings of the 21st Inte...· 0 citations
This lecture note develops, from first principles and within a single decision-theoretic setting, the link between the decision objective and the knowledge of an agent and the form of uncertainty representation that is sufficient to act optimally.
Artificial Intelligence (AI) has revolutionized decision-making systems of today, allowing automated data analysis, intelligent prediction, and real-time decision-making in a variety of application areas, including healthcare, finance, transportation, manufacturing, cybersecurity, and public administration. While deep learning and other advanced machine learning techniques have been able to deliver impressive results, numerous AI models can be considered as ‘black-box’ models, meaning that they give very accurate predictions without actually offering understandable explanations for their decisions. This lack of transparency has generated a number of concerns about trust, accountability, fairness, ethical compliance, and regulatory acceptance. Explainable Artificial Intelligence (XAI) is thus becoming an indispensable research field which aims to reconcile the predictive power and human interpretability. By explaining the reasoning behind AI system output, model importance, feature impact, and confidence scores, XAI helps users gain insights into how the system is working. This is done to build trust among stakeholders and promote responsible AI governance and decision-making. This paper offers a detailed overview of the concept of Explainable AI in contemporary decision-making processes, covering its theoretical underpinnings, its development, prominent explainability methods, implementation in practice, hurdles, and prospects. A methodology is advanced to embed explainability in the AI decision-making process, starting from data preprocessing to generating explanations and human evaluation. The paper also delves into the implications of explainability on decision quality, user trust, model reliability, and regulatory compliance. The results highlight the potential of explainability to enhance human comprehension and foster responsible use of AI systems in high-stakes decision-making scenarios.
Mahabala H.N· International Journal of Mod...· 0 citations
Neural networks increasingly guide decisions in high-stakes domains such as medical diagnosis, credit approval, and energy bidding. Audit in these settings requires case-level evidence: which training cases support an action and what outcomes they carried. Case-based decision theory (CBDT) formalizes this reasoning by aggregating outcome support from remembered cases. We show that an OLS action readout fitted on a fixed neural representation admits an exact case-based decomposition. Each action score is a weighted sum of training-case returns, with coefficients determined by empirical Gram geometry. We identify a sufficient regime for CBDT similarity semantics; outside it, the coefficients should generally be treated as signed Gram-geometric influence. The decomposition yields audit signals that trace scores to training cases, measure action coherence, and identify weak support. Across synthetic CBDT, PJM, Adult Income, and Default Credit tasks, the method recovers case-level preference structure and achieves the highest mean Top-30 consistency among compared attribution baselines, while remaining competitive on support reconstruction. The audit requires only fitting an OLS top-layer probe, without retraining the representation or accessing the original optimization trajectory; probe fidelity is measured by score reconstruction.
Manli Yan, Yu-Erh Lin, Yaowen Yu et al.· 1 citation
It is shown that a key factor is the error correlation structure between human and AI predictions, and when the AI's prediction errors are negatively correlated with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility.
