The Risks, Challenges, and Potential Opportunities with GenAI

An Overview of Explainable Artificial Intelligence (XAI) and Its Application

XAI provides a powerful framework for responsible AI development, challenges such as the performance-interpretability trade-off, lack of standardized evaluation metrics, and potential for human misinterpretation remain areas of active research.

The Emergence of Explainable AI in Modern Decision Systems

Artificial Intelligence (AI) has revolutionized decision-making systems of today, allowing automated data analysis, intelligent prediction, and real-time decision-making in a variety of application areas, including healthcare, finance, transportation, manufacturing, cybersecurity, and public administration. While deep learning and other advanced machine learning techniques have been able to deliver impressive results, numerous AI models can be considered as ‘black-box’ models, meaning that they give very accurate predictions without actually offering understandable explanations for their decisions. This lack of transparency has generated a number of concerns about trust, accountability, fairness, ethical compliance, and regulatory acceptance. Explainable Artificial Intelligence (XAI) is thus becoming an indispensable research field which aims to reconcile the predictive power and human interpretability. By explaining the reasoning behind AI system output, model importance, feature impact, and confidence scores, XAI helps users gain insights into how the system is working. This is done to build trust among stakeholders and promote responsible AI governance and decision-making. This paper offers a detailed overview of the concept of Explainable AI in contemporary decision-making processes, covering its theoretical underpinnings, its development, prominent explainability methods, implementation in practice, hurdles, and prospects. A methodology is advanced to embed explainability in the AI decision-making process, starting from data preprocessing to generating explanations and human evaluation. The paper also delves into the implications of explainability on decision quality, user trust, model reliability, and regulatory compliance. The results highlight the potential of explainability to enhance human comprehension and foster responsible use of AI systems in high-stakes decision-making scenarios.

Understanding Risk Associated With Artificial Intelligence for Generalized Assessment and Decision Making: A Design Science Approach

Artificially Intelligent technology has great potential; however, contained within that potential are myriad threats and unintended consequences that range from minor inconveniences to global catastrophes. This research articulates a framework that can be used to quantitively evaluate existing, as well as emergent, artificially intelligent technologies from a perspective of risks associated with increased capability. Grounded in existing AI risk management and quantification methods and remedies, but using the Design Science Research methodology, we use quantitative measures to create a framework called Ordinal-W that provides a novel and intuitive way of “scoring” any risks associated with autonomous entities. This score can be used to evaluate emergent technologies, as well as to compare disparate technologies, and identify specific characteristics that may lead to potential risks prior to their introduction into mission-critical or life-threatening situations. The research identifies 24 dimensions of AI-enabled autonomous entities. These dimensions represent a set of collectively comprehensive, yet mutually exclusive, individual characteristics of AI-enabled technology for classification and scoring with an aim to finding the smallest set that still captures subtle variations in all AI-enabled technology. Each dimension is given an ordinal scale with a range of 0-9 to measure the amount of associated risk. Each dimension’s scale value is then combined to create an overall Ordinal-W Score for the entity. To account for the increased risk of AI at scale (from one low-risk entity to millions or billions of entities) the Score is logarithmic in nature with a range of 0-1000. In addition to the framework to evaluate risk, a second artifact is a web site and online app to demonstrate the framework’s use. Users can enter all the requisite information for scoring any entity. Over 100 entities were entered in the system during testing. The system allows any user to enter values for the twenty-four dimensions. It guides the user, providing descriptions and use cases along the way. Anyone can add an entity to the system and receive feedback about which of the twenty-four dimensions present greater risk, as well as how much risk is associated with any dimension when compared to others. This allows users of all levels of technical expertise to understand risks associated with AI that may otherwise have gone unnoticed. And it allows stakeholders at all phases of a development cycle to allocate research and other resources to proactively mitigate those risks. The Ordinal-W framework, as presented herein has been shown to facilitate conversations about the very complicated topic of AI risk management and mitigation. By necessity, a person needs an understanding of many factors when trying to evaluate the risks posed by individual AI entities. The dimensions and scales used in the framework create a frame of reference for users of all disciplines and technical capabilities. This frame of reference guides the user to deliberately think through how the various dimensions affect their entity. In the process, the system may introduce ideas or risks about which the user may not have considered. Also, results may suggest when a corporate or governmental agency needs to recalibrate their method of identifying or mitigating specific threats. While the system does not set a threshold as to when a given aspect of an entity becomes too risky, it does provide quantitative evidence of a condition. In the same way as two doctors may agree or disagree on a course of treatment after seeing an MRI image, this system cannot prescribe a recommended treatment, but it should provide concrete ideas and values with which to make informed decisions. Our research demonstrates that the more experience a person has with AI at home, in the office, or for medical purposes, the more they have concerns about its use. This research seeks to move the discussions of AI risk in a proactive direction. Private organizations as well as political groups can and should increase transparency and trust by addressing these concerns and explaining to consumers how their product or regulation mitigates the risks. This framework provides a means of calibrating those expectations for all stakeholders. All AI researchers, developers, proponents, and regulators have a vested interest in reversing the current trend of increased AI use correlating to increased concern.

Baseline review of Formal Methods and Foundations of Artificial Intelligence

. Artificial Intelligence (AI) is advancing rapidly, yet many successful models remain opaque and provide limited assurance about reliability, safety, and failure modes. This motivates renewed interest in formal methods and foundational perspectives that can support trustworthy AI beyond empirical testing. This paper presents a baseline review of the selected papers volume of the inaugural International Conference on Formal Methods and Foundations of Artificial Intelligence (FMF-AI 2025), published as Annales Mathematicae et Informaticae , Vol. 61 (2025). The goal is twofold: (i) to map and summarize the first FMF-AI “snapshot” as a starting point for the Hungarian research ecosystem, and (ii) to define a reproducible baseline that can serve as a reference for measuring topical and methodological shifts in subsequent FMF-AI editions. The review clusters the twenty selected papers into five thematic groups and records their relative prevalence. In addition, it introduces simple baseline metrics that can be recomputed in future FMF-AI editions to observe structural changes in the research landscape. The main pattern is a clear imbalance between verification-oriented contributions and papers that primarily use AI methods in application or optimization contexts.

News-Informed Probabilistic Models for AI Risk Analysis

The Emergence of Explainable Artificial Intelligence in Modern Decision Systems

Through the application of Artificial intelligence (AI), there has been the establishment of the technology as a cornerstone in current decision systems in essential fields like health, finance, transport, industrial automation, and the governance of citizens. Although traditional AI and machine learning frameworks, specifically deep learning models, have shown outstanding predictive performance, their nature is not enlightened by default, and as such, they have cast considerable doubt on the issues of trust, accountability, fairness, and regulatory compliance. It is thanks to this limitation that Explainable Artificial Intelligence (XAI) as a paradigm has emerged, aimed at ensuring that AI-driven decisions are easy to understand and interpret by the human stakeholders without major performance reduction. This paper is a thorough and stepwise analysis of how XAI was created in current decision systems. It starts with the historical contextualization of the development of AI, as an expert system driven by rules, to a data-driven black-box model and the increasing demand to explain decision-making processes. The paper provides a critical literature review of the current research on XAI methods classifying them into model-intrinsic and post-hoc methods of explanation, and discussing their relevance to various fields. An elaborate methodology is suggested, that incorporates explainability protocols into the AI choice channel, such as information pre-processing, model order, explanation creation and human-centered assessment. Additionally, the paper evaluates the experimental findings and case-based debates on how XAI enhances transparency, end-user trust, compliance with regulations, and system resilience. Popular explainability methods are also compared and evaluated including SHAP, LIME, saliency maps, and rule extraction. The results indicate that explainable models, in addition to increasing interpretability, can also help to improve debugging, bias detection and ethical AI deployment. The paper ends by recommending the current challenges, areas of open research, and future roles of XAI in the development of responsible and human-centered intelligent decision systems.