Recent multimodal large language models (MLLMs) have achieved remarkable performance on visual question answering (VQA) and multimodal reasoning tasks. But one overlooked failure case stubbornly persists: when answers require simultaneous synthesis of evidence from natural images, free-form text, and structured tables, present models suffer from trimodal hallucination—generating content unattributed to any of the input modalities. Prior approaches to hallucination mitigation focus on image-text pairs, and existing retrieval-augmented generation (RAG) systems for multimodal settings do not readily include supporting evidence and often lack modality-attributed explainability. In this paper, we present MXRAG (Multimodal Cross-Modal Explainable Retrieval-Augmented Generation), a new approach to this trimodal evidence problem with three key innovations that work in concert: (1) a Trimodal Evidence Retriever (TER) that retrieves image patches, text passages, and table rows jointly using a shared semantic manifold; (2) a Cross-Modal Attribution Network (CMAN) that computes fine-grained, token-level attribution scores inline during generation, mapping each generated token to supporting evidence from all three modalities; and (3) a Hallucination-Aware Constrained Decoding (HACD) strategy that penalises generation steps with attribution entropy above a calibrated threshold, suppressing unsupported factual tokens at inference time. CMAN is trained with novel cross-modal attribution and modality-coherence losses using token-level gold annotations; HACD requires no additional training and is calibrated per dataset on the validation split. We cast joint retrieval-generation as a constrained variational problem over a trimodal evidence space and introduce MMTabQA, a new trimodal VQA benchmark derived from WikiTableQuestions and MSCOCO with 12,847 instances and token-level attribution labels. Evaluation on four benchmarks (MMTabQA, WebSRC, ChartQA, MIMIC-CXR-VQA) shows that MXRAG achieves state-of-the-art. +21.4% points (p.p.) exact match accuracy, − 38.6% relative reduction in hallucination rate (− 12.1 p.p. absolute) versus the best multimodal RAG baseline, and 89.4% modality coherence score. Ablation experiments confirm the contribution of each component, with CMAN providing the largest accuracy gain (+ 14.2%) and HACD the largest hallucination reduction (− 23.1%). A supplementary human evaluation on 200 instances confirms that the entropy-based hallucination metric tracks human judgement (human-judged HR: 21.3% vs. metric HR: 19.1%). MXRAG advances the state of the art for reliable, interpretable, evidence-based multimodal AI.
Various recent Artificial Intelligence (AI) system failures, some of which have made the global headlines, have highlighted issues in these systems. These failures have resulted in calls for more ethical AI systems that better take into account their effects on various stakeholders. However, implementing AI ethics into practice is still an on-going challenge. High-level guidelines for doing so exist, devised by governments and private organizations alike, but lack practicality for developers. To address this issue, in this paper, we present a method for implementing AI ethics. The method, ECCOLA, has been iteratively developed using a cyclical action design research approach. The method aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.
Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson· EUROMICRO Conference on Soft...· 64 citations· ⚡6
In this paper, we present a novel approach to improving software quality and efficiency through a Large Language Model (LLM)-based model designed to review code and identify potential issues. Our proposed LLM-based AI agent model is trained on large code repositories. This training includes code reviews, bug reports, and documentation of best practices. It aims to detect code smells, identify potential bugs, provide suggestions for improvement, and optimize the code. Unlike traditional static code analysis tools, our LLM-based AI agent has the ability to predict future potential risks in the code. This supports a dual goal of improving code quality and enhancing developer education by encouraging a deeper understanding of best practices and efficient coding techniques. Furthermore, we explore the model's effectiveness in suggesting improvements that significantly reduce post-release bugs and enhance code review processes, as evidenced by an analysis of developer sentiment toward LLM feedback. For future work, we aim to assess the accuracy and efficiency of LLM-generated documentation updates in comparison to manual methods. This will involve an empirical study focusing on manually conducted code reviews to identify code smells and bugs, alongside an evaluation of best practice documentation, augmented by insights from developer discussions and code reviews. Our goal is to not only refine the accuracy of our LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education.
Z. Rasheed, Malik Abdul Sami, Muhammad Waseem et al.· arXiv.org· 62 citations· ⚡3
In agile software development, maintaining high-quality user stories is crucial, but also challenging. This study explores the use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams. We developed a reference model for an Autonomous LLM-based Agent System and implemented it at the company. The quality of user stories in the study and the effectiveness of these agents for user story quality improvement was assessed by 11 participants across six agile teams. Our findings demonstrate the potential of LLMs in improving user story quality, contributing to the research on AI role in agile development, and providing a practical example of the transformative impact of AI in an industry setting.
Zheying Zhang, M. Rayhan, Tomas Herda et al.· International Conference on...· 48 citations· ⚡4
Systematic literature review (SLR) is foundational to evidence-based research, enabling scholars to identify, classify, and synthesize existing studies to address specific research questions. Conducting an SLR is, however, largely a manual process. In recent years, researchers have made significant progress in automating portions of the SLR pipeline to reduce the effort and time required for high-quality reviews; nevertheless, there remains a lack of AI-agent-based systems that automate the entire SLR workflow. To this end, we introduce a novel multi-AI-agent system designed to fully automate SLRs. Leveraging large language models (LLMs), our system streamlines the review process to enhance efficiency and accuracy. Through a user-friendly interface, researchers specify a topic; the system then generates a search string to retrieve relevant academic papers. Next, an inclusion/exclusion filtering step is applied to titles relevant to the research area. The system subsequently summarizes paper abstracts and retains only those directly related to the field of study. In the final phase, it conducts a thorough analysis of the selected papers with respect to predefined research questions. This paper presents the system, describes its operational framework, and demonstrates how it substantially reduces the time and effort traditionally required for SLRs while maintaining comprehensiveness and precision. The code for this project is available at: https://github.com/GPT-Laboratory/SLR-automation .
Malik Abdul Sami, Z. Rasheed, Kai-Kristian Kemell et al.· arXiv.org· 43 citations· ⚡2
Context: Manual qualitative data analysis is time-intensive and can compromise validity and replicability, affecting analysis design, implementation, and reporting. Large Language Models (LLMs) enable human-bot collaboration in Software Engineering (SE), but their potential for qualitative data analysis in SE remains largely unexplored. Objective: The objective of this study is to design and develop an LLM-based multi-agent system that synergizes human decision support with AI to automate various qualitative data analysis approaches. Methods: We used LLM-based multi-agents systems to assist the qualitative data analysis process, deploying 27 agents, each responsible for a specific task, such as text summarization, initial code generation, and extracting themes and patterns. Results: The main findings are: (1) the LLM-based multi-agent system accelerates the qualitative data analysis process, (2) the system effectively automates tasks such as text summarization, initial code generation, and theme extraction, and (3) the publicly accessible code facilitates validation and further evaluation. Conclusion: The proposed LLM-based multi-agent system automates qualitative data analysis process, creating opportunities for researchers and practitioners. Future improvements focus on enhancing multilingual performance and integrating continuous expert feedback. The source code of proposed system and system details can be found here: https://github.com/GPT-Laboratory/Qualitative-Analysis-with-an-LLM-Based-Agentts
Z. Rasheed, Muhammad Waseem, Aakash Ahmad et al.· arXiv.org· 40 citations
The growing influence and decision-making capacities of Autonomous systems and Artificial Intelligence in our lives force us to consider the values embedded in these systems. But how ethics should be implemented into these systems? In this study, the solution is seen on philosophical conceptualization as a framework to form practical implementation model for ethics of AI. To take the first steps on conceptualization main concepts used on the field needs to be identified. A keyword based Systematic Mapping Study (SMS) on the keywords used in AI and ethics was conducted to help in identifying, defying and comparing main concepts used in current AI ethics discourse. Out of 1062 papers retrieved SMS discovered 37 re-occurring keywords in 83 academic papers. We suggest that the focus on finding keywords is the first step in guiding and providing direction for future research in the AI ethics field.
Ville Vakkuri, P. Abrahamsson· International Conference on...· 39 citations· ⚡2
A new method, called CW-Net, translates the reasoning process of an autonomous vehicle’s AI system into understandable concepts that explain its behavior.