Jul 2026· Annual International Computer Software and Applications Conference· pp. 334-345· 0 citations· 55 references
Abstract
Context: The quality of software requirements is a critical determinant of success in mission-critical and large-scale software systems. Requirements smells, which indicate potential quality issues in software requirements, such as imprecision, ambiguity, and poor verifiability, can significantly impact project outcomes. Despite their crucial role in software projects, software requirement documents often contain inconsistencies, ambiguities, and conflicts that are challenging to detect and manage manually. Objective: We aim to develop a novel approach using data representation and a large language model (LLM) based approach to improve the detection of conflict and ambiguity requirements smells. Method: We construct domain-specific vector databases from software requirements and domain knowledge documents, where vectors represent embedded requirements and contextual domain information. These databases are populated offline and utilized during inference time for semantic similarity-based retrieval and smell detection. We compare our domain-augmented generation based approach with traditional reasoning-based methods for performance evaluation using diverse retrievers. Additionally, we include zero-shot and one-shot prompt techniques. Results: Our experimental results demonstrate that our RAGbased approach detects requirements smells across multiple datasets using five LLMs: Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o, o3-mini, and GPT-5. For conflict smell detection, compared to previous approaches on cross-datasets, these models have superior F1 scores, up from 0.566-0.804 to 0.676–0.857. The integration of domain knowledge showed mixed results, generally improving recall while decreasing precision and F1 scores. For ambiguity smell detection, the GPT-5 model achieved an F1 score of 0.783 compared to 0.762 of previous approaches. LLM-based approaches, besides GPT-4o, had better precision scores compared to previous approaches. Domain knowledge integration substantially increased the F1 score of the tested LLMs, with the most significant improvement observed in Claude 3.7 Sonnet, which rose from 0.526 to 0.677. In contrast, misleading or low-quality domain knowledge decreased F1 scores. Conclusions: The proposed RAG-based approach offers a promising automated solution for detecting requirements smells, demonstrating competitive performance compared to existing methods. Integrating domain knowledge enhances detection capabilities while highlighting the importance of balancing precision and recall.
Requirements elicitation is essential for developing interactive software systems, as it helps ensure that the resulting product meets stakeholder needs. Since elicitation typically relies on natural language (NL), misunderstandings can arise from its inherent ambiguity. Formal specifications can reduce ambiguity but require technical expertise. GUI prototyping therefore provides a valuable alternative by turning requirements into tangible visual artifacts that support communication, elicitation, and validation. However, creating high-fidelity prototypes remains time-consuming and costly. Similarly, requirements verification, which ensures that implementations conform to specified requirements, is still largely manual, while existing automated approaches are often limited to static, rule-based techniques. This work addresses two challenges: (C1) reducing the effort required to transform NL requirements into GUI prototypes, and (C2) reducing the effort required for requirements verification in GUI applications and prototypes. For C1, we introduce novel NL-based GUI retrieval and reranking methods, new benchmarks, and techniques for efficiently adapting LLMs to GUI generation, including proprietary GUI representations. Their effectiveness is demonstrated on a large benchmark with human annotations. For C2, we propose LLM-based methods for verifying semantically complex NL requirements on static GUI prototypes and introduce a multimodal LLM-based agent for verifying complex functional and non-functional requirements in dynamic GUI applications through automatically generated and evaluated interaction trajectories. Overall, the proposed methods substantially reduce manual effort in GUI prototyping and requirements verification.
— Use case and class diagramming are essential requirements engineering techniques that play a pivotal role in modeling software specifications and facilitating the software development process. However, software requirements are often expressed in Natural Language (NL), which can be ambiguous, noisy, immeasurable, and open to interpretation. This research addresses these challenges by automatically extracting the required elements to generate use case and class diagrams from software requirements documents written in natural language. Accordingly, an automated framework is proposed based on Natural Language Processing (NLP) techniques—such as tokenization and part-of-speech tagging—to parse the software requirements syntactically using a set of heuristic rules. These rules facilitate the extraction of actors, use cases, entities, relationships, and attributes required for generating the corresponding diagrams. Furthermore, to enhance the framework’s performance, the k-nearest neighbor (k-NN) algorithm is employed to predict previously processed requirements and reduce redundant computation. The framework’s effectiveness was evaluated using two performance metrics: recall and precision. Experimental results show that the proposed approach achieves an average recall of 96% and an average precision of 92%, confirming its robustness and reliability.
Thamer A. Alrawashdeh, Adnan Hnaif, Mustafa Alrifaee et al.· Journal of Communications So...· 1 citation
In fast-evolving software systems, effective 'natural language requirements parsing' and downstream change effect analysis capability across a multitude of codes represents low-hanging-fruit in this regard. We present a structured framework to deploy Large Language Models (LLMs) for automating two essential software engineering tasks, namely requirement interpretation and change impact analysis Utilizing the inherent understanding of semantics offered by transformer-based LLMs, the novel approach advances by converting vague and unstructured requirement documents into structured but machine-readable specifications to offer a direct traceability mapping from requirements to system components. Additionally, the framework leverages LLM-driven dependency analysis to predict and quantify how change effects percolate through connected modules which can minimize manual effort and human errors. This approach combines prompt engineering and retrieval-augmented generation (RAG) for domain-relevant accuracy plus fine-tuning techniques. On open-source and enterprise-grade software projects, experimental evaluations show that disambiguation accuracy, traceability precision, and change impact coverage of our approach are orders of magnitude better than state-of-the-art rule-based or static analysis tools. Notes: The results illustrate the application of LLMs at scale and demonstrate how these can alter software engineering workflows by removing bottlenecks (at a massive scale) at different stages of the software development lifecycle. In this research, we provide a generalizable pipeline that helps to bridge the gap from NLP advancements into practice for software lifecycle management.
Nithya Krishnan, Kumaran Ramanujam, Suresh Babu Narra et al.· 2026 International Conferenc...· 0 citations
LLM-based software (LBS) integrates large language models as core components to deliver flexible, personalised responses. Unlike traditional software with deterministic outputs, LBSs exhibit context-dependent, stochastic behaviour that renders classical acceptance testing and test oracles insufficient: the same query may require fundamentally different responses depending on user personas and software context. This gap creates an urgent need for automated acceptance testing frameworks that autonomously interpret user instructions, while reliably inferring user intentions in a changing environment. In this paper, we present an automated acceptance testing framework for LBS with calibrated verdict reliability via two technical contributions. First, we introduce Requirements-Augmented Generation (REAG), which interprets user intentions by retrieving relevant software requirements, domain knowledge, and personas via adaptive RAG and self-reasoning to generate context-aware test oracles. Second, recognising that oracle generation may retrieve irrelevant constraints, misinterpret intent, or hallucinate requirements, we introduce a confidence-calibrated cascade judgment. This method quantifies verdict reliability via simulated expert agreement -- accepting high-confidence verdicts, escalating ambiguous cases, or abstaining when uncertain -- with empirical reliability guarantees backed by conformal risk control. An industrial case study on a production nutrition advisory application demonstrates that REAG achieves a 3.91/5 oracle quality score, reaching qualified or marginal oracle quality in 82% of cases. The confidence-calibrated cascade achieves 98.8% accuracy, improves oracle quality from 3.91 to 4.30 by filtering unqualified outputs, and delivers a 31.7% cost-efficiency improvement over single-judge baselines, validating industrial viability
Fanyu Wang, Chetan Arora, Zhenping Xie et al.· 0 citations
A defect database is a structured collection of information about software bugs, errors, and their resolutions throughout the development lifecycle. These databases have valuable information for improving software quality, but their growing volume and complexity make manual analysis inefficient and error-prone. This paper presents an empirical study using real-world data investigating the feasibility of Retrieval-Augmented Generation (RAG) to support interactive exploration of software defect repositories. The study is structured with reference to selected phases of the CRISP-DM framework, which was used as an organizational guideline. Using a real-world software defect database as the empirical basis, the approach automates insight extraction from defect repositories, aiming to improve decision-making in quality assurance and software development by enabling scalable and intelligent exploration of software failures.
Pedro Lucas de Lavor Farias, Levi Almeida da Silva, P. A. M. Oliveira et al.· Revista Eletrônica de Inicia...· 0 citations
Artificial intelligence (AI) is transforming requirements elicitation: machine learning, natural language processing (NLP), and large language models (LLMs) now identify software requirements automatically from the textual data that surrounds every project—user feedback, specifications, regulations, and stakeholder transcripts. This paper presents a systematic mapping study of 74 peer-reviewed primary studies on AI-based automated requirements elicitation published between 2021 and 2025, identified from five databases following PRISMA 2020 and classified by AI technique, textual source, elicitation activity, and application domain. The evidence is divided into two equally sized source families—user feedback and agile artefacts versus formal documentation—each coupled to the AI techniques that suit its signal profile. Fine-tuned transformer encoders set the performance ceiling and, task-for-task, still outperform far larger generative models, while LLMs extend elicitation to long regulatory documents, multilingual feedback, and structured outputs. The central finding concerns automation depth. AI identifies requirements with consistently high accuracy (routinely F1 0.8 and above), but automation thins at every subsequent step: 51% of approaches structure what they identify, 23% consolidate them, and only 8% engineer stakeholder validation into the loop. This leaves the steps that turn candidates into agreed requirements largely manual. Benchmark fragmentation (77% custom datasets), thin industrial validation (14%), and skewed non-functional coverage compound this gap. The resulting map gives researchers an evidence-derived agenda for deepening automation, and practitioners guidance on which techniques the evidence supports for each elicitation task and textual source.
Safaa Eltahier, S. Al-Ghuribi, Mawal A. Mohammed et al.· Information· 0 citations
