This study examines the processability of commissioning and testing specifications in natural language by proposing a pipeline designed to systematically transform these specifications into a machine-processable format and introduces a unified schema that serves as an input format for the large language models tasked with the transformation.
Abstract
In vehicle production, commissioning and testing processes of electric and electronic components are essential for value creation and quality assurance. The emergence of software-defined vehicles, however, leads to an increased scope and complexity of these processes as software functions depend on electric and electronic components for perception, execution, and processing tasks. In this context, this paper tackles a common challenge: Software that is deployed in vehicle production to implement commissioning and testing processes is developed upon specifications that define prerequisites, procedures, and target results in natural language. Therefore, extensive human interpretation and manual translation into executable code are needed being susceptible to errors as well as time-consuming. The large number of vehicle configurations and rapid changes in vehicle software further complicate the development of commissioning and testing software, particularly as verbose textual dependency descriptions risk impairing comprehensibility. Machine-processable specifications facilitating automated validation and code generation or direct execution could consequently ensure consistency, reduce manual effort, and accelerate the development process. For this purpose, we examine the processability of commissioning and testing specifications in natural language by proposing a pipeline designed to systematically transform these specifications into a machine-processable format. In particular, we introduce a unified schema that serves as an input format for the large language models tasked with the transformation. Subsequently, several large language models are evaluated in practical trials, based on their ability to translate commissioning and testing specifications into a machine-processable notation. In summary, this study aims to enable more efficient and data-driven software development based on textual requirements. This work offers valuable insights into the suitability and applicability of large language models within the planning of automotive commissioning and testing processes, targeting enhanced automation and efficiency.
In the context of industrial digitalization and the widespread adoption of process automation systems, Knowledge Management Systems (KMS) play a key role in providing operational personnel with up-to-date instructions and regulations. However, the inherent ambiguity of natural language in technical documentation remains a serious obstacle, leading to incorrect operator actions, process deviations, and increased safety risks. This article investigates the integration of Large Language Models (LLMs) into KMS and its impact on user experience and human–machine interaction in industrial automation environments. A method called Semantic Latent Choice Detection is presented, designed to systematically identify interpretation ambiguities in process instructions and operator commands. Unlike existing approaches that require access to the internal model architecture (“white box”) or token-level logits, the proposed method is logit-free and operates with closed commercial LLMs (“black box”) via standard API interfaces. The method analyzes the semantic similarity of binary text blocks and polysemous terms within the context of a specific technological process. Using a metallurgical production case study, we demonstrate how the system detects hidden semantic collisions (e.g., the difference between “adding ferroalloys into the ladle” and “feeding ferroalloys onto the conveyor”) that are missed by traditional rule-based validation methods. Instead of arbitrarily selecting an interpretation, the system initiates a clarification request to the human operator, thereby reducing cognitive load, preventing erroneous automated decisions, and increasing trust in the KMS. An empirical evaluation conducted in a real-world industrial setting (unit control rooms and dispatch centers) shows a statistically significant reduction in errors related to misinterpretation of process regulations. The article contributes to the fields of automation engineering, knowledge management, and human-centered automation by proposing a novel method for validating operational instructions in high-risk industrial environments.
Viktor A. Vedeneev, V. Kondratiev, K. Suslov et al.· Automation· 1 citation
In fast-evolving software systems, effective 'natural language requirements parsing' and downstream change effect analysis capability across a multitude of codes represents low-hanging-fruit in this regard. We present a structured framework to deploy Large Language Models (LLMs) for automating two essential software engineering tasks, namely requirement interpretation and change impact analysis Utilizing the inherent understanding of semantics offered by transformer-based LLMs, the novel approach advances by converting vague and unstructured requirement documents into structured but machine-readable specifications to offer a direct traceability mapping from requirements to system components. Additionally, the framework leverages LLM-driven dependency analysis to predict and quantify how change effects percolate through connected modules which can minimize manual effort and human errors. This approach combines prompt engineering and retrieval-augmented generation (RAG) for domain-relevant accuracy plus fine-tuning techniques. On open-source and enterprise-grade software projects, experimental evaluations show that disambiguation accuracy, traceability precision, and change impact coverage of our approach are orders of magnitude better than state-of-the-art rule-based or static analysis tools. Notes: The results illustrate the application of LLMs at scale and demonstrate how these can alter software engineering workflows by removing bottlenecks (at a massive scale) at different stages of the software development lifecycle. In this research, we provide a generalizable pipeline that helps to bridge the gap from NLP advancements into practice for software lifecycle management.
Nithya Krishnan, Kumaran Ramanujam, Suresh Babu Narra et al.· 2026 International Conferenc...· 0 citations
The results indicate that language-model assistance can add value in regulated automotive requirements workflows when it is embedded in deterministic, provenance-rich, and expert-governed processes.
This work proposes TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics, and demonstrates the effectiveness of TraceDev in repository-level code generation from requirements.
Mingyu Chen, Yakun Zhang, Zihao Xie et al.· 0 citations
Enterprise regression testing for microservice systems is often constrained by incomplete or outdated documentation. In practice, QA engineers frequently rely on real execution traffic to reconstruct business scenarios; however, turning raw traffic into replayable regression tests with stable validation logic remains labor-intensive and error-prone. This paper presents NL2Test, an end-to-end approach and tool that generates executable API regression tests from (i) a natural-language scenario description and (ii) a traffic capture recorded while executing the scenario. NL2Test addresses two coupled tasks: test case carving, which extracts a minimal replayable request sequence and reconstructs data dependencies so that dynamic values are bound from their responses rather than hard-coded; and assertion generation, which produces assertions aligned with business intent while avoiding non-deterministic fields and hallucinated paths. To improve reliability, NL2Test uses LLMs for semantic interpretation and constrained code synthesis, and uses deterministic algorithms for request filtering, dependency confirmation via value consistency, and assertion-path validation. We evaluate NL2Test on 51 industrial regression scenarios extracted from a large consumer-facing Internet company. NL2Test achieves an exact-match rate of 82.4% (42/51), and produces a functionally usable draft in 98.0% (50/51) of scenarios when allowing minor post-edits. In a 9-month production deployment starting in March 2025, NL2Test generated 3,196 test cases with an overall code adoption rate of 85.4%. These results indicate that traffic-grounded generation with deterministic guardrails can substantially reduce manual effort while improving regression automation in complex microservice environments.
Haozhen You, Zhen Dong, Jingjing Wang et al.· 0 citations
The process of authoring a Software Requirements Specification (SRS) document is a resource-intensive task in software development that requires coordination among multiple stakeholders and is often time-consuming, costly, and prone to human error. Latest advancements in artificial intelligence have enabled the generation of specification documents using Large Language Models (LLMs). However, such approaches still depend on manual prompt engineering and prompt optimization to extract relevant knowledge and do not consistently ensure structural coherence, completeness, and reliability. This paper presents a systematic approach to SRS generation in which input requirements from stakeholders are classified into semantically meaningful topics, followed by the construction of an initial skeleton document based on these topics. The document is then incrementally expanded using reinforcement learning to improve consistency, completeness, and coverage. The proposed approach also improves the dependability of the output by reducing hallucinations that may arise from the unstructured, raw nature of user inputs. The experimental evaluation of the proposed framework increases topic classification accuracy from 0.40–0.70 to 0.75–0.95 across six requirement topics, thereby improving document structure and generation quality. Compared with a ChatGPT Model 5.2 baseline, the framework achieved significant improvements in key text-generation metrics, including a 19.8% increase in ROUGE-L and an 11.8% increase in METEOR, while maintaining contextual relevance with an average semantic cross-similarity score of 0.615. These results indicate that the proposed method can produce SRS documents that are contextually reliable and structurally coherent while requiring less manual prompts such as requirements from analysts or stakeholders.
Muhammad Ali Akhtar, Raheela Asif· Information· 0 citations