Semi-Automated Requirement Refactoring for INCOSE Guideline Compliance

Abstract

As requirements are typically written in natural language, they may be well-suited to refactoring with large language models (LLMs). However, asking LLMs to improve requirements naively yields unsatisfactory results. While some quality issues in the requirements can be solved that way, new issues are introduced and LLM hallucinations add redundant or incorrect information. To overcome these challenges, we present an approach for semi-automated refactoring of requirements, supported by LLMs. Based on the INCOSE Guide to Writing Requirements, an industry standard, our tooling semi-automatically improves the requirements while preserving their original semantics. In cases of ambiguity or missing information, the human requirements engineer is automatically asked to provide the necessary details. We evaluate this approach for improving 75 requirements we had manually labelled as violating or not violating each of the INCOSE rules. The results show a 59% decrease in the mean number of violated INCOSE rules per requirement. Furthermore, we found no semantic changes in 92% of the refactored requirements generated by our system. By keeping the human in the loop, we can improve requirements while ensuring both minimal risk of hallucinations and maximal automation, at a reasonable cost.