Oct 2026· Applied Sciences· 0 citations· 40 references
TL;DR
STLC Manager is presented, an LLM-supported framework that transforms requirements, UML/XML models, and source code into structured test scenarios and test cases and supports Serial, Bulk, and Parallel strategies for semantic redundancy reduction and shows greater configuration sensitivity for Parallel under the evaluated protocol.
Abstract
Large Language Models (LLMs) can generate software tests from heterogeneous artifacts, but semantic redundancy and stochastic output variability complicate reliable pre-execution refinement. This paper presents STLC Manager, an LLM-supported framework that transforms requirements, UML/XML models, and source code into structured test scenarios and test cases and supports Serial, Bulk, and Parallel strategies for semantic redundancy reduction. Validation combines archived cross-model evidence with a controlled local study of 60 runs over two fixed 64-case suites, three installed models, partitioned local Bulk and Controlled Local Parallel, and five repetitions per configuration. The evaluation also reconstructs a three-rater human consensus and includes a deterministic TF–IDF reference. In the controlled study, mean Test Redundancy Ratio ranged from 0.0063 to 0.5000 for partitioned Bulk and from 0.1250 to 0.6375 for Parallel. Item-level modal agreement ranged from 98.44% to 100% for Bulk and from 75.31% to 97.19% for Parallel, while mean artifact-scoped Scenario Coverage Preservation ranged from 0.900 to 1.000. The results show greater configuration sensitivity for Parallel under the evaluated protocol. However, the workloads differ and do not support a causal ranking of the strategies. The implementation is publicly available. The conclusions are limited to pre-execution semantic refinement and do not establish executability, coverage, mutation adequacy, or fault detection.
DSL-Bench is presented, an executable benchmark and analysis framework for cross-DSL code generation and capability profiles and DSL-Profiler summaries for interpreting data-scarce DSLs are derived.
Zi-Han Zhou, Ming-Xi Ye, Yan Gao et al.· 0 citations
Large language models (LLMs) increasingly generate executable software, but functional correctness alone does not establish production readiness. This study evaluates GPT-5.6 Sol, Terra, and Luna across 15 algorithmic and production-oriented tasks using five generations per model-task pair, yielding 225 generated progr...
Ramesh Venkatachalapathy· International Journal of Sof...· 0 citations
Large language models (LLMs) have recently shown promise in automating formal specification generation, but their effectiveness has mainly been demonstrated on self-contained benchmark tasks. This paper examines the changes that occur when LLM-generated specifications are applied to an industrial verification project....
Du-Gang Liu, Ding Wang, Chao Jiang et al.· Proceedings of the 41st IEEE...· 1 citation
Reliable assessment of LLM-generated tests should treat executability as a gate and combine coverage with mutation testing and structural quality indicators, and in practice, model selection should precede prompt tuning.
Bilal I. Al-Ahmad, M. Harshvardhan, Khaled El-Fakih et al.· arXiv.org· 0 citations
Large language models (LLMs) can generate register-transfer-level (RTL) code from natural-language specifications, but compilation alone does not establish structural completeness, functional correctness, or implementation efficiency. This paper presents a framework for evaluating LLM-generated RTL beyond compilation,...
Hepziba Susan, R. ShivaranjaniG, Malik Imran et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.