Aug 2026· Bulletin of Electrical Engineering and Informatics· Vol 15, pp. 3268-3279· 0 citations· 42 references
TL;DR
A dual-LLM pipeline integrating lightweight specification generation with reasoning-oriented code synthesis is presented, establishing a robust foundation for scalable, artificial intelligence-driven automation in software engineering.
Abstract
Manual generation of unified modeling language (UML) diagrams creates bottlenecks in agile development due to inconsistency and labor intensity. While large language models (LLMs) offer generative capabilities, existing solutions suffer from data scarcity and inadequate evaluation tools. To address this, we present a dual-LLM pipeline integrating lightweight specification generation with reasoning-oriented code synthesis. Uniquely, this framework employs a weighted multimodal validation module utilizing diverse vision-language models (VLMs) to assess diagrammatic fidelity. We further address the data shortage by releasing benchmark datasets comprising 5,000 class and 3,000 use case diagrams. Empirical results demonstrate a 95.8% rendering success rate for class diagrams and strong semantic alignment for use case models. By mitigating structural and behavioral reasoning conflicts, this research provides a replicable architecture and rigorous assessment methodology, establishing a robust foundation for scalable, artificial intelligence-driven automation in software engineering.
BRIDGE is presented, a structured prompting framework that decomposes verification into three interconnected domains: Code (implementations), Specifications (formal intent), and Theorem State-ments (constructive correctness claims), and elicits domain-specific intermediate reasoning to connect them.
Robert Joseph George, Carson Eisenach, Udaya Ghai et al.· 0 citations
Background: Bridging the gap between requirement analysis and system design remains a critical challenge in software engineering. Existing Problem Frames (PF)–based approaches lack automated mapping mechanisms to Unified Modeling Language (UML) models, resulting in requirements-design disconnection, inefficient manual transformation, and insufficient traceability. Methods: We propose PF2UML, a model-driven transformation method that automatically converts PF specifications into comprehensive UML requirement models. The approach comprises three components: (1) an extended PF meta-model with semantic annotations addressing structural, behavioral, and servicelevel gaps; (2) a graphical modeling platform built on Eclipse Modeling Framework (EMF) and Sirius following the Meta-Object Facility (MOF) three-layer architecture; and (3) 24 declarative transformation rules implemented in Atlas Transformation Language (ATL), generating four UML diagram types: conceptual class diagrams, use case diagrams, service diagrams, and SSDs. Results: Validation on five representative opensource systems with 12 participants demonstrates a transformation accuracy of 93.6% (130% improvement over baseline), an average reduction in end-to-end time from Requirements Modeling and Model Transformation (RM+MT) of 26.8%, and a practical average ROI of 236.6%. Conclusion: PF2UML establishes seamless and traceable integration from requirements modeling to system design, significantly reducing manual effort and improving consistency for complex interdisciplinary system development.
Hongbin Xiao, Zhi Li, Yilong Yang et al.· International journal of sof...· 0 citations
This paper addresses automated unit test generation with large language models (LLMs). LLM-based test generation has not yet attained a quality level sufficient for practical use in industry. Although LLMs often reproduce API syntax faithfully, they frequently disregard semantic usage constraints and execution-environment dependencies, leading to assertion failures, mock-related errors, and reference/resolution errors. A prior failure analysis of Java unit test generation using GPT-4o classified 2980 trials into eight failure patterns and identified three root-cause mechanisms: external context ignorance, internal context ignorance, and a syntax–semantics gap. Building on that analysis, this paper proposes a prompt design comprising three strategies: (1) making the execution state explicit in the generated test, (2) stating semantic constraints explicitly, and (3) injecting environment constraints prior to generation. In contrast to generic techniques such as few-shot learning or chain-of-thought prompting, each proposed strategy is tied to a specific root-cause mechanism, yielding a systematic design in which each rule is explicitly justified by its correspondence to a specific root-cause mechanism. Experiments on 298 methods with five models (GPT-4o, GPT-5, GPT-5.1-Codex, Claude Sonnet 4.5, and Gemini 2.5 Pro) show improved test execution success rates for every model, with absolute gains ranging from 1.1 to 21.1 percentage points (pp). Mock-related errors were reduced by 61.9%–99.2% relative to the baseline prompt, demonstrating effectiveness against the targeted failure patterns. Finally, conditions under which the strategies transfer to other code-generation tasks are discussed, along with limitations on their scope.
This work proposes TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics, and demonstrates the effectiveness of TraceDev in repository-level code generation from requirements.
Mingyu Chen, Yakun Zhang, Zihao Xie et al.· 0 citations
Conceptual modeling supports the design, analysis, and communication of the properties of complex systems, yet conceptual models can be difficult to understand when domain-level abstractions must be encoded through low-level constructs required mainly for semantic conformity. Prior work has mainly improved how existing individual constructs are visually represented. We shift the focus from individual constructs to recurring configurations of constructs, and propose abstract notation engineering as a language-agnostic method for replacing such configurations with higher-level, semantically transparent constructs. The method comprises pattern identification, pattern formalization, visual notation design, and empirical validation. We instantiate it for Dynamic Condition Response (DCR) graphs, where common workflow patterns require elaborate low-level configurations. The resulting extension, DeCleaR, replaces such configurations with compact pattern-based abstractions. The results of our empirical validation show that DeCleaR improves perceived empirical quality, pragmatic quality, and user preference over standard DCR graphs.
Amine Abbad-Andaloussi, Daniel Jeppe Schütt, Kasper Solhøj Jørgensen et al.· 0 citations
Large Language Models (LLMs) are predominantly assessed based on their common sense reasoning, language comprehension, and logical reasoning abilities. While models trained in specialized domains like mathematics or coding have demonstrated remarkable advancements in logical reasoning, there remains a significant gap in evaluating their code generation capabilities. Existing benchmark datasets fall short in pinpointing specific strengths and weaknesses, impeding targeted enhancements in models’ reasoning abilities to synthesize code.
To bridge this gap, this thesis introduces two novel contributions: CodeEval and CodeQual. CodeEval is an innovative, pedagogical benchmarking method that mirrors the evaluation processes encountered in academic programming courses. It comprises a multi-dimensional benchmark dataset of 602 hand-crafted problems designed to rigorously evaluate LLMs across 24 distinct aspects of Python programming, covering three proficiency levels—beginner, intermediate, and advanced—and includes both class-based and function-based problem types with detailed problem specifications and comprehensive test suites achieving 99.1% coverage. To facilitate widespread adoption, we developed RunCodeEval, an open-source execution framework that provides researchers with a ready-to-use evaluation pipeline. Our evaluation of 15 state-of-the-art LLMs revealed consistent performance degradation with increasing complexity (validated statistically, Cohen’s d = 0.790) and universal struggles with advanced concepts like concurrency.
Code quality is inherently subjective, encompassing dimensions like readability, efficiency, and adherence to language idioms that traditional static metrics fail to capture adequately. While large language models can assess these subjective qualities, lightweight models offer practical advantages: seamless CI/CD pipeline integration, lower operational costs, and full control over model behavior. We investigate whether such models can learn to assess code quality by training on synthetic LLM annotations. We introduce CodeQual, a dataset of 5,819 code samples derived from five established sources spanning diverse domains—competitive programming, pedagogical problems, software engineering, and general benchmarks—scored by LLMs across five quality dimensions, with 655 human-annotated samples for evaluation. Our fine-tuned model, CodeQualBERT, not only matches LLM performance but exceeds inter-human agreement on all five dimensions, achieving 16–100% improvement over the inter-human agreement baseline.
Together, these contributions provide a comprehensive framework for evaluating and improving LLMs in software engineering contexts, encompassing both functional correctness assessment and subjective code quality evaluation.