Jun 2026· International Joint Conference on Computer Science and Software Engineering· pp. 321-326· 0 citations· 11 references
Abstract
Traditional security auditing and compliance are resource-intensive manual processes. Recent work has applied Large Language Models to automate compliance tasks, but these approaches typically rely on textual inputs, limiting their applicability to real-world audit artifacts. This study proposes a Multimodal Large Language Model (MLLM)-based system for security requirements verification using evidence-grounded analysis of multimodal audit artifacts. The model is constrained to only consider artifacts provided at inference time. Evaluation is conducted through an industrial case study using real audit data, including artifacts, requirements, and ground truth labels. Results show good accuracy and explainability (up to 77%), high consistency, and high recall but lower precision. These findings demonstrate the potential of MLLMs to support automated security requirements verification in practical audit settings.
A partially-automated method for assessing LLM knowledge of a security area using authoritative information from Consumer Protection Agencies to identify instability in LLM responses that can be indicative of knowledge gaps is introduced.
Shufan Chai, Liangliang Sun, J. Staddon· 0 citations
A hybrid framework that takes a BPMN process model and a security requirements document as input and automatically generates security annotations adhering to the SecBPMN2 specification is presented, providing a scalable foundation for security-by-design BPM.
Empirical evidence is provided that generative AI can effectively support security requirements engineering when embedded within human-centered workflows and organizational governance structures, offering practical insights for adoption in regulated software development contexts.
F. Martins, Elaine Venson· SIGSOFT FSE Companion· 0 citations
This tutorial introduces a reusable, end-to-end evaluation pipeline grounded in empirical software engineering practices, focusing on post-generation validation rather than prompt design, allowing for validating AI-generated code in modern development workflows.
Vibe coding democratizes software development by allowing users to generate code via natural-language (NL) interaction with large language models (LLMs). However, the code is reliable only when it faithfully implements the user's intent, which is difficult and labor-intensive for users to validate. Existing validation methods either rely on LLM-assisted automated testing, which suffers from prompt ambiguity and model fallibility, or involve users only in partial software artifacts such as prompts and test cases, which may overlook corner cases and program details. Motivated by a bug study of LLM-generated code, we find that detailed human feedback is essential, as failures often stem from underspecified requirements or subtle semantic deviations. This paper presents verifiable literate programming (VLP), a human-in-the-loop framework designed to make the review/validation process of LLM-generated code accessible to users at all programming levels. At its core, VLP proposes unambiguous NL-based documentation as a readable intermediate layer between prompts and code. The documentation demonstrates concrete program semantics and enables users to provide feedback on potential intent-code mismatches. It supports human-involved, end-to-end repair and validation via three techniques: (i) an NL-style literate language with unambiguous syntax and mostly deterministic code-to-documentation translation, (ii) LLM-based fine-grained mismatch detection that uses trace links between prompts and documentation to focus users'review effort on suspicious documentation lines, and (iii) a verification module that leverages user-validated documentation to derive API-usage checks and formal properties, which are then verified against the generated code using model checking. Our evaluation shows that VLP improves code pass@1 from 28.7%-73.2% to 65.4%-93.5% with reasonable user effort.
Ziqi Yuan, Wenhao Lu, Haohang Wu et al.· 0 citations
An empirical benchmark evaluating whether LLMs and SLMs can generate security-compliant AWS Terraform configurations suggests that prompt design is a critical factor, highlighting the need for a proper pipeline for developing and validating LLM-assisted secure IaC generation.
Francis Luis Santos Vargas, R. Mansilha, Diego Kreutz· Anais do I Simpósio de Infra...· 0 citations