Findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.
Abstract
Large language models (LLMs) frequently generate code with security vulnerabilities, yet these weaknesses are rarely isolated: they often span multiple concern areas simultaneously, reflecting the cross-cutting nature of security in software. We present a framework that combines security-oriented Metamorphic Relations (MRs) with Association Rule (AR) mining to detect vulnerabilities in LLM-generated code, uncover their co-violation structure, and trace that structure back to prompt-level risk factors. We define nine MRs covering major CWE categories, including SQL injection, XSS, command injection, path traversal, hard-coded credentials, weak cryptography, and memory-safety errors, and apply them using an LLM-based judge to 3,700 code snippets generated by five open models from the LLMSecEval benchmark. The results show that 68.8% of snippets violate at least one MR, with hard-coded credentials (79.1%) and command injection (74.4%) among the most prevalent applicable failures. AR mining reveals strong cross-cutting co-violation patterns, notably that XSS and weak cryptography co-violations predict hard-coded credentials with 82.5% confidence (lift = 3.23), along with tightly coupled clusters linking authentication, credential handling, and cryptographic weakness, as well as input-handling and memory-safety failures. We then perform prompt-level risk analysis and find that database- and authentication-related prompts are strong predictors of broad cross-cutting insecurity, while 65.5% of prompts yield consistent violation outcomes across all five models. These findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.
The Security Mutation Score (SMS) is proposed, a metric that classifies mutant kills into semantic, functional, incidental, and crash categories using operator-aware heuristics, distinguishing genuine security awareness from coincidental detection.
This work systematically generates syntactic variants of security-relevant code generation prompts and evaluates their impact on code security across multiple open LLMs and programming languages, identifying prompt syntax as a concrete security control surface and providing actionable guidance for reducing vulnerability risk in LLM-assisted development.
Matteo Cicalese, Antonio Della Porta, Stefano Lambiase et al.· 0 citations
Large language models (LLMs) can rapidly generate web user interface (UI) code for intelligent multimedia applications, but the resulting code may contain avoidable client-side security weaknesses. This paper presents a security governance framework that combines pre-generation rule embedding, ESLint-based static analysis, structured feedback, and LLM-based repair. The framework was evaluated through a controlled pilot ablation study involving 10 multimedia UI requirements, five experimental groups, and three repeated runs, producing 150 requests. Across all groups, 132 outputs passed structural and task-specific validity checks, while 18 were recorded as generation failures rather than being treated as secure code. For the Full Framework, the matched-pair end-to-end static finding reduction rate was 95.16% (SD = 4.76 percentage points), the valid output rate was 90.00%, and the static-secure valid output rate was $\mathbf{7 6. 6 7 \%}(\mathbf{S D}=$ 11.55 percentage points). Holm-adjusted paired Wilcoxon tests showed statistically significant differences, with lower observed final finding counts for the Full Framework than for Vanilla, Prompt Only, and the variant without structured static feedback. Repair-enabled configurations increased the average runtime from approximately 51 s to 79 s. These results concern static candidate findings and output validity; they do not replace server-side controls or exhaustive dynamic security testing.
Shu-Ning Ping, Guangchi Liu· 2026 International Conferenc...· 0 citations
This study presents RAVE-Code, a three-layer framework that calibrates the verification effort based on the risk associated with each detected weakness, and demonstrates how consistent two ESBMC configurations rather than agreement with an independent standard can be.
Maher Alharby, Ali Alssaiari· Computers, Materials & C...· 0 citations
: As modern software systems grow in complexity, the number of vulnerabilities has grown rapidly, rendering traditional security strategies—including manual code reviews and rule-based static analyzers—increasingly resource-intensive and unable to keep pace with emerging threats. This paper presents a hybrid pipeline combining static vulnerability candidate selection using Latent Semantic Analysis with dynamic verification through Large Language Model-assisted fuzzing. The approach identifies semantically similar code patterns to known vulnerabilities without exact syntactic matches, and automates the generation of fuzzing harnesses and seed inputs to overcome coverage plateaus. We evaluate our pipeline on real-world C libraries (LibTIFF, libexpat, FreeType) with documented vulnerabilities. Results demonstrate that vulnerability candidates are ranked within the Top 3–8% of all functions, and model-generated artifacts yield a 5-fold speedup over random fuzzing. The LLM Gemini 3 Pro achieves the highest harness generation success rate at 83% (Pass@3). Our analysis reveals fundamental limitations: the mixed-vector problem dilutes vulnerability signals in multifunctional code, safety alignment mechanisms can inadvertently neutralize vulnerabilities through defensive code insertion, and static feature extraction without build context achieves only 19.93% recall. The pipeline reduces manual effort while still requiring human validation.
Kevin Nguyen, Dominik Schoop· Proceedings of the 23rd Inte...· 0 citations
The risk is not tied to any particular model but rather to the task category, and organizations should ask not which tool to trust, but instead whether LLM-generated automation code should be deployed without review.
