2026· Computers, Materials & Continua· 0 citations· 31 references
TL;DR
This study presents RAVE-Code, a three-layer framework that calibrates the verification effort based on the risk associated with each detected weakness, and demonstrates how consistent two ESBMC configurations rather than agreement with an independent standard can be.
Abstract
: Large Language Models (LLMs) are increasingly being used to generate source code. However, a substantial proportion of their output contains security vulnerabilities. Existing defenses typically apply uniform analysis to all code fragments, irrespective of their risk profiles. This study presents RAVE-Code, a three-layer framework that calibrates the verification effort based on the risk associated with each detected weakness. The Detection layer employs Bandit for pattern-based static analysis, annotating findings with their respective Common Weakness Enumeration (CWE) classes. The Risk Scoring layer calculates a composite risk score for each weakness instance by integrating the Common Vulnerability Scoring System (CVSS) severity, the prevalence of CWEs in AI-generated code, and actual exploitability data. The Verification layer directs each code fragment to an appropriate depth of Bounded Model Checking (BMC) using the Efficient SMT-Based Bounded Model Checker (ESBMC), with solver parameters customized for each CWE class. The framework is evaluated on three public benchmarks. On SecurityEval, Bandit identifies 28.5% of vulnerabilities with zero false positives, establishing a baseline for static analysis coverage. On FormAI, RAVE-Code achieves a detection rate of 86.0% ( F 1 = 0.909), compared to 73.8% ( F 1 = 0.834) for uniform verification, and uniquely detects 41 vulnerabilities that uniform analysis misses. Because FormAI’s labels are based on ESBMC, these results show how consistent two ESBMC configurations rather than agreement with an independent standard. On the Juliet Test Suite, which provides structurally independent ground truth, RAVE-Code achieves a detection rate of 57.3% ( F 1 = 0.614) compared to 40.8% ( F 1 = 0.479), detecting a total of 104 vulnerabilities. Applying all flags uniformly results in a detection rate 18.8 percentage points lower than the proposed per-CWE tailoring. This indicates that detection gains arise from routing rather than flag selection.
Findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.
Investigating the security vulnerabilities present in AI-generated JWT authentication code for Java Spring Boot Representational State Transfer Application Programming Interfaces (REST API) reinforces that AI-generated JSON Web Token (JWT) authentication code requires dedicated security review.
Hoang Long Nguyen, Mezid Hmudda, Benjamin Powley· 0 citations
The risk is not tied to any particular model but rather to the task category, and organizations should ask not which tool to trust, but instead whether LLM-generated automation code should be deployed without review.
AI-assisted development tools generate vulnerable code at significant rates, yet few automated mechanisms exist to detect, enrich, fix, and verify security issues at development velocity, particularly ones that ground remediation in real-world threat context. This paper presents an automated security evaluation pipeline that generates Python code from LLMSecEval prompts, scans for vulnerabilities using CodeQL and Bandit in parallel with an independent Code Validator LLM, enriches the Code Validator findings with MITRE ATT&CK techniques, CWE Observed Examples, and Python best practice guidelines, generates fixes via the Code Generation LLM, and re-scans with CodeQL and Bandit to verify outcomes. Two pipeline configurations were evaluated: Pipeline 1 (P1), using enriched Code Validator findings only, and Pipeline 2 (P2), where it additionally receives the initial CodeQL and Bandit findings. Both configurations were run across four Claude models: Opus 4.8, Sonnet 4.6, Sonnet 5, and Haiku 4.5, producing 80 runs against 26 LLMSecEval prompts covering 9 CWE categories. P1 reduced static analyzer findings across all four models, ranging from -9% (Opus 4.8) to -54% (Sonnet 5). P2 deepened these reductions further, ranging from -29% (Opus 4.8) to -69% (Haiku 4.5), with P2 outperforming P1 for every model. Verdict consistency averaged approximately 81% modal agreement across all configurations, with P2 marginally more stable than P1. Remediation introduced new vulnerabilities in 15-22% of cases: roughly 70% involved a single new finding, and P2 reduced churn for three of four models, with Sonnet 5 as the sole exception. Notably, the best Code Generation LLM (Opus 4.8) was not the best pipeline performer, as Sonnet 4.6 produced the lowest residual findings and highest pass rate after P2 remediation, suggesting that pipeline effectiveness and first-draft security are distinct properties.
The Security Mutation Score (SMS) is proposed, a metric that classifies mutant kills into semantic, functional, incidental, and crash categories using operator-aware heuristics, distinguishing genuine security awareness from coincidental detection.
Modern software systems are increasingly complex, and static analysis tools are commonly used to identify potentially vulnerable code by issuing warnings. However, these warnings often require manual inspection to confirm whether the reported issues are real, making the process time-consuming and error-prone. Directed fuzzing has emerged as a powerful automated technique to validate the warnings. However, applying it to the entire project in response to each warning is computationally infeasible, often requiring days of execution to achieve only incremental improvements in code coverage. We present SNIPTEST, an execution-based warning triage framework that generates and fuzzes compiled code slices centered around static-analysis warnings. Rather than proving exploitability in the full program, SNIPTEST provides evidence about how a warning behaves under progressively expanded sliced execution contexts. It employs a layer-by-layer slicing strategy, incrementally expanding context around the target location to validate potential vulnerabilities with increasing precision. We evaluate SNIPTEST on a benchmark of 97 true vulnerabilities and 97 false alarms across three real-world projects. SNIPTEST produces Possible True Positive evidence for 53 of 97 confirmed vulnerabilities (54.6%) by triggering the corresponding bug oracle consistently across all three analyzed slice levels, while the remaining cases are unreachable. Particularly, in 40.2% of these cases, it exploits the vulnerability along the observed execution path, matching the top three stack frames. On the 97 confirmed false alarms, SNIPTEST produces Possible False Positive evidence for 54 cases (55.6%) by reaching the warning without triggering the bug oracle, but misclassifies 28 cases (28.8%),and the remaining cases are unreached. Finally, we demonstrate the practical relevance of SNIPTEST by identifying CVE-2025-11964.
Aniruddhan Murali, N. Mathews, Mahmoud Alfadel et al.· IEEE Transactions on Softwar...· 0 citations