CodeVulReason: Incentivizing reasoning for code vulnerability detection

Large Language Model Vulnerabilities

SecVulEval: Context-Aware Benchmarking of LLMs for Vulnerability Detection

Large Language Models (LLMs) show promise for vulnerability detection, but their evaluation is limited by the lack of high-quality benchmarks. Most existing datasets rely on coarse function-level labels, overlook fine-grained vulnerability patterns, and lack critical program context such as data/control dependencies. They also suffer from data quality issues, including mislabeling and duplication, leading to unreliable evaluation and limited real-world relevance. To address these limitations, this paper introduces SecVulEval, a context-aware benchmark designed to evaluate LLMs on vulnerability detection with rich contextual information. SecVulEval focuses on real-world C/C++ vulnerabilities at the statement level. This granularity enables more precise evaluation of a model’s ability to localize and understand vulnerabilities, beyond simple binary classification at the function level. By incorporating rich contextual information, SecVulEval sets a new standard for benchmarking vulnerability detection in realistic software development scenarios. This benchmark includes 25,440 function samples covering 5,867 unique CVEs in C/C++ projects from 1999 to 2024. We evaluated state-of-the-art LLMs in both standalone and multi-agent settings. Results on our dataset indicate that current models remain far from accurately identifying vulnerable statements within a given function, although agent-based approaches provide modest but promising improvements. The best-performing Claude-3.7-Sonnet-driven agent achieves an F1-score of 23.83% for vulnerable statement detection. We believe this benchmark can serve as a foundation for advancing context-aware vulnerability detection with LLMs.

Detecting Vulnerability-Inducing Commits via Multi-Stage Reasoning with LLM-Based Agents

VIC-RAGENT is proposed, an LLM-based multi-agent framework for effective and explainable vulnerability detection that employs a multi-stage reasoning process that progressively refines candidate vulnerabilities through preliminary inspection, reanalysis, and a final decision stage.

BashCoder-R1: Towards Robust and Explainable Bash Script Generation with Robustness-Aware Group Relative Policy Optimization

Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Task

Background: Large Language Models (LLMs) are increasingly being applied to Software Engineering (SE) tasks, achieving high accuracy across problems such as clone detection, vulnerability prediction, and code summarization. However, their high computational demands and energy consumption raise sustainability concerns and hinder their use on consumer hardware and resource-constrained platforms. A common way to report the computational cost of an LLM in the literature and industry is to use the number of Floating Point Operations (FLOPs) required to perform a pass over the network. Aims: This paper investigates the implications of energy-aware knowledge distillation for SE, aiming to improve model efficiency while maintaining performance and to determine whether FLOPs is a reliable energy-aware metric. Method: We conduct a controlled experiment using Morph, a Many-Objective Optimization-based distillation methodology, to empirically examine whether FLOPs accurately reflect energy consumption in Clone Detection and Vulnerability Prediction tasks. We extend this methodology to include energy-surrogate models that directly estimate CPU and GPU energy consumption during optimization, and we apply Morph to generative tasks using CodeT5+ for code summarization. Results: Our results show that FLOPs is not always a reliable indicator of energy consumption, and better results can be achieved by using energy-surrogate models. Distilled student models can reduce inference energy consumption by up to 90\% and memory usage by 86\%, with only modest accuracy trade-offs. Conclusions: Energy-aware knowledge distillation when guided by direct energy surrogates rather than FLOPs can improve the energy consumption, sustainability, and deployability of LLMs for SE applications, enabling efficient models on consumer hardware.

Function-Level Execution Feedback for Code Preference Optimization

STEP-KTODER is proposed, a framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels via automatically generated unit tests and shows that execution-based labels are essential.