Aug 2026· Proceedings of the ACM on Software Engineering· Vol 3, pp. 2225 - 2247· 0 citations· 60 references
Computer Science
TL;DR
This work proposes EAVA, a novel framework that effectively leverages LLMs to perform SV assessment and provide supporting evidence and enables large-scale reasoning trajectory annotation using off-the-shelf LLMs and adopts a two-stage training paradigm.
Abstract
Software Vulnerability (SV) assessment is a vital phase in SV management, which characterizes discovered SVs to locate hot spots and prioritize their remediation. To reduce the overhead and latency of manual assessment, prior works have explored automatically predicting assessment results from SV reports (SVRs). However, existing approaches fail to process the information conveyed by the rich text content (e.g., screenshots and code snippets) embedded in SVRs and miss information about vulnerable projects. More importantly, they primarily focus on assessment accuracy while neglecting to provide explanations or evidence supporting their predictions. As a result, these approaches remain impractical in real-world settings, where imperfect accuracy necessitates manual validation. LLMs offer a promising opportunity to address this limitation by performing SV assessment while simultaneously providing supporting evidence. Nevertheless, our extensive evaluation reveals that mainstream LLMs perform poorly on SV assessment tasks, largely due to a lack of assessment specific knowledge. To address the above challenges, we propose EAVA, a novel framework that effectively leverages LLMs to perform SV assessment and provide supporting evidence. EAVA employs specialized LLM agents to process rich text content in SVRs and incorporate information about vulnerable projects. EAVA builds a dedicated assessment LLM by injecting assessment-specific knowledge through finetuning. Specifically, we enable large-scale reasoning trajectory annotation using off-the-shelf LLMs and adopt a two-stage training paradigm, i.e., supervised instruction tuning to inject domain knowledge, followed by reinforcement learning to enhance the model’s intrinsic reasoning capability. Evaluations on a newly collected SVR dataset demonstrate that EAVA outperforms the best-performing baseline by 5.3%-35.2% across multiple evaluation metrics. Ablation studies validate the effectiveness of our design choices for both assessment specific model training and SV information enrichment. Finally, a user study with security experts confirms that the evidence provided by EAVA is useful and practical for real-world SV assessment.
Patch presence testing determines whether a binary incorporates the security fixes associated with a known vulnerability and has become increasingly important in software supply-chain security. However, despite numerous proposed techniques, the field still lacks a comprehensive understanding of the practical capabiliti...
Xiao-Bei Zhang, Yao-Wen Zheng, Wu Luo et al.· Proceedings of the ACM on So...· 0 citations
This work proposes Failure-Driven Prompt Refinement (FDPR), a methodology that analyzes recurring model failures to guide evidence-based prompt refinement, and shows that failure-driven refinement improves the reliability of LLM-based vulnerability analysis while yielding reusable prompt design principles.
Vulnerability patching is critical for safeguarding an enterprise's software ecosystem from cyber attacks. However, patching requires adequate preparation for potential system downtime and for unforeseen effects of the patched software on a live system. Thus, effective risk mitigation requires prioritization strategies...
Automated security scanners produce many alerts, but disclosure requires evidence of attacker control, reachability, impact, and reproducibility. We present an evidence-gated workflow that screens recently updated Python AI/ML and data-science packages then uses two separately fine-tuned open-weight agents to distingui...
Sai Teja Erukude, Lior Shamir· Electronics· 0 citations
HawkEye is introduced, a modular, web-based vulnerability auditing platform designed to streamline security analysis by integrating multiple scanning tools within a unified dashboard and illustrates how consolidated reporting improves vulnerability prioritization for development teams.
D. R. Patil, Varad Salgare, Devaj Arya et al.· International Journal for Re...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026