Skip to content
Open access

PoCE: Automated Proof-of-Concept Synthesis using Large Language Models for Robust Validation

Oct 2026 · Proceedings of the ACM on Software Engineering · Vol 3, pp. 986 - 1009 · 0 citations · 82 references

Abstract

Vulnerability reports play a critical role in software repair, with Proof-of-Concept (PoC) tests serving as one of their most essential components. PoC tests enable software developers to reliably reproduce reported vulnerabilities and subsequently deploy patches. However, generating effective PoCs is costly, expertise-intensive, and increasingly challenging due to the diversity of modern software ecosystems and their complex dependencies. Inadequate or incorrect PoCs can significantly delay patch deployment, thereby increasing the window of exposure to attacks. Prior work on automated PoC generation struggles to produce comprehensive and reliable testing. In this work, we present an automated PoC generation framework, PoCE, capable of generating PoCs across diverse software systems by handling varied input formats and complex execution contexts using large language models. PoCE integrates structured in-context learning, retrieval-augmented generation, and iterative chain-of-thought reasoning to expand an initial successful PoC into multiple validated variants. These variants are executed in controlled environments to confirm success. We evaluate PoCE on thirteen widely used software projects, including TensorFlow, Yasm, Zlib, Liblouis, Cflow, Pytorch, Node.js, TCPDUMP, Fig2dev, Binutils, libsndfile, LibTIFF, and libsixel. Our approach achieves a success rate of 77.7% and generates multiple PoC variants for the most vulnerable cases, uncovering alternative trigger paths and edge conditions. We discover 68 zero-day PoCs and identify 26 previously unknown zero-day vulnerabilities in cross-layer software.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.