NeuroAssertion is presented, a coverage-driven assertion generation framework that combines formal trace generation, syntax-guided synthesis (SyGuS), and an agent-inspired refinement process within a unified framework that delivers around 2X more assertions and about 2X higher mutation coverage than traditional assertion mining methods.
Abstract
Hardware functional verification relies on high-quality assertions to expose design bugs and establish confidence in Register Transfer Level (RTL) designs. Yet existing assertion mining methods still struggle to produce complete and reliable assertion sets: random or limited traces fail to cover hard-to-reach behaviors, and one-shot generation provides little feedback about what remains unverified or how the assertion set should be improved. As a result, critical design behaviors can remain uncovered even when many assertions are generated. We present NeuroAssertion, a coverage-driven assertion generation framework that combines formal trace generation, syntax-guided synthesis (SyGuS), and an agent-inspired refinement process within a unified framework. Our framework first converts hard-to-reach control-flow conditions into formal reachability objectives, uses model checking to generate behaviorally diverse traces, and mines initial assertions from these traces with SyGuS. It then performs targeted agent-inspired refinement under verification feedback: one LLM first proposes candidate assertions for uncovered regions, and if a candidate fails formal checking, a second LLM generates a repair grammar that guides constrained symbolic synthesis in a neuro-symbolic repair procedure. Experimental results show that this framework delivers around 2X more assertions and about 2X higher mutation coverage than traditional assertion mining methods.
Search-based regression-test generation effectively explores complex program structures, yielding high structural coverage, but its oracles are derived from the system under test: faults already present are recorded as expected behavior rather than exposed. Property-based testing offers independent semantic oracles, but depends on high-quality properties and gives little guidance for reaching deep states or satisfying selective preconditions. We present PROGRESS (PROperty-Guided REgression Search for Semantic Falsification), integrating intent-driven properties into coverage-guided, search-based evolutionary test generation to reach deep program states and detect violations of intended behavior. PROGRESS (1) extracts intent-bearing code context and uses a language-model pipeline to generate executable jqwik properties while limiting implementation leakage; (2) extends EvoSuite's DynaMOSA with a search objective and property-aware fitness function per property, rewarding progress through preconditions and prioritizing falsifying executions; and (3) binds property parameters and uses jqwik-provided generators to connect quantified inputs to evolving test sequences, steering generation toward coverage and bug-detection goals. We evaluate PROGRESS on 25 large-scale Java systems against regression-test generation, standalone property-based testing, and context ablations. PROGRESS detects 328/562 current-version bugs (58%) versus none for regression-test generation, and satisfies all preconditions for 70/150 hard-to-reach properties versus 18 for standalone jqwik. Ablations show documentation and caller/callee context are key to generating valid executable properties. PROGRESS preserves structural exploration while exposing faults missed by regression-derived assertions; we release a comprehensive artifact package.
D. Mo, Noshin Ulfat, Matthew B. Dwyer et al.· 0 citations
Test assertions are critical elements of unit tests, serving as checkpoints to validate expected behavior and ensure software correctness. Numerous techniques have been proposed to automate assertion generation, with recent progress notably driven by large language models (LLMs). Despite the promise, existing approaches such as ChatAssert suffer from modest accuracy, heavy reliance on oversampling, and vulnerability to model randomness due to one-shot prompting. To address these limitations, we propose AssertMate, a novel agent-based assertion generation framework that enhances the quality and reliability of LLM-generated assertions through three key components: (1) actual value construction that identifies assertion targets via static analysis and type-aware heuristics; (2) multi-perspective expected value prediction using code generation, retrieval-augmented generation (RAG), and chain-of-thought (CoT) reasoning agents; and (3) an LLM-as-a-Judge collaboration mechanism to select the most appropriate assertion. Evaluation on the Defects4J benchmark demonstrates that AssertMate significantly outperforms state-of-the-art techniques in compilation success and pass rates, along with substantially higher bug detection capabilities. Integration with EvoSuite further validates AssertMate's practicality, yielding superior mutation coverage and kill counts. Ablation studies reveal that each of the three components makes a significant and complementary contribution to the overall performance. This work affirms the great potential of aggregating diverse perspectives to enhance the effectiveness of LLM-based assertion generation.
Dong Wang, Qiaoyu Han, Lin Yang et al.· 0 citations
Software vulnerabilities are persistent, but validating them remains difficult: a Proof-of-Vulnerability (PoV) requires a concrete input that triggers the vulnerable behavior, yet public triggering inputs are often unavailable for disclosed vulnerabilities. Existing techniques make different tradeoffs in effectiveness, scalability, cost, and controllability, leaving room for complementary designs. To complement them, we present POVGEN, a low-cost neuro-symbolic framework that makes PoV generation cost-effective via semantic focusing and LLM-guided constraint reasoning using open-weight models. POVGEN first localizes vulnerability-relevant regions (utilizing patch information if available), then performs path-sensitive reachability analysis, and finally generates PoVs by extracting and solving constraints with LLM-guided reasoning backed by an SMT solver. POVGEN successfully generates PoVs for 78.98% of vulnerabilities in a recent benchmark, outperforming fuzzing (up to 50.20%) and symbolic execution (2.45%). On 250 real-world CVEs without public PoVs, it generates valid PoVs for 74.80% of cases and reproduces 65.1% when without patch information. The fine-tuned open-weight models match frontier commercial LLMs on key sub-tasks (i.e., the core constraint-reasoning steps) while running locally at no per-sample API cost. Applying the generated PoVs revealed six flawed patches in disclosed CVEs (all subsequently fixed) and five previously unreported vulnerabilities (of which four have been confirmed and fixed by the developers).
Functional verification dominates integrated circuit (IC) front-end engineering effort, and a single missed bug that escapes to silicon can trigger a costly respin. Recent large language models (LLMs) offer new opportunities to automate this process, yet existing LLM-based approaches generate each component through independent single-turn calls with no shared context, leaving interface mismatches undetected and reported coverage disconnected from specification requirements. To address these challenges, we present GoGoTB, an agentic framework that achieves end-to-end verification closure through three subsystems: an agentic execution control layer, an evolvable knowledge system, and specification-grounded coverage closure. The execution control layer separates deterministic enforcement from LLM reasoning at every tool and stage boundary. The knowledge system dispatches methodology and design-specific expertise on demand. The coverage framework anchors every bin to a named specification behavior so that each residual gap has a diagnosable root cause and a targeted remedy. Tested on 8 register transfer level (RTL) designs without any human intervention, GoGoTB achieves 100\% environment generation success and averages 98.4\% line, 97.2\% branch, 97.0\% toggle, and 83.2\% functional coverage. No prior work successfully generates a complete verification environment or achieves meaningful coverage on the same benchmarks.
Xin Xin, Jincheng Lou, Junhui Li et al.· 0 citations
ChipFuzzer is proposed, a hardware fuzzing framework that leverages the semantic reasoning capabilities of large language models (LLMs) to improve fuzzing effectiveness and improves average condition coverage and bug detection rate over the strongest baseline.
Kun Wang, Cangyuan Li, Kaiyan Chang et al.· 0 citations
System models and artifacts continuously require validation and refinement to address imprecise specifications and early-stage requirements in order to derive executable simulations. We propose a multi-layer approach for the automated formalization and execution of partial generative specifications derived from high-level descriptions. The first layer processes model seeds—potentially produced by large language models—that capture coarse structural information such as node relations, ordering, and timing estimates. Rather than requiring fully specified executable models from generative sources, we restrict their role to producing partial specifications, which are then completed through formal synthesis. We implement a synthesis engine based on Boolean satisfiability that constructs executable control flow structures from these partial specifications while enforcing structural consistency and execution semantics. Satisfiability modulo theories are further used to verify temporal properties and establish simulation baselines. The resulting models are then transformed into a set-theoretic discrete event system specification, enabling executable simulation via generated code artifacts. This pipeline establishes a unified pathway from partial generative artifacts to formally validated and executable models. It enables reliable and interpretable exploration of design alternatives and experimentation under formally grounded structural and temporal constraints, while providing a foundation for integrating generative modeling with rigorous execution semantics.
Abdurrahman Alshareef, Bernard P. Zeigler· Logics· 0 citations
