FaultLens is introduced, a method for learning compact behavioral test suites while preserving an auditable connection to executed evidence and provides a prioritized evidence mechanism, not a proof of correctness, and makes its budget, evidence source, generalization split, and misses explicit.
Abstract
Generated operational programs are often validated with either a few hand-written examples or exhaustive regression suites. The former can miss sparse boundary and interaction faults, while the latter can be unnecessarily expensive. We introduce FaultLens, a method for learning compact behavioral test suites while preserving an auditable connection to executed evidence. It executes a rich probe domain once, stores the fault-probe kill relation as a sparse outcome cache, and learns probe orderings only from earlier program generations. A fault-driven greedy component exploits known kill structure, while a mutation-independent diversity component covers probe families, cases, templates, and temporal bins. Their alternating hybrid remains useful when a new program contains a fault mechanism absent from ordering construction. We evaluate twenty generated operational policies across four environments, ten execution seeds, 1,200 measured run summaries, 2,160 controlled program transformations, and 4,120,200 executed program-probe pairs. Of 1,960 intended faulty transformations, 1,779 alter a contract or output somewhere in the finite audit domain; 200 additional controls preserve behavior. A 32-probe hybrid learned on generations 1-3 covers 576/582 (99.0%) dynamically killable faults in generations 4-5 using 1.2-2.0% of the exhaustive domain. With an entire fault family withheld from training, diversity raises scenario-family macro coverage from 84.6% to 94.9%. In a downstream deployment study, a conservative admission rule reduces severe tail regressions from 15/20 program-environment groups to 0/20. FaultLens provides a prioritized evidence mechanism, not a proof of correctness, and makes its budget, evidence source, generalization split, and misses explicit.
Self-supervised automated program repair (APR) leverages project-specific perturbations to generate training data and uses test execution diagnostics to guide patch generation. In practice, however, diagnostics are heterogeneous (e.g., exception messages, stack traces, assertion diffs, and dynamic execution signals) and must fit within a strict context budget. Naive concatenation either truncates critical evidence or amplifies noisy artifacts, especially for deep bugs where the failure symptom is far from the root cause. We present TraceStructRepair, a diagnostic structuring and budgeting approach for execution-aware self-supervised APR. TraceStructRepair (1) extracts a compact set of execution diagnostics from a single failing test, including exception type and message, stack trace frames, assertion diffs, and optionally dynamically loaded classes; (2) normalizes and ranks diagnostic elements to reduce redundancy and framework noise; and (3) assembles a field-aware representation under a fixed token budget with per-field caps and lexicographic priority rules. We implement TraceStructRepair on top of the SelfAPR pipeline and evaluate it on Defects4J using a project-wise heldout protocol. Beyond end-to-end repair outcomes, we analyze robustness under noisy fault localization and component ablations. We release artifacts to facilitate replication and future work on execution-aware, budget-constrained APR.
Pan Lu, Dongcheng Li, W. E. Wong· Annual International Compute...· 0 citations
EvoMem is introduced, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge and provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.
Viktor Volkov, V. Khrulkov, Andrey V. Galichin et al.· 0 citations
Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differently under compression. We give a runtime observability contract that covers all four memory classes with three operators, instantiate it on six model configurations across five architecture families, and compose the per-stage bounds into an executable request-level risk ledger. Contracts carry their error metric as a type -- composition is only defined when metrics match, and this check rejected our own first composed chain; the repaired chain crosses metrics through two proved bridges, and whatever no formal system can certify is measured instead, dropping the composed tier to empirical automatically: every claim is certified, partially certified, or empirical, composition inherits the weakest tier, and the tier is decided by the machine. Replayed over $12.4$M entry reads and run under eight-way concurrency with per-request budgets and fail-closed identity attribution, the ledger quantifies the honest trade-off on today's witness and holds its risk budget with zero violations. A fused always-on probe observes a declared one-layer subset under CUDA graphs inside the serving noise floor. Applied to a served DeepSeek-V4 stack with a packed compressed-KV prototype, the same machinery localizes a silent corruption to a precise structural boundary -- exact in the eviction-free, identity-isolated regime, with every observed failure in an eviction or slot-reuse regime -- through a machine-adjudicated discrimination campaign whose calculus rejected two of our own confounded inferences along the way. All artifacts, guards, and the Lean development are released at https://github.com/metask-ai/witprobe-attention-memory; every number in this paper regenerates from the shipped artifacts by one command.
Fanzhe Wei, Li Liu, Ziyang Wang et al.· 2 citations
Overall, TraceGate shows that rethinking debugging through controlled observability, rather than relying solely on stronger models or larger prompts, can make LLM-assisted repair more effective, efficient and controllable.
Nicolas Schuler, †. MateVincenzoScotti, †. RaffaelaMirandola· 0 citations
Frontier coding models now match or exceed strong human reference points on programming benchmarks, yet benchmark success does not imply maintainable software. Prompt-driven"vibe coding"is additive: new branches, guards, and fallbacks accumulate faster than obsolete logic is removed. We study the inverse problem-how an Al system should remove code when execution-verification capacity is finite. We formulate redundant-code reduction as proposal scheduling: a ranker orders single-statement deletion candidates, an execution suite accepts the first candidate that passes, and a budget bounds how many candidates may be tested. Our central observation is that candidate order, not model confidence, is the control surface a deployment can reason about. DELSCOUT instantiates two schedules. Given representative target-domain validation, a five-slot budget spends three slots on deterministic shortest-first candidates and two on complementary learned candidates; across nine MBPP replications with 0.5B, 0.6B, and 8B rankers this raises verified-deletion coverage by 9.5% relative (+6.7 accepted tasks) while consuming slightly fewer verifier calls than the matched static baseline. Without such validation the same rankers can lose coverage under shift, so we instead evaluate the complete static prefix first and append learned candidates only afterwards; for a deterministic verifier this makes coverage and character reduction non-decreasing by construction, at a measured 4.8-62.5% increase in verifier calls. MBPP+ then erases the in-domain advantage, showing that scheduling governs search while the test suite alone governs what"preserving behavior"means. The result is an auditable division of labor: models widen the search for removable code, order bounds the damage a mis-ranked proposal can do, and execution retains authority over every committed deletion.
Ruitong Li, Binjie Guo, Aisheng Mo et al.· 0 citations
Repository-level code repair generates rich tool traces, but most LLM agents discard this data and keep restarting from a fixed debugging loop. We present Evolutionary Self-Debugging Agents (ESDA), which mines tool traces into structured failure signatures and uses them to maintain a strategy bank of reusable debugging policies. Policies are stored as modular prompt genomes with typed slots, enabling slot-level reuse, mutation, and crossover as new tasks arrive. A cost-aware ranking objective prioritizes strategies that are likely to succeed in the first few evaluator calls under tight budgets. On RepoBench, ESDA solves 58.4% of tasks within the first two evaluator calls and reduces median wall-clock time by 3.0x compared to strong baselines. We further analyze transfer across languages and build systems and find that mining failure signatures yields consistent gains under distribution shift.
Shuang Cao, Rui Li· Proceedings of the 32nd ACM...· 0 citations