Skip to content
#software testing Open access

Fuzzing the Boundary between Models and Code in Hybrid AI-Enabled Systems

Oct 2026 · Proceedings of the ACM on Software Engineering · Vol 3, pp. 2613 - 2636 · 0 citations · 77 references

TL;DR

Neude is presented, a lightweight and extensible coverage-guided fuzzing framework specifically designed for hybrid AI-enabled systems that combines observations of program execution and neural model coverage to guide input mutations toward unexplored state spaces, enabling systematic testing of the entire hybrid system.

Abstract

Deep learning (DL) techniques are increasingly integrated into traditional software systems, giving rise to hybrid AI-enabled systems that combine neural models with program logic. While these systems exhibit remarkable capabilities, their complex and heterogeneous architectures pose significant challenges for reliability and testing, particularly in safety-critical domains such as autonomous driving. Existing testing approaches either target traditional code or isolate neural networks, overlooking failures arising from their interactions. In this paper, we present Neude, a lightweight and extensible coverage-guided fuzzing framework specifically designed for hybrid AI-enabled systems. Unlike existing tools, Neude combines observations of program execution and neural model coverage to guide input mutations toward unexplored state spaces, enabling systematic testing of the entire hybrid system. Moreover, Neude employs domain-aware mutation operators coupled with metamorphic relations, allowing automated bug detection without manual assertions. We evaluate Neude on Pylot, a complex autonomous driving system with tightly coupled neural and program components. Experimental results show that Neude uncovers diverse errors, and further analysis reveals how model uncertainty propagates through deterministic program logic to trigger downstream module failures. Our findings highlight the fragility of current hybrid architectures, calling for a paradigm shift from model-centric testing to system-centric quality assurance that accounts for the intricate interplay between neural and procedural components.

Read PDF

Similar papers

#explainable ai Book Open access Oct 2026

Automated Modelica Model Repair Using Generative AI

This PhD contributes an agentic LLM-based framework for diagnosing and repairing Modelica models, together with the empirical foundations that make it feasible and reproducible.

Masoud Sadrnezhaad · 0 citations
Preprint Oct 2026

A Neuro-Symbolic Reasoning Engine for Compilation-Free Patch Presence Testing

Code reuse in modern software ecosystems facilitates the spread of 1-day vulnerabilities, posing significant security risks. Determining if patches are present is crucial, but traditional methods suffer from limited usability and accuracy, often depending on a fragile compilation process. In our prior work, we introduc...

Si-Yuan Li, Jia-Yi Zhou, Zhi-Yuan Xia et al. · 0 citations
Book Open access Oct 2026

Hybrid Fuzzing in Input Generation Logic

This paper presents hybrid generator space fuzzing, a method that combines the strengths of hybrid fuzzing and generator-based fuzzing to improve the syntactic validity and semantic diversity of test inputs and implements GenEvo, an LLM-driven hybrid generator space fuzzer.

Si-Wei Wei, Shi-Hao Zhu, Rui-Jie Meng et al. · 1 citation
Review Open access Sep 2026

Artificial Intelligence-Driven Software Test Automation: A Comprehensive Survey

This paper focuses on two key tasks—AI-driven test case generation and defect detection—and provides a systematic review of the technological evolution from traditional automation to intelligent testing, contrasting traditional deep learning and large language model approaches in defect detection.

Jia-Lei Chen · 0 citations
#natural language process... Preprint Sep 2026

CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning

CovR is presented, an agentic framework for automated testbench generation that combines self-reflection loops with simulation-based feedback to maximize coverage and a reinforcement learning (RL) framework tailored for coverage-driven testbench generation, leveraging tool-derived rewards from simulation and coverage f...

Manar Abdelatty, Maryam Nouh, Sherief Reda · 0 citations
#software testing Book Open access Oct 2026

Beyond Next-Token Prediction: Autonomous Verification and Visual Testing in the Agent-First Era (Keynote)

This talk explores how agent-first development platforms like Google Antigravity shift the paradigm from manual test authoring to autonomous software verification, and describes recent advances in autonomous visual testing.

Paige Bailey · 0 citations

Related blog posts

Microsoft Research Blog Nov 12, 2026

Microsoft Research Focus – Edition #98

What's new in research at Microsoft? Host Anson Ho explores the latest, including AI-powered universal translation, a breakthrough in a 31-year-old mathematics challenge, and advances making AI coding agents faster and more efficient.  The post Microsoft Research Focus – Edition #98 appeared first on Microsoft Research.

Microsoft Research Blog Nov 5, 2026

Formally verifying agent-written full-stack web apps with PDD Kit

AI agents can generate code at a scale that’s hard for humans to review line-by-line. PDD Kit helps ensure the code is doing what the developer intended by creating formal specifications and testing against them. It converts developer intent into a Verus specification, proves AI-generated code meets it for every input, and uses tests where proofs aren't practical. Developers review intent, not every line. The post Formally verifying agent-written full-stack web apps with PDD Kit appeared first o…

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.