LLM agents can spend millions of tokens during vulnerability discovery without producing a working proof of concept (PoC). What consumes that budget, and why does it fail to produce results? We diagnose these costs and failures through a multi-axis open-coding study of 200 CyberGym traces, spanning four agents (i.e., C...
Li Lu, Yan-Jie Zhao, Hong-Jie Chen et al.· 0 citations
Static taint analyzers are widely used in software security, yet aggregate vulnerability-detection metrics provide limited insight into the specific language constructs and analysis capabilities that cause tools to succeed or fail. This paper presents xAST, a fine-grained benchmark containing 844 paired capability case...
Yan-Jie Zhao, Shen-Ao Wang, Jian Zhao et al.· ACM Transactions on Software...· 0 citations
This study presents a comprehensive analysis of user reviews for GenAI-powered mobile apps, focusing on identifying both their issues and strengths, and derived 18 key findings that shed light on the rapid growth of GenAI-powered apps across various genres, the common issues leading to user dissatisfaction, and the fea...
Wu-Xia Bai, Yan-Jie Zhao, Kai Chen et al.· ACM Transactions on Software...· 0 citations
HGFuzzer, an automated framework that augments DGF with predicate-guided execution synthesis, and successfully triggered 19 bugs, unearthed 9 previously unknown, CVE-assigned bugs, confirming its efficacy in real-world security auditing.
Hanxiang Xu, Yan-Jie Zhao, Junqing Yu et al.· ACM Transactions on Software...· 0 citations
Large Language Model (LLM) agent frameworks such as LangChain, LlamaIndex, and CrewAI have become critical infrastructure powering production AI systems, yet they remain severely under-tested due to fundamental challenges in automated testing. Unlike traditional software, where crashes serve as reliable oracles, defect...
JSTestCraft reconstructs missing context via three enrichment agents: library, structural, and semantic via three enrichment agents, demonstrating that context reconstruction and agentic collaboration enable LLMs to perform more reliable and adaptive testing in dynamic JavaScript environments.
LeanGuard is presented, a neuro-symbolic framework that assigns each act to the side equipped for it, and it is argued that the remedy is not better prompting but a separation of roles: the component that interprets the code must not also be the one that decides a safety obligation is met.
Yanjie Zhao, Hongjie Chen, Li Lu et al.· arXiv.org· 0 citations
IAL-Scan is proposed, a static analysis tool for detecting IAL failures in real-world LLM agent projects, and builds an Agentic Loop Dependence Graph (ALDG) to recover explicit and framework induced feedback paths, and checks whether these paths can repeatedly reach costly or state growing operations without an effecti...
Xinyi Hou, Shenao Wang, Yanjie Zhao et al.· 2 citations
The evaluation shows that AgentFlow recovers richer agent entities and dependencies than existing AST-based agent static analysis tools, generates more dependency-aware Agent BOMs, and uncovers 238 taint-style prompt-to-tool risks in real-world agent programs.
Shenao Wang, Xinyi Hou, Yanjie Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
