Experimental results show that Jev integration substantially reduces decision cost and latency across all five agents, with task-level gains varying by workflow, and highlight the importance of preserving progress, supporting recovery, and verifying completion when integrating efficient decision models into agent workflows.
Abstract
AI agents depend on repeated decisions to accomplish tasks, with large language models (LLMs) commonly serving as their core decision makers. The recent emergence of Jev offers an opportunity to make these decisions faster and at lower cost. However, it remains unclear whether such local savings can be translated into efficient and reliable task completion. In this paper, we investigate this efficiency transfer across five agents in two representative application settings, embodied tasks in ALFWorld and mobile app use in AndroidWorld. Across 4,500 task executions, we jointly evaluate task effectiveness, decision-level and task-level resource use, and execution behavior. Experimental results show that Jev integration substantially reduces decision cost and latency across all five agents, with task-level gains varying by workflow. Task success remains close to baseline levels on ALFWorld and declines more substantially in two mobile agents. Auxiliary reasoning and additional interactions absorb decision savings, while failed recovery and premature completion limit successful delivery. Mean cost per successful outcome increases in one agent even as spending per attempt falls. These findings highlight the importance of preserving progress, supporting recovery, and verifying completion when integrating efficient decision models into agent workflows. Their practical value should be assessed through successful task delivery, recovery behavior, and resources per delivered outcome.
The latency with respect to the resource dynamics of processing multiple requests and tasks concurrently is characterized and new optimization opportunities that exploit the resource dynamics of tasks are demonstrated: CPU-aware tool admission and task-aware CPU allocation.
Wonmi Choi, Minuk Park, Zhi-Xiong Niu et al.· arXiv.org· 1 citation
Jev-LiteMAS is introduced, a multi-agent framework that bridges both gaps by combining a lightweight decision model (Jev) with generative execution and feedback, and reduces generative-LLM tokens by 12.7-54.6% over LAS.
Kai-Xiang Wang, Zhai-He Zhou, Jiong Lou et al.· 0 citations
LLM agents increasingly perform consequential actions through external tools, yet their proposed actions are often executed without independent verification of policy compliance or task completion. We introduce JevAct, a runtime verification framework that intercepts selected agent actions before execution and enables...
Iftach Shoham, Yuval David, Offer Akrabi et al.· 0 citations
The challenges of reliable asynchronous task execution in LLM agents are highlighted and the need to evaluate how agents handle incoming results alongside end-to-end task success is highlighted.
De-Tao Zeng, Gui-Lin Ou, Nuo Chen et al.· 0 citations
To balance task performance and computational cost, agentic workflows can use a lower-cost model for execution and selectively consult a stronger model. In such executor-advisor systems, the executor decides when to seek advice, making escalation depend on self-assessment, although current language models struggle to d...
Pascal Sowietzki, Steffen Holter, Yannick Metz et al.· 0 citations
It is found that SFT consistently improves text-turn success, and that overall next-turn success increases for every model under gold-history evaluation, but these improvements do not transfer to autonomous workflow execution.
Md Tahmid Rahman Laskar, Xue-Yong Fu, Gundeep Singh et al.· Proceedings of the Second Wo...· 1 citation· ⚡1