Skip to content

ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments

Sep 2026 · arXiv.org · Vol abs/2609.33618 · 1 citation · 50 references
Computer Science

Abstract

Language model agents are increasingly deployed in open-world tool environments, which require balancing exploring unknown capabilities and exploiting known ones. Existing methods face a performance-efficiency tradeoff: they either rigidly decouple exploration and execution or interleave them without coordination. We argue that the key lies not in whether to decouple or interleave them, but in how to coordinate them across granularities. We introduce ParaAct, a structured parallel-action loop that combines phase-level Exploration $\rightleftharpoons$ Execution with action-level parallelism. To learn this loop, ParaAgent combines multi-agent cold-start demonstrations with reinforcement learning under multi-level advantage decoupling, making planning structure explicit and supervising it with step-, phase-, and trajectory-level rewards. Learning is supported by our ToolEnv, a scalable simulator grounded in 50,011 realistic tool interfaces. On two open-world tool benchmarks, ParaAgent-4B achieves the best average success among all baselines, including GPT-4.1 systems, with the largest gains on multi-tool tasks. Behavioral analyses show that these gains stem from this action organization, highlighting its importance for capable and efficient open-world agents.

View source

Similar papers

UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learning

UnifiedPlayers, a cooperative framework comprising a Planning Player that generates tasks, an Execution Player that produces multi-turn trajectories with Python tool calls, and an Evaluation Player that constructs executable verifiers, highlights cooperation among specialized players as a promising path toward self-enh...

Wen-Jie Liao, Liang Zhao, Ze-Hong Cao · 0 citations
Sep 2026

AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs

To quantify collaboration effectiveness in addition to conventional binary task success, Causal Collaboration Effectiveness (CCE) is proposed, a graph-based metric that traces causal dependencies between agent actions and measures what fraction of a team's effort actually contributed to the outcome.

Raphael Shu, Yu-Sen Zhang, Y. Cho et al. · 2 citations
Preprint Aug 2026

Prime Agent: A Self-Improving RLM Harness

Low-friction, expressive membrane prevents harness failures from becoming model failures and pushes measurement toward the model's true maximal underlying capability, on Factorio, where refinement allows for continuous technology progression and dedicated subagents enable parallelized work.

Seth Karten, Alex L. Zhang, Kevin Thomas et al. · 18 citations · ⚡2
Sep 2026

Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models

FIND is introduced, an agentic real-world RL framework that closes the loop between scene understanding, weakness-aware practice, self-evaluation, and policy improvement in a persistent workspace and reframes autonomous practice as a scene-conditioned, performance-aware task-selection problem.

Yuan Fang, Ze-Chu Li, Hao-Lei Tong et al. · 3 citations · ⚡1
Preprint Aug 2026

Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses

Harness-RL is introduced, a structured reinforcement learning framework that combines Conflict-Aware Policy Optimization (CAPO) with interface-level black-box trajectory construction and supports both central-only and joint multi-agent training.

Xin-Ke Jiang, Zhi-Xin Zhang, Zhi-Bang Yang et al. · 1 citation
#artificial intelligence Preprint Sep 2026

No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents

Results indicate that conclusions drawn in a single environment may not generalize, and that hybrid planner-executor designs should be motivated by specific failure modes rather than the assumption that one architecture is universally preferable.

A. Shaikh, Arunesh Sinha, Nathaniel D. Bastian et al. · 1 citation · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.