An Agentic Reward Black-box Optimization (ARBO) framework is proposed, in which an LLM agent builds the search strategy at run time from an evaluation history maintained as its persistent workspace, which comprises two components: observations maintained by the evaluation oracle and an agent-maintained belief that records diagnoses and intended next steps.
Abstract
Designing dense reward functions for low-level reinforcement learning (RL) control remains difficult. Recent work uses large language models (LLMs) to iteratively generate and refine reward functions using policy-training feedback within scripted search algorithms. However, evaluating each candidate requires a full RL training run, making sample efficiency a central challenge for reward search on complex control tasks. To address this limitation, we propose an Agentic Reward Black-box Optimization (ARBO) framework, in which an LLM agent builds the search strategy at run time from an evaluation history maintained as its persistent workspace. The evaluation history comprises two components: observations maintained by the evaluation oracle, including candidate scores, per-term training curves, and error tracebacks; and an agent-maintained belief that records diagnoses and intended next steps. The agent queries both with tools and generates the next batch of reward candidates, rather than generating them in a single pass from a fixed prompt. Across four control domains, ARBO achieves gains of 29.9% in manipulation success rate and 192.8% in power-grid score over baseline means under a shared evaluation budget. Ablations examine each component's contribution and sensitivity to backbone choice.
T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier by executing each task's own verifier.
Jun-Yao Yang, Yu-Cheng Shi, Zhong-Zhi Li et al.· 9 citations
Harness-RL is introduced, a structured reinforcement learning framework that combines Conflict-Aware Policy Optimization (CAPO) with interface-level black-box trajectory construction and supports both central-only and joint multi-agent training.
Xin-Ke Jiang, Zhi-Xin Zhang, Zhi-Bang Yang et al.· 1 citation
Open-ended tool agents can complete the same objective through different tool paths, and their quality depends jointly on the interaction trace, environment state, and final artifacts, so such tasks rarely admit an executable ground-truth reward. The central challenge is therefore to construct a reward from existing ex...
Linda Feng, Ji Qi, Yi-Ran Cheng et al.· 0 citations
RewardWeaver is introduced, a self-evolving reward adaptation framework for language agents in long-horizon interaction that maintains a validated capability space in which the semantics of admitted Rubrics remain fixed, and closes the loop between policy optimization, task evaluation, failure attribution, and reward a...
Heng-Bo Xiao, Bo-Yao Zhang, Pu-Rui Liu et al.· 0 citations
Multimodal agents commonly generate free-form reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured R...
Feiyu Zhu, Xiao-Yu Zhu, Ji-Qian Yang et al.· 0 citations
Fully Automated Reinforcement Learning is introduced, a framework that automates RL design by jointly searching observation and reward programs, learning algorithms, and training settings and outperforms the evaluated automated baselines on challenging navigation, locomotion, and manipulation tasks.
Tian Jin, Liu-Yu Xiang, Chao-Wen Yang et al.· 0 citations