Skip to content

HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning

Sep 2026 · 1 citation · ⚡ 1 influential · 39 references
Computer Science

TL;DR

This work develops HarnessBandit, an online scheduler that selects one harness per optimizer step, and improves over mixed-batch multi-harness training on both benchmarks, while training diagnostics indicate that learnability and transferability provide distinct, evolving signals.

Abstract

Language-model agents are increasingly deployed through diverse harnesses that differ in system prompts, tool schemas, control loops, and trajectory formats. The same model can perform unevenly across these interfaces, making robustness to harness variation an important objective. A natural approach is to train a shared policy through multiple harnesses, but doing so introduces a scheduling problem: each training step should favor a harness that currently provides a useful learning signal while also producing an update that benefits the other harnesses. We develop HarnessBandit, an online scheduler that selects one harness per optimizer step. After a group-relative policy optimization (GRPO) update, it observes learnability -- the mean absolute advantage on the batch -- and transferability -- the cosine between a low-dimensional gradient sketch of the current harness and exponential moving averages of the remaining harnesses. The two signals are fused after pooled sliding-window min-max normalization and sampled with a visit-dependent bonus and an explicit exploration floor. We train Qwen3.5-2B across six harnesses on ClawGym and evaluate on PinchBench (held-out tasks, in-distribution OpenClaw) and ClawEval (held-out tasks and harness). HarnessBandit improves over mixed-batch multi-harness training on both benchmarks, while training diagnostics indicate that learnability and transferability provide distinct, evolving signals.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents

Agent reinforcement learning (RL) increasingly runs through full execution harnesses, and a multi-harness recipe mixes two choices: exposing the policy to several harnesses, and comparing their rewards inside one relative-advantage group. We isolate the second choice in repository-level coding. From one Qwen3-8B superv...

Chen-Qian Le, Jia-Yi Cheng, Qi-Jia He et al. · 1 citation
#natural language process... Preprint Oct 2026

CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution

Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Existing harness-model co-evolution approaches improve both components, yet often treat trajectories produced during harness search as an undifferentiated replay buffer. This...

Ji-Xuan Chen, Jia-Xin Zhang, Qinyuan Ye et al. · 1 citation
Preprint Aug 2026

LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents

Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward ha...

Yi-Ming Du, Yu-Xin Jiang, Tao Yuan et al. · 3 citations
Preprint Aug 2026

Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses

Harness-RL is introduced, a structured reinforcement learning framework that combines Conflict-Aware Policy Optimization (CAPO) with interface-level black-box trajectory construction and supports both central-only and joint multi-agent training.

Xin-Ke Jiang, Zhi-Xin Zhang, Zhi-Bang Yang et al. · 1 citation
Preprint Aug 2026

ClawGym II: Exploring Black-Box RL on Agent Harness

This work presents a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses, supporting unified training across heterogeneous execution systems.

Hua-Tong Song, Fei Bai, Ming Yang et al. · 2 citations
#artificial intelligence Preprint Sep 2026

Harness-Zero: Harness Distillation via Agent-as-Harness

This work introduces Harness-Zero, which enables harness distillation through agent-as-harness, and shows that for frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness.

Hao-Ran Ye, Yuxing Lu, Hao-Nan Dong et al. · 5 citations · ⚡1

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.