Mid-Harness is introduced, which samples and verifies candidate actions before forwarding one for execution, while keeping the generator and harness unchanged, and identifies action scaling as a promising target for test-time compute scaling in terminal agents.
Abstract
Terminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution. A poor command (e.g., wrong package install) can change the environment in ways that hinder subsequent progress, even when the model could generate a better alternative. We investigate whether allocating test-time compute at the model-harness boundary can improve action reliability and trajectory success, and what makes this allocation effective. To study these questions, we introduce Mid-Harness, which samples and verifies candidate actions before forwarding one for execution, while keeping the generator and harness unchanged. With a TMAX-9B generator, more action sampling yields little benefit under weak verification, whereas a capable verifier can exploit useful alternatives from the same generator. On TerminalBench-Lite, a GPT-5.6 Sol verifier raises Pass@1 from 50.00% for the base agent to 68.03% with 8 sampled actions. When the same TMAX-9B model serves as the verifier, pairwise verification performs best among the evaluated verification mechanisms. Distilling responses from the stronger verifier into TMAX-9B further improves Pass@1, while leaving the action generator unchanged. With TMAX-9B on TerminalBench-Lite, combining action and trajectory scaling reaches higher success at lower estimated token cost than generating more trajectories alone. Mid-Harness also improves performance across additional models, benchmarks, and harnesses. These findings identify action scaling as a promising target for test-time compute scaling in terminal agents.
RiseHarness is a framework for iteratively improving agent harnesses with model weights fixed that treats a specific model as the unit of harness adaptation, studying how changes to shared interaction mechanisms can make existing capabilities more reliable across tasks.
Ru-Han Wang, Yu-Cheng Shi, Zong-Xia Li et al.· 0 citations
This work takes an RSI-inspired approach at the harness layer, scaling auto-research loops across increasingly numerous and diverse environments for harness rollouts, and yields reusable improvements that transfer beyond their development setting.
Hao-Zhe Liu, Tian Ye, Sen-Sen Gao et al.· arXiv.org· 13 citations
This work introduces AGNI, an automated pipeline that extracts trajectory-relevant assumptions, injects targeted environmental changes, and validates that the resulting novel tasks remain solvable and highlights a gap between task competence and adaptive capability and motivate environmental variation as a core dimensi...
The Causal Intervention Router (CIR) is introduced, a lightweight policy that uses information available before recovery to decide when intervention is worthwhile and leaves all evaluated trajectories with correct observations untouched.
Shu-Yao Xiao, Sheng-Ling Wang, Xuan Chen et al.· 3 citations
Agent harnesses enable large language models to act through tools, observe the results, and iteratively work toward a task goal. Verification checks and revises agents'work to improve output quality. Yet there is limited guidance on how harnesses should organize this effort. We study verification as an axis of test-tim...
Zheng-Xin Zhang, Er-Kang Zhu, Yin Lin et al.· 0 citations
Harness self-evolution offers a practical route for LLM agents to improve from execution experience without updating model parameters. Its effectiveness depends on whether models can successfully act on the guidance provided by their harnesses. However, we find that harness benefits are not uniform across model capabil...
Zhi-Hao Yang, Haojin Wang, Jun-Cheng Wu et al.· 0 citations