Skip to content

QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents

Sep 2026 · arXiv.org · Vol abs/2609.33848 · 2 citations · 58 references
Computer Science

TL;DR

QwenGyre elastically reallocates GPUs between rollout and training without interrupting live executions, while its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs.

Abstract

Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model--environment interactions, and nearly 1M tokens per rollout. Applying online reinforcement learning (RL) to such executions poses two fundamental challenges: (1) severe execution variance and prolonged rollout delays cause massive GPU idling; and (2) complex non-linear branching generates massive trajectory redundancy, crippling training efficiency. To address these, we presents QwenGyre, an end-to-end framework for xlong-horizon online RL. QwenGyre elastically reallocates GPUs between rollout and training without interrupting live executions, while its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs. Scaled to our flagship model, Qwen~3.8 2.4T, with 700K tokens per rollout, QwenGyre yields a 6.0% absolute gain on NL2RepoBench (52.5% $\to$ 58.5%) in 48 steps. Across our evaluations on diverse domains of training datasets, QwenGyre delivers up to $1.85\times$ and $1.78\times$ speedups over Colocate and Async, respectively.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learning

This work introduces ThunderSyncRL, which starts gradient computation as soon as all required inputs are fixed, without policy staleness, and proves that gradient streaming produces the same GRPO and OPD updates as batch-synchronous training, without changing either objective.

Seil Kang, Hangoo Kang, Tarun Suresh et al. · 0 citations
#artificial intelligence Preprint Sep 2026

T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier by executing each task's own verifier.

Jun-Yao Yang, Yu-Cheng Shi, Zhong-Zhi Li et al. · 9 citations
Aug 2026

SmoothRL: Online Reinforcement Learning During Asynchronous Execution

SmoothRL is presented, an online RL framework that fine-tunes a pretrained policy within an asynchronous inference loop, and is evaluated on real-world robotic tasks requiring high precision, as well as highly dynamic tasks that necessitate asynchronous execution.

Guang Gao, Yuxuan Nong, Bai-Fu Huang et al. · 1 citation
#machine learning Preprint Aug 2026

HARTS: Efficient Agentic Reinforcement Learning for Hybrid-Attention Models over Arbitrary Rollout Trees

HARTS is the first system to demonstrate arbitrary-rollout-tree prefix-sharing speedups on a real hybrid-attention model, and its numerical differences are comparable to baseline self-rerun variation, and its reward trend is similar to the baseline over the first 120 steps of SWE-bench training.

Bo-Yuan Meng, Pei-Hua Bao, Hong Liu et al. · 0 citations
#natural language process... Preprint Oct 2026

LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches

Reinforcement learning has greatly advanced the capabilities of large language models, but its memory demands remain a barrier to broader adoption. We introduce LoGRA, an approach to RL post-training that reduces memory by retaining useful learning signals in low-rank gradient sketches. These compact representations su...

Shao-Kun Zhang, Yi-Fan Zhang, Jian Hu et al. · 1 citation · ⚡1
Preprint Aug 2026

LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents

Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward ha...

Yi-Ming Du, Yu-Xin Jiang, Tao Yuan et al. · 3 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.