Skip to content
Preprint

LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation

Aug 2026 · 0 citations · 38 references
Computer Science

TL;DR

LoongReflect is proposed, a training framework that formulates reflection as a memory-control policy over a reversible trajectory tree using explicit reflect and backtrack actions and demonstrates consistent improvements over outcome-only reinforcement learning and self-distillation baselines.

Abstract

Large language model agents increasingly rely on long-horizon reasoning to solve complex tasks involving planning, tool use, and memory. A critical capability in such settings is reflection: assessing trajectory progress, identifying missing evidence and unreliable intermediate states, and deciding whether to continue, revise, or abandon the current branch. Learning effective reflection, however, is challenging because reflection is performed locally within the current branch, whereas its utility can only be determined by its contribution to the final trajectory outcome. This local-global mismatch makes outcome-based reinforcement learning provide only local, sparse and delayed supervision for reflective decisions. To solve these, we propose LoongReflect, a training framework that formulates reflection as a memory-control policy. The agent operates over a reversible trajectory tree using explicit reflect and backtrack actions. Reflection consolidates verified facts, missing evidence, and branch-specific risks into working memory, while backtracking removes an unreliable branch from the active context and preserves a concise corrective lesson. To learn this policy, LoongReflect combines two complementary signals through a look-ahead, extragradient-style coordination mechanism. A fast channel distills globally informed reflective behavior from a privileged teacher, with supervision restricted to reflection and backtracking tokens. A slow channel optimizes complete trajectories using outcome-based GRPO, aligning local control decisions with final task success. Experiments on multi-hop retrieval-augmented generation and mathematical reasoning benchmarks demonstrate consistent improvements over outcome-only reinforcement learning and self-distillation baselines.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis

Deep search requires LLM agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories int...

Xing-Yu Wu, Yuchen Yan, Zhengxi Lu et al. · 0 citations
Sep 2026

SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation

This work introduces SeeQ (Subtask-elicited Q-functions), which instead learns Q-values for the currently active subtask, which substantially improves best-of-N policy steering on real-world manipulation tasks.

Saksham Singh, Zhe-Yuan Hu, Max Sobol Mark et al. · 2 citations
#artificial intelligence Preprint Sep 2026

APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering

APEx is proposed, a hierarchical experience utilization framework that organizes interaction history into instance-level trajectory memories and category-level procedural skills, and couples them through a closed-loop architecture of Executor, Distiller, and Planner, allowing ground-truth-free self-improvement with ski...

Jie Ding, Rui Sun, Xin-Yi Zhang et al. · 2 citations
#natural language process... Preprint Sep 2026

From Base Rollouts to RL Reasoning: A Budgeted Search Perspective

Results support a qualified internalized-search reading: under the recipe the authors test, much of the measured RL gain corresponds to a change in sampling efficiency toward operating points the base model can already reach under search.

Wen-He Sun, Cun-Xiang Wang, Zi-Jun Yao et al. · 0 citations

AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents

Coding agents solve repository-level software engineering tasks through long trajectories of code inspection, search, editing, and testing. As a task progresses, earlier exploration becomes stale, so managing context is more than avoiding overflow: an agent must decide when to compact, what working state to preserve, a...

Xuan Zhang, Long-Tao Zheng, Cun-Xiao Du et al. · 7 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.