#artificial intelligence #machine learning

Skill Reuse as Compression in Agentic RL

Zhikun Xu Yu Feng Jacob Dineen Taiwei Shi Jieyu Zhao Ben Zhou

May 2026 · arXiv.org · Vol abs/2605.31509 · 1 citation · 37 references

Computer Science

TL;DR

This work proves a PAC-Bayes bound guaranteeing that a dictionary extracted from successful trajectories has bounded expected description length on future successful behavior, and introduces ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle.

Abstract

Large language model agents trained with reinforcement learning (RL) often learn brittle, task-specific shortcuts. We hypothesize that agents generalize better when their successful trajectories are structurally compressible, decomposed into a small set of reusable abstract patterns. To formalize this, we introduce ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle. ReuseRL extracts a shared skill dictionary from successful trajectories and augments the RL objective with a segmentation cost, explicitly penalizing idiosyncratic behaviors that encode poorly. We prove a PAC-Bayes bound guaranteeing that a dictionary extracted from successful trajectories has bounded expected description length on future successful behavior. Across ALFWorld, TextWorld-Cooking, and Countdown-Stepwise, ReuseRL improves in- and out-of-distribution success over vanilla GRPO and strong round-length baselines.

DOI arXiv Semantic Scholar

View source

Similar papers

#machine learning Preprint Aug 2026

Learning Generalizable Behaviors for Terminal Agents

River, a simple training recipe that improves reward quality by filtering low-quality environments and augmenting outcome rewards with process-level behavior regularization is proposed, which achieves the best performance among evaluated open-source RL-trained 8B models across four terminal-agent benchmarks.

Yi-Fan Yao, Bo Pang, Xuan-Phi Nguyen et al. · 2 citations

#artificial intelligence Preprint Sep 2026

X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization

X-Tree improves over standard recipes at matched data and budget by up to 4.5% SR on WebArena, 5.8% SR on ScienceWorld and 4.1% success on WebShop, and matched analyses attribute the gains to the X-Tree structure and the three integrations.

Si-Tao Cheng, Xun-Jian Yin, Zhi-Yuan Sun et al. · 1 citation

#artificial intelligence Preprint Sep 2026

CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents

By shifting RL from token-level exploration to experience-level reasoning, CodeSkill improves optimization efficiency and long-horizon behavioral coherence, highlighting the effectiveness of explicit behavioral abstraction for scalable agentic code generation.

Song-Li Wu, Jing-Yi Wang, Zhao-Cheng Du et al. · 0 citations

#machine learning Preprint Sep 2026

Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents

SPACE induces two-level programmatic skills from successful trajectories, where subskill boundaries serve as direct chunk-boundary supervision from trajectory-induced programmatic skills, and distilled into a primitive-chunk policy via hybrid on-/off-policy optimization.

Yan-Ting Yang, Can Jin, Jin-Man Zhao et al. · 3 citations

#artificial intelligence Preprint Oct 2026

Cross-Benchmark Transfer from RL on Agentic Coding Tasks

Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic codin...

Sushant Mehta, Logan Ritchie, Edwin Chen · 0 citations

Review Open access Sep 2026

Reinforcement Learning Post-Training for Reasoning Large Language Models: Methods, Systems, and Evaluation

Reinforcement learning (RL) has become a central post-training approach for reasoning and agentic large language models (LLMs), particularly when task outcomes can be verified automatically. Comparisons across this literature remain difficult because a reported gain may combine changes to the learning signal, policy co...

Liu Yang, Han Zhu, Zheng-Yang Zhong et al. · Unmanned Systems · 1 citation