Skip to content

Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

Sep 2026 · arXiv.org · Vol abs/2609.31076 · 0 citations · 52 references
Computer Science

TL;DR

Results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient, and releases CodeHack, the library of code-based skills with natural-language descriptions.

Abstract

Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting them invoke reusable skills instead of repeatedly selecting individual actions. The code handles recurring local decisions, while the language model decides which skills to use and how to combine them. Yet abstractions are leaky, and situations beyond a skill's capabilities may require a return to primitive actions. Motivated by this tradeoff between productivity and flexibility, we systematically study how code-based action abstraction affects the performance, inference cost, and learning of language agents. We study this in NetHack, a challenging, long-horizon game environment, using CodeHack, our library of code-based skills with natural-language descriptions. We use this library to compare agents restricted to primitives with those using semantic skills alone or in combination with primitives. We evaluate these agents in three settings: zero-shot prompting, supervised fine-tuning, and reinforcement learning. Across a broad zero-shot evaluation on NetHack, we find that compared with primitives, skills nearly triple game progression, while reducing inference cost per episode by 86%. Combining skills with primitives retains much of this benefit while preserving a path back down to low-level actions. Finally, in RL, we find that skill-based agents learn significantly faster than agents acting on primitives, achieving a 7.2x larger average gain in dungeon level over the same training budget. These results show that a supplied skill library can improve performance, efficiency, and learning, while retaining primitives provides flexibility when the library is insufficient. We release CodeHack together with training and evaluation code.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

Exploring Collaboration between a language and a non-language agent

To solve LLM collaboration with non-language agents, latent state internalization is introduced, which projects the subagent's continuous representations directly into the LLM's token stream as learned state tokens, with dynamic re-encoding as actions advance the environment state.

Harini S.I., Somesh Singh, Yaman Kumar Singla et al. · 1 citation

SAGE: Symbolic Action-Gating and Editing for LLM Task Planners

SAGE (Symbolic Action-Gating and Editing), a single-LLM planner built from two lightweight mechanisms: a domain-agnostic symbolic gate that blocks precondition-violating actions with typed reasons as a runtime safety monitor, and a local edit that regenerates only the failed sub-goal's suffix, keeping completed and unt...

T. Bui, Jongsul Moon, Youngouk Kim et al. · 0 citations
#artificial intelligence Preprint Aug 2026

CEDAR: Automata as Verifiable Interfaces for Language-Guided Embodied Action

CEDAR is presented, a counterexample-guided framework that grounds instructions as regular languages over environment event traces and represents both skills and specifications as deterministic finite automata, suggesting that regular languages offer a practical verification layer between natural-language instructions...

Le Chen, Alvaro Velasquez, Ashutosh Trivedi · 0 citations
#machine learning Preprint Oct 2026

Flow Policies as Actions of Skill-Level World Models: Learned and Symbolic Abstractions for Long-Horizon Planning

This work constructs skill-level actions from the inputs of a flow-matching policy trained on demonstrations segmented into complete skills, and proposes four action abstractions with increasing task knowledge: a compressed seed, two discrete codes learned from the demonstrations, and a symbolic label.

Andreu Matoses Gimenez, Andrei-Carlo Papuc, Christian Pek et al. · 0 citations
Preprint Aug 2026

SKILLER: Language-Level Reinforcement Learning for Reusable Skill Extraction in Small Language Models

SKILLER is a natural-language-driven reinforcement learning framework designed to automatically generate executor-specific skills for small models, which employs a strong model as the actor and critic, treats the small-model agent system as the environment, and propagates all reinforcement learning signals entirely via...

Chen-Hao Dang, Siyuan Xiong, Cong-Hui He et al. · 4 citations
Sep 2026

Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence

Vision-language-action (VLA) and world-action (WAM) models map observations and instructions directly to robot actions. This directness ties a policy to training: minor layout or viewpoint changes cause failure, and instructions generalize poorly. The root cause lies in representation: task requirements, conditions, pr...

Hong-Cheng Gao, Jing-Jing Zhou, Ze Zheng et al. · 7 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.