Wayfarer is presented, a general, domain-agnostic, online deep RL agent that discovers options through Laplacian representation learning from high-dimensional observations and leverages them for control, enabling substantially faster learning of complex policies.
Abstract
Temporal abstractions, often instantiated as options, have long been regarded as a mechanism for accelerating credit assignment, facilitating exploration, and enabling generalisation in reinforcement learning (RL). However, developing general option discovery methods that are effective in large-scale, high-dimensional domains remains a fundamental challenge. Existing option discovery methods are either confined to relatively simple domains, depend on handcrafted or quasi-symbolic representations, or offer little improvement over learning without options. We present Wayfarer, a general, domain-agnostic, online deep RL agent that discovers options through Laplacian representation learning from high-dimensional observations and leverages them for control. We show that the resulting options simultaneously improve exploration, accelerate credit assignment, and generalise effectively to unseen settings, enabling substantially faster learning of complex policies. Wayfarer achieves state-of-the-art performance among single-stream agents on the most challenging Atari 2600 games, with the largest gains in games that require long-horizon exploration and strategic behaviour, such as Montezuma's Revenge and Private Eye.
Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas absent from its prior training data. In practice, however, augmenting RLVR with str...
While reinforcement learning with verifiable reward (RLVR) enhances reasoning in large language models, its efficacy in long-horizon multimodal tasks remains underexplored. Visual Sudoku puzzles pose a representative challenge, requiring multi-step deductive reasoning over both local constraints and global dependency c...
Si-Da Li, Yuan-Xin Liu, Lin-Li Yao et al.· Conference on Empirical Meth...· 0 citations
This work introduces ThunderSyncRL, which starts gradient computation as soon as all required inputs are fixed, without policy staleness, and proves that gradient streaming produces the same GRPO and OPD updates as batch-synchronous training, without changing either objective.
Seil Kang, Hangoo Kang, Tarun Suresh et al.· 0 citations
TrajEvolve reformulates step-level continuous credit assignment as a binary classification task, whether each step is necessary for task completion, and simplification not only reduces the learning difficulty but also enables step labels to be obtained through counterfactual verification, achieving precise identificati...
This work introduces an algorithm that identifies a small, relevant subset of features for each subgoal, yielding options that generalize broadly and accelerate exploration in three sparse-reward, image-based domains, including the Atari game MontezumasRevenge.
Akhil Bagaria, Anita de Mello Koch, G. Konidaris· arXiv.org· 1 citation
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026