Preprint

Alaya-EVOKE: From Linear-Scaling Supervision to Endless World

Yuanyang Yin Gongxuan Wang Y. Zhan Chuanhao Li Kaipeng Zhang Feng Zhao

Aug 2026 · 2 citations · 63 references

Computer Science

TL;DR

Alaya-EVOKE (Evoke) addresses both limitations by externalizing persistent world state and redesigning the teacher for long-horizon interactive generation, and achieves state-of-the-art performance on WBench while remaining competitive on VBench-Long and VBench-2.0.

Abstract

Interactive world models must support persistent memory, responsive interaction, and long-horizon generation, yet these requirements place conflicting demands on the model. Maintaining history in the denoiser context or key-value cache incurs growing cost, forcing a trade-off between session length and retained memory, while low-latency interaction relies on few-step generation whose capabilities are bounded by its teacher. Alaya-EVOKE (Evoke) addresses both limitations by externalizing persistent world state and redesigning the teacher for long-horizon interactive generation. Scene geometry is maintained in an external, camera-indexed world state bank, from which only view-relevant information is retrieved, keeping the denoiser context bounded as the session grows. Rather than treating the teacher as a fixed generator, we design it for long-horizon supervision: its sparse attention combines chunk-wise grouping, retrieval of selected distant frames, and a linear-attention global state, yielding linear growth in memory and compute while enabling supervision over long horizons. Such supervision exposes content drift that stays locally plausible within short windows, while per-chunk conditioning enables prompt changes and event control throughout the sequence. A 30-second distribution-matching objective, applied under self-forced rollouts, transfers both capabilities to a three-step student that uses no classifier-free guidance, improving resistance to long-term drift while preserving responsive conditioning. With bounded context and recurrent external memory, Evoke supports open-ended, continuously evolving generation; on a single H200 at $384\times 640$, each $1.5\,\mathrm{s}$ chunk is generated in $2.11\,\mathrm{s}$. As a three-step world model, Evoke achieves state-of-the-art performance on WBench while remaining competitive on VBench-Long and VBench-2.0.

arXiv Semantic Scholar

View source

Similar papers

Preprint Jul 2026

ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU

This work progressively distill a bidirectional action-conditioned teacher into a causal student through teacher forcing and ODE distillation, and introduces LongForcing to align long student self-rollouts with an extended-horizon teacher, mitigating accumulated distribution shift and autoregressive drift.

Fan Jiang, Zhaoxu Sun, Mengchao Wang et al. · 4 citations

Conference Open access 2026

Octopus: Gated Selective Attention for Memory-Bounded Long-Context Inference in Large Language Models

O CTOPUS is proposed, a framework that confers fixed-memory inference onto pretrained Transform-ers without the information loss of linearization and outperforms state-of-the-art linearized baselines on the GSM8K benchmark, demonstrating that learned sparse retention serves as an effective regular-izer for long-horizon reasoning.

C. Nguyen, Ryan A. Rossi, L. Van et al. · Annual Meeting of the Associ... · 0 citations

Preprint Aug 2026

ReWorld: An Interactive World Model with Long-Horizon Memory

ReWorld separates the two during training and bounds them at inference, and under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity.

Zhifei Chen, Luozhou Wang, Guibao Shen et al. · 0 citations

Preprint Aug 2026

MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents

MemOPD records the inputs and sampled outputs of each model invocation, restores its original token positions and causal visibility, and packs the reconstructed invocations for efficient teacher scoring, which improves F1 over PPO by up to 4162%, while packing yields up to a 1.63x speedup in actor computation during training.

Zhiyuan Liu, Tinghong Ye, Chenghao Liu et al. · 0 citations

Preprint Aug 2026

FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory

GUI agents must remember both useful experience from earlier tasks and unfinished progress in the current interaction. Latent memory offers a compact solution by compressing multimodal trajectories into a few continuous tokens. Existing methods, however, usually map each trajectory to one fixed memory block and train it mainly through next-action supervision. This creates three practical problems: important details may be lost during compression, the same memory block must serve different decision stages, and irrelevant retrieved trajectories may still mislead the agent. We introduce FocusMem, which separates these responsibilities within a compact latent-memory interface. A role-aware content basis encourages episodic memory to retain reusable experience and working memory to retain task progress. A state-conditioned readout generates a decision-specific view of the same stored evidence, while a lightweight trust gate can suppress memory blocks that appear irrelevant to the current step. All components are trained while the GUI policy remains frozen. Across five GUI-agent benchmarks, FocusMem consistently outperforms a fully matched action-only fixed-memory baseline and prior latent memory adaptations. Further analysis shows that semantic and functional supervision preserve complementary information, state-conditioned readout is more robust as surrounding trajectory context grows, and the trust gate reduces the harm caused by injected irrelevant episodic evidence. These results show that effective latent memory depends not only on compressing past interaction, but also on what is retained, what is exposed, and what is allowed.

Zhuoran Zhang, Bowen Li, Jingcheng Ju et al. · 0 citations

Preprint Aug 2026

Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality

ADU is presented, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling, and achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks.

Xunlei Chen, Qirui Ye, Yuang Li et al. · 0 citations