Skip to content

Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation

Sep 2026 · arXiv.org · Vol abs/2609.20744 · 4 citations · 70 references
Computer Science

TL;DR

This work instantiates VDN on MiniMax H3, which combines local Softmax attention with bidirectional linear memory for long-range video context and introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens.

Abstract

Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major computational bottleneck. Linear attention offers an appealing alternative and has been widely adopted in recent large language models, but directly applying it to video models often fails to preserve the fine-grained interactions required for high-quality generation. We present Video DeltaNet (VDN), which combines local Softmax attention with bidirectional linear memory for long-range video context. Its linear branch introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens. Separate output projections and learnable gates calibrate the two branches, while a staged teacher-alignment recipe progressively introduces the new pathway into pretrained models. We instantiate VDN on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax for interactions involving text or audio. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising for a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, corresponding to a 14.5x speedup over the 50-step dense H3 baseline on the same GPU count. GitHub code available at: https://github.com/OpenVDN/vdn-minimax-h3. Weights available at: https://huggingface.co/OpenVDN/vdn-minimax-h3

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Recast the Past: Stable Long Video Generation with Full Context Memory

This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer, which learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-le...

Yu-Kuo Ma, Cong Liu, Junke Wang et al. · 0 citations
Preprint Oct 2026

Velox: Spatial Memory Makes World Models Efficient

Video world models turn actions and camera motion into long visual rollouts, making video a natural interface for interactive simulation. Such interaction requires responsive generation, yet coherent rollouts must retain visual memory of what has already been generated. Chunked video diffusion models usually keep this...

Qiuhong Shen, Shi-Zun Wang, Xin-Chao Wang · 1 citation
Preprint Oct 2026

Unified Generative Video Compression with Autoregressive and Bidirectional Video Diffusion

Video diffusion models can reconstruct visually compelling content from severely compressed representations at ultra-low bitrates, but they impose a rigid deployment trade-off. Bidirectional (BI) models achieve high quality through dense full-sequence attention, yet require the entire sequence before reconstruction and...

Tian-Yu Zhang, Shuai Huo, Deng-Chao Jin et al. · 0 citations
Open access Sep 2026

Bridging Text and Motion: Generative AI Models for Video Synthesis

Text to video generation has advanced significantly in recent years, largely due to the development of extremely sophisticated diffusion models. In this work, we present a novel ap- proach to producing excellent video content based on descriptions by utilizing diffusion tech- niques. Using a multi-stage diffusion proce...

Mohammad Shahnawaz Shaikh · 0 citations
Preprint Oct 2026

Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training

Natively training joint video-audio generation models at higher resolutions empowers them to learn richer visual details and sharper motion dynamics. However, full attention incurs quadratic cost and, as resolution increases, spreads attention over increasingly redundant tokens, diluting learning signals for informativ...

Shu-Yuan Tu, Qi Tian, Yin-Ming Huang et al. · 1 citation
Preprint Oct 2026

SGF+: Decoupling Gradient Flows for Autoregressive Video Generation

Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint op...

Zi-Han Su, Junhao Zhuang, Yao-Wei Li et al. · 4 citations · ⚡1

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.