Video generation has progressed from isolated clips to coherent short-form audiovisual sequences, and the bottleneck is shifting from rendering to semantic control. This report summarizes the observations and insights on multimodal understanding gained from successive generations of Kling. We argue that generation qual...
Kimi Team, Dong Chen, Jia-Lu Chen et al.· 0 citations
Fully Sharded Data Parallelism (FSDP2) maintains parameters and gradients as per-parameter tensors, whereas its collective path operates on flat, rank-major buffers. Converting between these layouts copies all-gather outputs into parameter storage and packs gradients into full-group fp32 buffers. Layout conversion alon...
Ze-Jian Wang, Yi-Fan Zhao, Kai-Hui Ren et al.· 0 citations
Full 3D attention is costly in video Diffusion Transformers. Block-sparse attention reduces this cost by computing only selected attention blocks. However, existing methods either overlook the potential to optimize sparsity allocation or rely on heuristic rules or local metrics to guide it. In this work, we observe tha...
Tian-Ze Xia, Xiao-Ling Zhou, Ru-Ya Lan et al.· 0 citations
This work introduces Schur Replay, a scale-selection algorithm that reproduces the GPTQ updates caused by each block scale and scores the resulting block error after accounting for compensation from unquantized columns.
Rui-Ying Ding, Jie Li, Kang He et al.· 0 citations
This work formalizes per-batch dispatch as a fixed-charge makespan problem---NP-hard on two fully replicated GPUs, polynomial in degenerate limits---and presents TEMPO, a makespan-aware dispatcher solving it in milliseconds off the critical path; its SGLang integration runs out-of-process and fuses dispatch with count...
Jie Li, Chen-Xin Jia, Jinliang Shen et al.· 1 citation
JAGG approximates intermediate-step Jacobians via $t$-weighted interpolation of the endpoint Jacobians, then aggregates per-step upstream signals into two composite gradients applied through a single joint backward pass, and it is proved this interpolation is exact when the velocity is linear in $(z,t)$, and a cosine-s...
Rui-Ying Ding, Jie Li, He Kang et al.· arXiv.org· 0 citations
This work proposes Latent Reward Registers, a mechanism that estimates terminal preference directly from intermediate noisy latents, and achieves significant reward improvement with a favorable reward-quality balance against training-free baselines.
Yuan-Shen Guan, Zipeng Feng, Cheng-Ru Song et al.· 0 citations
A lightweight Burst-Gap model parameterized by backpressure-free issue time, effective drain rate, and outstanding capacity predicts issue overhead, recovery between bursts, and the onset of backpressure, and two communication-computation fused kernels are redesigned.
Jianwen Xian, Zhiyuan Xu, Yucheng Li et al.· arXiv.org· 0 citations
HeadCast is proposed, a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's attention heads exhibit stable, heterogeneous behaviors that accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and lar...
Jinliang Shen, Li Su, Zheming Li et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.