Skip to content

6 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Sep 2026

Delta-Matching: Closing the Final Gap of Native 8-bit Training for LLMs

Reliable FP8 attention remains a barrier to fully native 8-bit large language model training. We derive how forward-backward inconsistencies produce stale delta and empirically show how it distorts training dynamics. Our stale-delta hybrid runs show a modest loss gap at 569M parameters but substantial loss increases an...

Hao-Zhan Tang, Hao Kang, Han Cai et al. · 0 citations
#machine learning Preprint Sep 2026

Delta-Matching: Closing the Final Gap of Native 8-bit Training for LLMs

Reliable FP8 attention remains a barrier to fully native 8-bit large language model training. We derive how forward-backward inconsistencies produce stale delta and empirically show how it distorts training dynamics. Our stale-delta hybrid runs show a modest loss gap at 569M parameters but substantial loss increases an...

Hao-Zhan Tang, Hao Kang, Han Cai et al. · 0 citations

Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

Jet-Long is proposed, a tuning-free zero-shot method that pairs a local RoPE-faithful window with a long-range window whose rescaling factor adapts dynamically to the current sequence length via a parameter-free analytic schedule, recovering the base model exactly at short inputs while extrapolating cleanly at long one...

Hao-Zhan Tang, Zerui Wang, Yu-Xian Gu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition

Lightning Weave is introduced, a post-training framework that extracts and composes independently learned capabilities in a single student through on-policy distillation and achieves a state-of-the-art accuracy-efficiency frontier.

Ye-Cheng Wu, Song Han, Han Cai · 0 citations

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

DC-Gen, a general framework that accelerates text-to-image diffusion models by leveraging a deeply compressed latent space, uses an efficient post-training pipeline to preserve the quality of the base model to reduce the latency of 4K image generation.

Wen-Kun He, Yuchao Gu, Jun-Yu Chen et al. · 9 citations · ⚡1

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.