Reliable FP8 attention remains a barrier to fully native 8-bit large language model training. We derive how forward-backward inconsistencies produce stale delta and empirically show how it distorts training dynamics. Our stale-delta hybrid runs show a modest loss gap at 569M parameters but substantial loss increases an...
Hao-Zhan Tang, Hao Kang, Han Cai et al.· arXiv.org· 0 citations
Reliable FP8 attention remains a barrier to fully native 8-bit large language model training. We derive how forward-backward inconsistencies produce stale delta and empirically show how it distorts training dynamics. Our stale-delta hybrid runs show a modest loss gap at 569M parameters but substantial loss increases an...
Hao-Zhan Tang, Hao Kang, Han Cai et al.· 0 citations
This paper systematically investigates two principal test-time scaling axes of LLM agents: sequential scaling through extended interaction and parallel scaling through trajectory sampling, and finds that neither scaling axis can consistently yield meaningful gains from additional test-time compute in realistic environm...
Xiao-Chuan Li, Ryan Ming, P. Setlur et al.· 23 citations· ⚡1
This work proposes SkillLift, which decouples skill search from oracle cost by learning an oracle-aligned rubric as a structured evaluation space, and outperforms existing auto-skill methods with 40--70\% less token cost compared to frontier evolving methods.
Hao Kang, Ming Wen· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.