Looped Transformers increase effective depth by repeatedly applying shared layers, offering a way to scale computation without increasing parameter count. However, full-stack looping places context construction and iterative state refinement within the same recurrent computation, without explicitly distinguishing their...
Cheng Liang, Teng-Fei Zhang, Wenjie Du et al.· 2 citations
UpLoop is a pretraining schedule that uses upcycling to train looped Transformers while exactly matching a standard dense baseline in final parameter count, total training tokens, and training FLOPs, showing improvements in language modeling and downstream performance.
Cheng Liang, Teng-Fei Zhang, Ya Zhang et al.· 0 citations
World action models (WAMs) have emerged as a promising paradigm for robotic control by jointly predicting future visual dynamics and actions from an initial observation and instruction. However, existing WAMs struggle with long-horizon prediction, as generating dense video rollouts is highly inefficient. Some recent WA...
Fei Zhang, Zhao-Chong An, D. Frost et al.· 1 citation
Alignment-Guided Flow Transformer (AGFT) is presented, a novel framework that explicitly enforces tri-modal alignment through a dedicated alignment loss, bridging the representational gap across modalities and enhancing task adaptation.
Sheng-Chao Hu, Peng Wang, Qi-Yang Zhou et al.· 0 citations
Q-learning Penalized Transformer policy (QPT), a framework that bridges conditional sequence modeling with constraint-aware value estimation, is proposed, characterizing how Q-penalized conditional generation improve safety and performance.
Sheng-Chao Hu, Peng Wang, Ji-Feng Hu et al.· 0 citations
Modular TTT is proposed, a framework that represents the inner learner as a directed acyclic graph and exposes the fast-weight network, loss function, learning rate, weight decay, and normalization as explicit design dimensions and finds that small learning-rate initialization, weight decay, and a single-layer nonlinea...
Bo-Hao Tang, Zhen Qin, Yu-Qi Pan et al.· 1 citation
FailForge is proposed, an agentic framework that converts failed rollouts into training signal, and recovers over 26% of previously failed instances at marginal additional cost, and training Qwen3.5-4B on the augmented corpus improves the SWE-bench Verified resolve rate by 6.6 points over a strong RFT baseline.
Dongyi Lv, E. Fushun, Aichen Cai et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.