In in-context Mealy-machine execution, where a simulator supplies intermediate targets and exact counterfactuals for hidden-state interventions,Looped transformers compute by applying one block repeatedly, so they can run for more loops than they were trained with.
Pei-Long Zhou, Hong-Tao Zhang, Yu-Long Huang et al.· 0 citations
This framework derives loop transitions in closed form from a projection, a local objective and an optimizer update rule, which combines explicit weight decay, adaptive step size and a delta objective and improves average generative performance over the compared looped and non-looped baselines under matched training FL...
LeapQuant is proposed, a training-free method that achieves near-lossless performance under 8-bit recurrent-state quantization and substantially reduces memory and compute costs during inference.
HyperTransfer is proposed, which constructs a Hyperball optimizer that reproduces the dynamics of a target Base Optimizer using only its initialization and learning-rate schedule, without running the target optimizer itself, to extend the framework to non-scale-invariant networks.
Jinghui Yuan, Hong-Tao Zhang, Jade Zou et al.· 0 citations
This work proposes Software-Interposed Datapath (SID), an efficient, software-flexible, and low-cost solution for rack-scale interconnects, particularly optimized for fine-grained memory access.
Chenxingyu Zhao, Yibo Wu, Hong-Tao Zhang et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.