Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal understanding, but their practical deployment is still limited by the substantial computation introduced by excessive visual tokens. Visual token pruning offers an effective solution by removing redundant visual tokens during inference...
Peng-Cheng Zheng, Chao-Ning Zhang, Ya Wen et al.· 0 citations
Temporal Self-Conditioning Diffusion Language Model (TDLM) is introduced, which lets each unresolved position attend to its own history of hidden representations within a frozen denoiser, which gives later predictions access to computation already performed.
Bo-Han Sun, Max Zhu, Francesco Caso et al.· 0 citations
Reducing memory use in large language model (LLM) training is an ongoing challenge. Activation checkpointing saves memory by recomputation, while lossy compression changes the stored activations. The reemergence of sparse $\relu$ and $\relu^2$ activations creates a third opportunity: losslessly compressing the activati...
Max Zhu, Bo-Han Sun, P. Lio' et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.