Large language models (LLMs) demonstrate exceptional accuracy in password-guessing tasks, but their massive memory cost, high computational demands, and high duplicate rates severely limit deployment in resource-constrained scenarios. This paper introduces the 1-bit/1.58-bit quantized BitNet architecture to the passwor...
Xinyi Zhang, Jialin Chen, Fan Zhang et al.· Journal of King Saud Univers...· 0 citations
Spiking forecasting framework SpikeLite is introduced, a spiking forecasting framework built around two modules: a Frequency-Selective Spiking Encoder for frequency-sensitive temporal encoding and a Sparse Spiking Channel Attention (SSCA) module for selective cross-channel interaction.
Bang Hu, Chang-Ze Lv, Ming-Jie Li et al.· 0 citations
QuantaSpike is proposed, a short-window spike-driven quantization framework for LLMs built around Logarithmic Ternary Integrate-and-Fire (LTIF) neurons that achieves state-of-the-art or competitive perplexity and zero-shot accuracy among spike-driven LLM quantization methods.
Bang Hu, Guo-Wei Zhu, Changze Lv et al.· 0 citations
This work proposes ATAC, an anchor-tail aware framework for jointly optimizing packed-document construction and context-parallel sharding in large language model training and demonstrates that input-structure-aware co-design of packing and context parallelism is an effective approach to improving calibrated pipeline-le...
Zhengyu Liu, Shuai-Kang Hou, Yan-Zhao Gao et al.· Journal of King Saud Univers...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
