On-policy self-distillation (OPSD) enables a language model to learn from a Teacher instantiated from the same model but given access to privileged information, such as a reference solution. We find that access to sufficient privileged information allows the Teacher to generate highly accurate trajectories; however, ac...
Lin-Fei Ji, Jia-Yu Li, Hao-Zhao Wang et al.· 0 citations
SimPattern is a similarity detection method motivated by the characteristics of delta compression that enables more fine-grained similarity detection and substantially improves the effectiveness of delta compression, reducing the final storage size by up to 42.68% compared with state-of-the-art methods.
Zhen-Yu Cai, Xu-Ming Ye, Wen-Long Tian et al.· Proceedings of the Internati...· 0 citations
Experiments show that GDLoRA consistently improves over LoRA and narrows the performance gap to FFT, and incorporates complementary normal gradients without increasing standard LoRA's optimizer-state memory budget under matched adapter and optimizer configurations.
Yi Ouyang, Shi-Wei Li, Hao-Zhao Wang et al.· arXiv.org· 0 citations
This paper proposes FR-AKP, which performs the augmented knowledge purification on the basis of data augmentation at both the intra-client and inter-client levels, and significantly outperforms state-of-the-art methods.
Yijing Shan, Hao-Zhao Wang, Yi-Chen Li et al.· Proceedings of the 32nd ACM...· 0 citations
This workshop seeks to bring together researchers and practitioners from academia and industry to explore the convergence of federated learning and multi-agent systems, with a focus on scalability, trustworthiness, adaptive coordination, and the broader societal impact of federated multi-agent intelligence.
Hao-Zhao Wang, Zhuangdi Zhu, Zheng Xu et al.· Knowledge Discovery and Data...· 0 citations
A novel method named IED is proposed, which leverages the information gain to construct the vector of which each dimension represents the information entropy of each word, and then adopts a classifier to conduct the detection.
Xiao-Quan Yi, Hao-Zhao Wang, Jing-Cai Guo· Proceedings of the Thirty-Fi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.