K-Means clustering is a classical unsupervised learning method widely used for its simplicity, efficiency, and broad applicability. In this work, we first analyze the numerical distributions of representative K-Means datasets and identify an opportunity for low-precision acceleration through hardware-native microscalin...
Rong-Tian Fu, Dong-Bo Lv, Xue-Ying Wang et al.· Proceedings of the Internati...· 0 citations
Graph Neural Networks (GNNs) have achieved widespread success from social networks to AI-for-Science. Most existing GNN frameworks adopt scatter-first (edge-centric) or gather-first (vertex-centric) scheduling paradigms for message passing. However, these paradigms are closely tied to traditional CUDA-core execution mo...
Jin-Liang Shi, Shi-Gang Li, Rong-Tian Fu et al.· IEEE Transactions on Paralle...· 0 citations
OmniPipe is proposed, a flexible bidirectional multi-pipeline parallelism scheme for unified dense and MoE LLM training that minimizes the pipeline bubble ratio while effectively overlapping EP communication with computation, enabled by the flexible and scalable parallelism scheme of bidirectional pipelines.
Jun Li, Zhi Ma, Shi-Gang Li· Proceedings of the Internati...· 0 citations
K-Means clustering is a classical unsupervised learning method widely used for its simplicity, efficiency, and broad applicability. In this work, we first analyze the numerical distributions of representative K-Means datasets and identify an opportunity for low-precision acceleration through hardware-native microscalin...
Rong-Tian Fu, Dong-Bo Lv, Xue-Ying Wang et al.· Proceedings of the Internati...· 0 citations
Mixture-of-Experts (MoE) has become the de facto architecture for scaling large language models, offering expanded capacity with manageable compute. Pipeline parallelism (PP) is indispensable for distributed MoE training, but state-of-the-art PP schemes face three major limitations: large pipeline bubbles, high per-sta...
Jun Li, Zhi Ma, Shigang Li· Proceedings of the Internati...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.