MoE facilitates the development of large models by making the computational complexity of the model no longer scale linearly with increasing parameters. The learning sparse gating network selects a set of experts for each token to be processed; however, this may lead to differences in the number of tokens processed by...
Peizhuang Cong, Aomufei Yuan, Shimao Chen et al.· 0 citations
This work is the first work to include hidden layer distillation for student MLP on graphs and to combine graph Positional Encoding with MLP, and is the first work to include hidden layer distillation for student MLP on graphs.
Junxian Li, Bin Shi, Er-Fei Cui et al.· arXiv.org· 1 citation
Network traffic analysis in cloud and internet service provider (ISP) networks increasingly relies on machine learning (ML) models, as encryption and service consolidation make rule-based classification unreliable. At high flow-arrival rates, however, new flows can arrive faster than end-to-end analysis pipelines can p...
Shinan Liu, Ted Shaowang, Gerry Wan et al.· 0 citations
This paper considers the setting in which local Mixture-of-experts (MoE) models are fitted independently on distributed datasets and must be combined into a single global MoE model. Weighted averaging the local mixtures, whose experts are not aligned across machines, often yields a large mixture with many experts inste...
Fa\"icel Chamroukhi, Nhat Thien Pham· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Diffusion models have shown promising results for a wide range of generative tasks with continuous data, such as image and audio synthesis. However, little progress has been made on using diffusion models to generate symbolic music since this class of generative models struggles with discrete data while its iterative s...
Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. However, the application of LLMs in the recommendation domain has not been thoroughly investigated. To bridge this gap, we propose LLMRec, a LLM-bas...
Junling Liu, Chao Liu, Peilin Zhou et al.· 0 citations
We present NeuSE, a novel Neural SE(3)-Equivariant Embedding for objects, and illustrate how it supports object-based Simultaneous Localization and Mapping (SLAM) for consistent spatial understanding with long-term scene changes. NeuSE is a set of latent object embeddings created from partial object observations. It se...
Jiahui Fu, Yilun Du, Kurran Singh et al.· 0 citations
Reasoning-based large language model (LLM) safeguards improve safety classification accuracy by generating explicit rationales before making decisions. However, generating explicit rationales adds substantial decoding overhead to every safety decision, while readable explanations are needed only for on-demand audits. T...
Zhinan Liu, Jie Li, Mingyu Kang et al.· 0 citations
Structure-preserving de-identification replaces each detected identifier with a realistic same-type surrogate ("Anna S." -> "Maria S.") rather than a tag such as [NAME]. If downstream PHI detectors no longer recognise surrogates, hiding-in-plain-sight protection and second-pass safety nets weaken. We ask: on the spans...
Qiming Bao, Sherry J. H. Feng, Kim Chester Eugenio et al.· 0 citations
Semantic caching, reusing responses to answer semantically similar queries, has seen growing adoption in LLM serving, offering faster responses and reduced costs. Yet existing schemes are fundamentally vulnerable to cache-collision attacks, wherein an adversary pollutes the cache by injecting adversarial queries, corru...
Jiacheng Liang, Yuhui Wang, Tanqiu Jiang et al.· 0 citations
The preliminary analysis reveals that benign and poisoned trajectories exhibit distinguishable relationships between the initial user intent and subsequent behavior, which mitigates information redundancy in multi-turn contexts while avoiding the overhead of repeated LLM auditing.
Failure attribution for LLM-based agentic systems, i.e., identifying which steps in a failure trajectory caused the task to fail, is critical for debugging and improving these systems. Existing approaches either rely on prompting-based pipelines, which are computationally expensive, or require post-training on failure...
Samuel Yeh, Yiwen Zhu, Shaleen Deep et al.· 0 citations
