Skip to content

Author

Junzhuo Li

We have 3 of 11 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#small language model Preprint Oct 2026

HiLo-RD: Cross-Budget Self-Distillation for Masked Diffusion Language Models

We propose HiLo-RD (High-to-Low Budget Representation Distillation) to improve masked diffusion language models (MDLMs) under a limited generation budget. Unlike autoregressive models, MDLMs choose both what to generate and where to commit it. This creates two coupled challenges: predicting useful content before reason...

Xiang Li, Cheng-Ting Yu, Jun-Gang Li et al. · 0 citations
#machine learning Preprint Sep 2026

SMAT: Simple and Efficient Merge-Aware Training

SMAT (Simple MAT) is introduced, which jointly optimizes expert loss and expected loss at simulated merged parameters generated by sampling scaling coefficients, masks, and additive noise, and improves the mean score across five merging methods.

Yang-Gan Gu, Yuan-Yi Wang, Zhen Li et al. · 0 citations

Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation

A systematic analysis of expert routing patterns in MoE models reveals Language Routing Isolation, in which high- and low-resource languages tend to activate largely disjoint expert sets, and proposes RISE, a framework that exploits routing isolation to identify and adapt language-specific expert subnetworks.

Kening Zheng, Wei-Chieh Huang, Jiahao Huo et al. · 4 citations · ⚡2

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.