Yuning Qiu

Bayesian sparse tensor wheel decomposition for missing data recovery

Shared Low-rank Basis Factorization for Data-free Mixture-of-Experts Compression

Mixture-of-Experts (MoE) large language models decouple capacity from compute through sparse routing, but their large parameter count creates storage and serving challenges. We analyze three MoE compression families: expert pruning, expert merging, and weight reconstruction, and derive structural error bounds showing t...

Bayesian adaptive tensor ring decomposition with automatic model selection.

A robust non-parametric Bayesian method known as the Bayesian adaptive tensor ring decomposition (BATR) method, which accomplishes an adaptive low tensor ring (TR) rank model by incorporating a more advanced generalized hyperbolic (GH) prior into the probabilistic framework, thereby facilitating automatic TR rank deter...

Adaptive Ranks for Personalized Federated Large Language Models Under Parameter Budget Constraints

The advent of Large Language Models (LLMs) represents a leap toward leveraging AI for the benefit of humanity. However, the realization of this potential requires addressing the growing need for personalization and data privacy across diverse entities. Federated Learning (FL) offers a vital paradigm for developing priv...