Paniz Halvachi

AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models

Muon improves large-scale training by applying a spectral-norm steepest-descent update to matrix parameters, but practical models also contain parameter blocks that do not fit dense-matrix geometry. One important case is the tied vocabulary table, which appears in language models and other token generators and can rece...