Gal Mishne

Reassessing Muon for Matrix Factorization

It is found that Muon does not consistently outperform AdamW in this setting and that several previously reported advantages are sensitive to hyperparameter choices, providing a more nuanced picture of when spectrum-aware orthogonalization is beneficial and arguing for evaluating modern optimizers on controlled problems in addition to end-to-end benchmarks.