Reassessing Muon for Matrix Factorization
It is found that Muon does not consistently outperform AdamW in this setting and that several previously reported advantages are sensitive to hyperparameter choices, providing a more nuanced picture of when spectrum-aware orthogonalization is beneficial and arguing for evaluating modern optimizers on controlled problems in addition to end-to-end benchmarks.