The Muon optimizer applies spectral orthogonalization to matrix-valued updates and has shown strong performance in large-scale neural network training, yet the mechanisms of this transformation in feature learning remain poorly understood. In this work, we investigate this question through a tractable factual-recall mo...
Xu-Heng Li, Qi-Wei Di, Yuan Cao et al.· 0 citations
The analysis from independent random search to a broader family of local search operators, and it is proved that when sensitivities vary across instances, no adaptive policy can avoid worst-case regret that grows linearly in the trajectory length.
Yuan Cao, Yi-Fu Tang, Hang-Qi Li et al.· 0 citations
This paper forms CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors and proposes a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula.
Guang Yang, Wei Shi, Yuan Cao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.