Training large language models is constrained by both wall-clock time and memory footprint, and recent matrix-aware optimizers sharpen this tradeoff: Muon can improve optimization but its dense polar update remains expensive. We propose Monarch-Muon, a structured matrix-aware optimizer for weights parameterized by Mona...
A. Miasnikov, R. Kabirov, G. Molodtsov et al.· 0 citations
This work develops a self-contained methodology for learning parameter-efficient rotational transformations based on Riemannian optimization and empirically validate the proposed rotation-based steering scheme, demonstrating its superiority in intervention efficiency.
Kirill Bunin, Dmitry Bylinkin, Vladimir Aletov et al.· 0 citations
A novel white-box attack inspired by locate-then-edit approaches from the field of Knowledge Editing is proposed, modifying the editing framework by incorporating as- sociative knowledge retrieved from the model, thereby extending constraint removal to an entire thematic category rather than being limited to prompts fr...
R. Maksimov, Vladimir Aletov, V. Solodkin et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.