MeRoTune: RoPE-Safe Merging with a Tunable Dial
This paper derives the specific class of matrices where this cancellation is exact under RoPE if and only if $M$ commutes with RoPE's per-position rotation, and derives the specific class of matrices where this holds.
Salman Faroz
· 0 citations