First-Order Steering: Translating Weight Adaptation into Activation Steering
This work introduces First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and develops a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher...