Language Model Activations Inhabit Privileged Error-Correcting Basins
Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize that this robustness is a result of passive dynamics, i.e., constraining mechanisms in the forward pass that funnel activations toward"good"r...
Matthew Finlayson, Francisco Pernice, Eric Todd et al.
· 0 citations