This work develops a self-contained methodology for learning parameter-efficient rotational transformations based on Riemannian optimization and empirically validate the proposed rotation-based steering scheme, demonstrating its superiority in intervention efficiency.
Abstract
Activation steering has emerged as a lightweight approach for controlling model refusal at inference time. A growing line of research explores trainable rotations of activations to develop geometrically principled intervention mechanisms. However, existing techniques rely on auxiliary constructs, such as refusal vectors, to define these rotations. In our work, we develop a self-contained methodology for learning parameter-efficient rotational transformations based on Riemannian optimization. We empirically validate the proposed scheme, demonstrating its superiority in intervention efficiency. An extensive ablation study highlights the importance of key design choices in our method. Our results identify the proposed rotation-based steering scheme as a promising direction for more reliable control over the behavior of LLMs.
Steering Vector Dissection is proposed, a framework to explicitly isolate individual and semantically consistent features from these composite directions that tend to entangle multiple semantic and stylistic concepts into a single composite direction, leading to unpredictable steering effects.
Takeru Hiramatsu, Kyohei Atarashi, Koh Takeuchi et al.· 2 citations
The proposed Riemannian LQT formulation is evaluated in a set of toy examples and the full framework is evaluated on a cutting task requiring high precision in a minimally invasive surgery setting on the da Vinci Research Kit (dVRK).
Maximilian Mühlbauer, Marcella Piacentino, R. Mancino et al.· 0 citations
Dexterous twisting is a fundamental yet challenging capability for anthropomorphic manipulation due to its high-dimensional action space, strong spatio-temporal coupling, and complex contact dynamics. While reinforcement learning (RL) often struggles with convergence under such conditions, learning from demonstration (...
Yan-Bin Feng, He Li, Fu-Zhang Han et al.· 2026 IEEE 22nd International...· 0 citations
Across multiple dexterous hands, EigenDEXplore consistently outperforms joint-space and learned action-space baselines in grasping, in-hand reorientation, and contact-rich manipulation, and are largest in settings with less reward shaping and curriculum design.
Harsh Gupta, T. Lum, Chang-Hao Wang et al.· 0 citations
As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inference-time alternative to fine-tuning methods (e.g., RLHF, DPO) for behavioral control. However, existing work typically validates steering on isolated behaviors, leaving it u...
Mohammad Mahdi Abootorabi, Armin Saghafian, Ali Bazshoushtari et al.· 0 citations
This work introduces First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and develops a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher...
Sri Pranav Kunda, Alexander Kurz, T. Dominik et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
