Representation Control for Large Language Models: Survey and Research Challenges

Abstract

Large language models (LLMs) are capable of completing a variety of tasks, but remain unpredictable and intractable. Representation Control (RepControl) seeks to resolve this problem through targeted interventions that modify high-level representations of concepts such as honesty, harmfulness or power-seeking. We formalize the goals and methods of RepControl to present a cohesive picture of work in this emerging field, focusing on techniques that steer model behavior by manipulating internal activations at inference time. We compare these control methods with alternative approaches, such as prompt-engineering and fine-tuning. We outline challenges such as performance degradation, computational overhead, and limitations in steering precision. We present a clear agenda for future research to build more steerable, personalized, and reliable LLMs through advances in RepControl techniques.