Lens methods inspect model computation by mapping intermediate activations to vocabulary tokens. Yet the concepts humans need to read out often span multiple tokens---entities, phrases, intermediate objects---making token-level readouts incomplete. Reliable multi-token readout with little model-specific preparation rem...
This work studies causal interventions in controlled multiple-choice tasks and identifies that persuasion mainly acts through a small set of mid-layer attention heads, which are decision heads, which mainly acts through a one-dimensional option-routing feature in the option-token representations.
Xiang-Kun Sun, Ling-Kai Kong, Ao Zhang et al.· arXiv.org· 2 citations· ⚡1
Across three model families, two small groups of attention heads are identified with distinct roles in introspective reporting supporting introspective detection and localization of large language models.
Jia-Hong Zou, Xiang-Kun Sun, Ling-Kai Kong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.