Semantic Association vs. Position: Which Information Do Attention Heads Prefer?

Abstract

In large language models (LLMs), attention heads generally encode both semantic association information (SAI) and positional information (PI). This raises a fundamental interpretability question: Which does each attention head relatively prefer, SAI or PI? To address this question, based on theoretical analysis and derivations, we propose a unified relative-preference measurement method. We also define two types of high-preference heads: SAI-preference heads and PI-preference heads. Specifically, we formalize the problem and restrict the research object to attention heads in RoPE-based LLMs (covering single-head/multi-head). We further derive a theoretical expression of the relative-preference measure by introducing an SAI-reduction mask. However, the expression involves an unknown function and lacks directly usable labeled datasets to learn it. For computation, we design an estimator of SAI-preference to approximate the theoretical expression and derive a sufficient condition for the maximum of the estimator. Finally, we propose a unified and computable relative-preference measurement method. In experiments, we present computed results and visualizations of information preference across multiple RoPE-based LLMs. We provide two interpretable applications. (1) We reveal interpretable layer-wise distribution patterns of the two types of high-preference heads. (2) We visualize and analyze the interpretable intervention effects of pruning high-preference heads. Then, we conduct intervention experiments on multiple datasets under a small budget and further find that there exists a certain type of high-preference head that can be pruned to yield a positive or at least non-negative gain on a given dataset in most cases.