Quantifying Logical Consistency in Transformers via Query-Key Alignment
This paper proposes a novel, lightweight evaluation strategy for logical reasoning that uses query-key alignments inside transformer attention heads that reveals latent representations that reliably separate valid from invalid inferences, offering a scalable alternative to traditional ablation-based techniques.
Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva et al.
· Conference on Empirical Meth... · 0 citations