Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models
Three attention-guided strategies are proposed: Attention-JSD, Attention-Entropy-Max, and Attention-Entropy-Min, which leverage structural information carried by internal self-attention mechanisms as a signal for layer selection and observe significant gains on multi-answer metrics.