Jingcheng Niu

Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs

This study provides new mechanistic insights into how LLMs internalize OOD challenges, and designs Sparsity-Guided Curriculum In-Context Learning (SG-ICL), a strategy that explicitly uses representation sparsity to schedule few-shot demonstrations, leading to considerable performance enhancements.