Mirror, Mirror on the Wall: Prompt Echoing in Small Instruct Language Models
This work investigates prompt echoing small language models from different families and shows that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model's induction heads.
Inez Okulska, B. Naskrȩcki, Jan Piotrowski et al.
· 0 citations