Vera Demberg

A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models

The results reveal an interpretable mechanism for context dependent state tracking in LLMs, a retrieval conditioned rebinding mechanism, a compact attention head circuit that propagated binding information and when the entity is queried, uses the updated binding to retrieve the corresponding attribute.