Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts
The complexity of ensuring consistent behavior as models become increasingly multimodal and retrieval-augmented is illustrated, showing that the longer processing time associated with resolving visual entities prevents the suppression of the model's usual factual recall mechanism, thus resulting in more parametric answ...