FlashAttention-Compatible Modality-Mutual Attention with Virtual Reordering
Multimodal large language models (MLLMs) process an interleaved image–text sequence with a single autoregressive backbone under causal attention. Since the causal mask prevents image tokens from attending to the question that follows them, their representations are computed without access to the query, which has been i...