As multimodal large language models (MLLMs) reason for longer, attention to the initial visual input diminishes, weakening visual grounding. Visual memory reintroduces visual evidence during reasoning. We conduct a controlled analysis of visual memory along three axes: curation, organization, and access. We find that l...
Hao-Wen Jiang, Zhan-Yu Guo, Chen-Wei Wu et al.· 0 citations
Multimodal large language models (MLLMs) have demonstrated impressive performance in multimodal understanding, but processing large numbers of visual tokens results in high computational costs. While many methods have been proposed to reduce the number of visual tokens, most of them rely on heuristics and are prone to...
Ting-Hao Wang, Yi-Chen Guo, Qi-Zhe Zhang et al.· 0 citations
These findings show that effective pruning should preserve broad visual coverage before fusion and progressively refine the retained tokens as cross-modal evidence evolves during fusion, and propose STAR-Pro (STage-Wise Adaptive Token Reduction with Progressive Refinement), a training-free two-stage framework.
Yi-Chen Guo, Ting-Hao Wang, Qi-Zhe Zhang et al.· 1 citation
Improved text-visual attention patterns are introduced to enhance the fidelity of query-aware vision token selection and the Attention Gravity effect is correct, and a rank-based strategy to adaptively determine the sparsification ratio for each layer is introduced.
Yuan Zhang, Junpeng Ma, Qizhe Zhang et al.· IEEE Transactions on Pattern...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.