Aug 2026· arXiv.org· Vol abs/2608.15058· 1 citation· 28 references
Computer Science
TL;DR
This work proposes Multi-Signal Event Modeling and Dynamic Rescoring (MEDR), a training-free and query-independent frame selection method that preserves the reusability of a fixed visual input while improving the coverage of informative events beyond uniform sampling.
Abstract
Frame selection is a fundamental component of multimodal large language models, enabling long videos to be processed under limited visual-token and computational budgets. Uniform sampling preserves temporal coverage but may miss informative content that appears only briefly. To alleviate this limitation, query-dependent methods can retrieve question-relevant frames. However, because the selected frames depend on the current question, the same visual input cannot be directly shared across different questions, and frame selection must be repeated in multi-turn video dialogue. This motivates us to seek a query-independent frame selection method that preserves the reusability of a fixed visual input while improving the coverage of informative events beyond uniform sampling. We propose Multi-Signal Event Modeling and Dynamic Rescoring (MEDR), a training-free and query-independent frame selection method. Multi-Signal Event Modeling organizes complementary visual, motion, and text signals into signal-specific temporal events. Dynamic Rescoring then iteratively reevaluates each candidate relative to the current selected set, updating its score according to frame-level signal strength, additional event coverage, and balanced temporal coverage. The resulting fixed frame set is constructed without observing the query and can be reused across different questions. Experiments on Video-MME and LongVideoBench demonstrate that MEDR achieves competitive performance across two video MLLMs. Under the 64-frame budget, MEDR improves Video-MME accuracy over the strongest evaluated query-independent baseline by 1.33-1.78 percentage points. We further evaluate the reuse of a fixed frame set in multi-question video dialogue.
This work introduces a frame-selection method designed specifically for multiple-choice questions, and is the first to directly utilize answer choices as inference-time cues for selecting frames relevant to answering a question.
In the era of big data, precise event retrieval from large video collections has become increasingly important and challenging. In particular, locating multiple events from the same video in their temporal order is difficult when different events are described by different types of evidence, such as visual content, on-...
This work introduces FORTE, a training-free framework that addresses the challenge of adaptive relevance scoring and global keyframe optimization through two stages: adaptive relevance scoring and global keyframe optimization.
Haifeng Huang, Bi-Yin Xu, Chun-Sheng Xin et al.· 0 citations
Long-video understanding with multimodal LLMs is fundamentally constrained by the mismatch between video length and the model's limited visual context budget, making frame selection essential for practical reasoning. Existing query-aware methods typically rely on heuristic pipelines: they assign one-dimensional query-c...
Long video understanding requires models to identify and reason over sparse, temporally distributed evidence within limited frame budgets. Existing keyframe selection methods typically retrieve query-relevant frames from a densely sampled candidate set before inference. However, global image-text matching incurs unnece...
Yi-Song Hu, Jun-Bo Wang, Yu-Ke Li et al.· 0 citations
Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most o...
Yu-Hao Qin, Jun-Bo Wang, Yu-Ke Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.