UvA-DARE (Digital Academic Repository) Lost in Time: A New Temporal Benchmark for VideoLLMs
· 0 citations
· 50 references
Similar papers
Preprint
Sep 2026
Not Another Text Benchmark: Putting the"Visual"Back in Visual Question Answering for Large Video Models
Three new vision-centric evaluation benchmarks in temporal frame retrieval, video future prediction, and causal memory distortion are introduced, all designed around evaluating visual understanding capabilities in large video models, which complements the existing approaches to evaluate video understanding in frontier...
Rwiddhi Chakraborty, Y. Wang, Cheng Zhang et al.
· 0 citations
UvA-DARE (Digital Academic Repository) Beyond Coarse-Grained Matching in Video-Text Retrieval
Experiments on a new approach for fine-grained evaluation demonstrate that this approach enhances a model’s ability to understand fine-grained differences.
Aozhu Chen, Hazel Doughty, Xirong Li et al.
· 0 citations
Preprint
Oct 2026
Video-Index: A Curated Meta-Benchmark for Video Understanding
A video benchmark should reward the capability it claims to measure, yet models can exploit answer options, question text, or partial visual evidence. We introduce the attack pyramid, five levels of shortcut attacks with increasing access to each item, and audit 115 video benchmarks with it. On 35 benchmarks, attackers...
En-Xin Song, Yi-Nuo Xu, Shu-Sheng Yang et al.
· 0 citations
2026
VideoEvent: Leveraging Relevance and LLMs for Video Question Answering
VideoEvent is a lightweight and efficient training-free framework that segments an input video into question-relevant temporal events and selectively supplements them with low-level visual cues such as background and object layout, making it highly suitable for practical deployment.
Chen-Chen Lin, Ming-Han Lee, Kun-Ru Wu et al.
· International Conference on... · 0 citations
Preprint
Aug 2026
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
This work conducts a large-scale evaluation of more than 20 recent MLLMs and shows that video instruction following remains challenging for current models, especially for instructions with many constraints, semantic constraints, or complex conditional structures that require selecting the correct branch or path based o...
Hongbo Liu, Peixian Chen, Siyuan Liu et al.
· 0 citations
TRACE: Transparent Retrieval for Abstract Concept Evaluation
This work compares six off-the-shelf VLMs against a transparent baseline that uses classical SIFT keypoint matching and a signal-quality index over retrieved images, and shows that a learned visual representation is not the bottleneck for this task: given retrieved images, a shape-appropriate classical similarity suffi...
Joe M. Bingham
· 0 citations