This work proposes Self-SiMS, a self-similarity-based Moment Proposal and Scoring that exploits intrinsic relationships within videos, enabling robust span generation and scoring and introduces a query-aware MLLM-based reasoning stage to further sharpen alignment between text and video.
Abstract
Zero-shot video moment retrieval aims to overcome the limitations of traditional approaches that require large-scale datasets annotated with text and its relevant temporal spans. Despite advances in pre-trained vision-language models and multimodal large language models, existing ZMR methods still heavily depend on query-to-video content similarity, making them vulnerable to modality and language-style gaps. These gaps lead to unreliable span proposals and unstable moment retrieval results. To address this issue, we propose Self-Similarity-based Moment Proposal and Scoring that instead exploits intrinsic relationships within videos, enabling robust span generation and scoring. By deriving self-similarity only from the video content, we circumvent the noisy and mismatched patterns of query-frame or query-caption similarities, thereby mitigating both modality and language-style gaps. Furthermore, we introduce a query-aware MLLM-based reasoning stage to further sharpen alignment between text and video. Extensive experiments demonstrate that Self-SiMS achieves state-of-the-art performance across ZMR benchmarks.
This work proposes Video-GAR, a novel framework that reframes the conventional retrieval task from superficial matching to generative understanding, positing that the capability for query reconstruction evidences deep semantic comprehension.
Mingjin Kuai, Qianyin Xiao, Juncheng Li et al.· Annual International ACM SIG...· 0 citations
A new prototype-based hierarchical alignment network (PHA-Net) to align individual/local/global level representations across modalities and introduces multiple modality-shared prototypes as the bridge to efficiently optimize text and video representations for cross-modal alignment.
Xiaolun Jing, Kezhao Yin, Xinxing Yang et al.· Neurocomputing· 0 citations
Video Corpus Moment Retrieval (VCMR) is pivotal to multimodal understanding. However, existing methods rely heavily on large-scale annotated data, which limits their generalization and scalability. To address this issue, we propose a training-free VCMR framework, termed Synergistic Collaboration and Adaptive Calibration (SCAC), enabling effective semantic parsing and precise temporal localization without parameter updates. SCAC introduces a Query Event Chain Generation module that leverages large language models to transform complex textual queries into structured event chains, while a Video Event Chain Generation module represents videos as semantically coherent event chains through subtitle segmentation and keyframe aggregation. Built on these structured representations, SCAC performs Event-Chain-Based Cross-Modal Retrieval with mean-variance joint scoring to suppress local mismatches and reinforce global consistency. During localization, a Synergy-Calibration Mechanism dynamically refines temporal boundaries via profit-setback feedback. Extensive experiments show that SCAC achieves comparable or superior results to supervised counterparts under training-free conditions, demonstrating strong cross-modal generalization and adaptive capability. The code of our method is available at https://github.com/cyanlll/SCAC
Jialong Zhao, Huafeng Li, Yafei Zhang et al.· IEEE Transactions on Image P...· 0 citations
Video-text temporal localization requires precise alignment between natural language queries and corresponding video segments, a fundamental challenge in multimodal understanding. We present a novel framework that addresses two critical limitations of existing methods: inadequate modeling of hierarchical temporal structure and inability to handle complex many-to-many correspondences between modalities. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities - from instantaneous frame transitions to extended action sequences. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations through structured agreement updates, enabling flexible modeling of non-monotonic alignments. These components are unified through a multi-task learning objective that jointly optimizes temporal boundary regression, cross-modal semantic alignment, and capsule diversity. Extensive experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU, surpassing strong transformer-based baselines while maintaining computational efficiency. Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained video-language understanding.
Gengtian Shi, Jinze Yu, Chenhao Wu et al.· 0 citations
With the rapid growth of real-world untrimmed videos, video moment localization (VML) has become a fundamental task in query-guided video understanding, aiming to identify the temporal moment that semantically matches a natural language query. Although fully supervised methods have achieved promising performance, their reliance on precise start–end annotations makes them difficult to scale to open-domain video scenarios, where visual contents are diverse, distracting contexts are common, and annotation resources are limited. Point-supervised VML provides a more data-efficient alternative by requiring only a single annotated point inside the target moment. However, such sparse supervision makes it challenging to infer the complete query-relevant interval and to suppress semantically similar distractors. To address these challenges, we propose an anchor-guided discriminative semantic expansion (ADSE) framework. ADSE treats the annotated point as a reliable semantic anchor, learns anchor-centered cross-modal alignment to generate a temporal relevance curve, and adaptively expands the anchor into a coherent target moment by integrating query relevance and temporal semantic continuity. Meanwhile, an anchor-guided discriminative learning strategy mines high-confidence anchor-excluded intervals as hard negatives, and an inside–outside separation objective further distinguishes target moments from surrounding contexts. Extensive experiments on public benchmarks demonstrate the effectiveness of ADSE and show consistent improvements over existing point-supervised methods under sparse point-level supervision.
Zhaoliang Zhou, Longqiang Pang, Zhen Li et al.· Electronics· 0 citations