Skip to content
Preprint

CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying

Sep 2026 · 0 citations · 38 references
Computer Science

TL;DR

CAM stores the entities and relations extracted from video clips in a knowledge graph that captures the high-level semantics of each entity or relation and summarizes the local subgraph of the target entity or relation once the subgraph reaches a predefined size.

Abstract

Memory facilitates question answering over long videos by extracting and retrieving facts to fit within the limited context windows of multimodal LLMs (MLLMs). Existing solutions typically extract independent memory entries from fixed-length video clips and thus cannot capture high-level semantics that need to be summarized over extended time periods, such as character traits and relations. Moreover, they rely solely on similarity-based retrieval and may fail to retrieve the fine-grained details required for question answering. To tackle these problems, we propose CAM, featuring continuous extraction for high-level semantics and adaptive querying for fine-grained details. In particular, CAM stores the entities and relations extracted from video clips in a knowledge graph. To capture the high-level semantics of each entity or relation, CAM summarizes the local subgraph of the target entity or relation once the subgraph reaches a predefined size. To retrieve the fine-grained details required for question answering, CAM supports multiple search methods, including knowledge graph traversal, video re-watching, and audio listening. It utilizes a planner-executor-verifier pipeline to adaptively compose these search methods according to question intent. Evaluations on three benchmarks show that CAM outperforms SOTA baselines and improves their accuracy by up to 23 percentage points. Code is available at https://github.com/Jake-Tian/CAM.

View source

Similar papers

Open access 2026

VideoEvent: Leveraging Relevance and LLMs for Video Question Answering

VideoEvent is a lightweight and efficient training-free framework that segments an input video into question-relevant temporal events and selectively supplements them with low-level visual cues such as background and object layout, making it highly suitable for practical deployment.

Chen-Chen Lin, Ming-Han Lee, Kun-Ru Wu et al. · 0 citations
Preprint Oct 2026

Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering

Long egocentric videos pose a central challenge for multimodal language models: the visual stream must be compressed into a memory representation that can be searched efficiently while still preserving the evidence needed for question answering. Building on an online textual-memory framework in which first-person video...

Raffaele Cali, Giuseppe Lando, Rosario Forte et al. · 0 citations
Preprint Oct 2026

EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding

Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most o...

Yu-Hao Qin, Jun-Bo Wang, Yu-Ke Li et al. · 0 citations
#natural language process... Preprint Sep 2026

Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering

This work proposes MemLoc, a unified Retrieve-Localize-Generate framework for long-term conversational memory QA, and introduces a reasoning-based evidence locator trained with Self-reflective Hint Policy Optimization, which performs progressive refinement by extracting query-relevant fragments within memory units to s...

Yi-Fan Wang, Xin-Kui Lin, Yong-Xiu Xu et al. · 0 citations
#machine learning Preprint Oct 2026

Towards a Learnable Retriever for Textual Graph Question Answering

Textual graph question answering (GraphQA) and graph-based retrieval-augmented generation (GraphRAG) have gained increasing attention as a way to ground large language models (LLMs) on structured knowledge. Despite extensive efforts, existing methods still face two key gaps. The first is an architectural gap: existing...

Kai-Feng Jin, Yu-Chen Yan, Zhichen Zeng et al. · 0 citations
Open access Sep 2026

PKSF: A Task-Aware Prior Knowledge Selection and Fusion Framework for TextVQA

This work proposes a TextVQA framework that integrates external prior knowledge to support multimodal reasoning and demonstrates that this approach effectively leverages external knowledge to improve performance on knowledge-intensive questions.

Zan-Xia Jin, Pin-Le Qin, Jia Qin et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.