Quantitative and qualitative evaluations on multi-shot story generation demonstrate that the Complementary Retrieval-Augmented Prompting method consistently outperforms recent-frame, memory-based, and entity-level retrieval baselines in cross-shot consistency and text-controllability.
Abstract
While recent video foundation models excel at generating high-quality short videos, long-form video generation remains a critical challenge, where a major bottleneck lies in conditioning independently generated shots to preserve consistent characters, scenes, and objects throughout a story. Existing training-free approaches typically condition target shots using retrieved historical visuals. However, these references often suffer from severe informational mismatch, either introducing irrelevant contextual redundancy or failing to provide the full combination of required elements for the target shot. To resolve this, we present Complementary Retrieval-Augmented Prompting, an agentic framework that strategically aggregates a compact set of mutually supportive historical references to achieve complete and targeted conditioning for long-form video generation without retraining or modifying the underlying generator. Specifically, our framework explicitly models the visual elements required by each target shot by parsing the narrative script into a text-grounded visual element registry that tracks characters, objects, scenes, and their shot-level states. A VLM-annotated keyframe library further maps these elements to past visual observations. Guided by the required elements, our agent retrieves complementary references that maximize target-element coverage while minimizing historical noise. Finally, the retrieved references, structured element states, and grounding instructions are assembled into a unified prompt for the frozen video generator. This element-aware process provides comprehensive conditioning while remaining fully interpretable. Quantitative and qualitative evaluations on multi-shot story generation demonstrate that our method consistently outperforms recent-frame, memory-based, and entity-level retrieval baselines in cross-shot consistency and text-controllability.
Video Question Answering (Video-QA) requires answers grounded in visual events, temporal context, and spoken or written evidence. Conventional large language models can generate plausible but unsupported responses when long videos are compressed into incomplete prompts. This paper presents a lightweight Video-RAG frame...
K. Akash, Pasunuri Abhinav, Poosarla Akshara Kruti et al.· International Conference on...· 0 citations
Generative video systems can produce short clips from textual and visual instructions, yet their ability to preserve the content of a human reference remains uncertain. This exploratory study examines 160 videos generated with Doubao-Seedance-2.0 from 40 human-made short-form videos drawn from Douyin and Xiaohongshu. T...
Chloe Zihan Jin· Communications in Humanities...· 0 citations
Creating a compelling social media video from raw clips requires decisions at multiple levels, from story structure and pacing to clip selection and local editing effects. Existing language-based editing systems typically respond to explicit edit commands, retrieve clips, or generate videos directly, but they do not pr...
Saumya Gupta, Ishita Dasgupta, Stefano Petrangeli et al.· Proceedings of the 34th ACM...· 0 citations
Novels tell rich stories, but their textual form cannot directly present the sights and sounds of the worlds they describe. Film adaptations offer an audiovisual experience but require substantial production effort and viewing time; film recaps are shorter but depend on existing footage. Neither route directly provides...
Kyrie You Wu, Chao-Zhuo Li, Song-Yang Liu et al.· 0 citations
MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based video generation, is presented and development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline.
Recent advances in autoregressive video generation have improved temporal consistency over extended durations, yet interactive storytelling requires more than continuous scene extension: a new shot may combine characters and backgrounds from different historical shots. Whole prompt retrieval can overlook the distinct r...
Zi-Yi Wang, Jun-Chi Yao, Heqian Qiu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.