Skip to content
Book Open access

Streaming-o1: Reinforcing Streaming Video Understanding with Adaptive Reasoning Depth

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 68 references

Abstract

In recent years, multimodal large models have made significant progress in video understanding and reasoning. However, most of them rely on an offline setting, which assumes that both videos and questions can be fully accessed in advance. In contrast, streaming video question answering scenarios require capturing dependencies across sequential questions and adaptively controlling reasoning depth under real-time constraints. To address these limitations, we propose Streaming-o1, an adaptive reasoning framework for streaming video understanding. By explicitly modeling dependencies across questions and adaptively selecting reasoning depth, it achieves efficient and consistent real-time interaction. In the supervised fine-tuning stage, we construct a streaming video question answering dataset that incorporates multi-turn reasoning and difficulty-aware sampling, featuring chain-of-thought with multidimensional signals such as confidence. In the reinforcement learning stage, we introduce a reinforcement learning method based on Group Relative Policy Optimization (GRPO) combined with multi-dimensional reward signals such as confidence. This approach guides the model to generate multi-turn responses that are consistent, reliable, and adaptive in reasoning depth. Experimental results show that Streaming-o1 improves both general and streaming video question answering, achieving a strong balance between multi-turn consistency, complex reasoning, and efficiency.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.