Digging Into Monocular Texture-Scale Feature Fusion for Optical Flow and Optical Expansion Estimation

Abstract

Visual perception in dynamic scenes is a crucial task in the fields of computer vision and autonomous driving. In this paper, we focus on monocular optical flow and expansion estimation. Optical flow describes the positional changes of pixels between two frames, while optical expansion describes local image-scale changes around pixels and their neighbourhoods. Monocular optical flow estimation has a well-established research foundation, with most existing methods achieving high-precision optical flow through image texture feature extraction and matching. However, research on optical expansion is limited. Existing optical expansion estimation methods all rely on optical flow results or model structures and do not accurately model the expansion task itself. In fact, object motion in dynamic scenes generates rich local scale information in images, which is an ideal characteristic for solving the optical expansion task. In this paper, we propose and explore methods for extracting and interacting image texture-scale features and apply them to achieve high-quality monocular optical flow-expansion estimation. We propose a novel pipeline, MonoFusion, which independently extracts texture and scale features, improving the performance of optical flow and expansion through feature fusion and updating. Additionally, we introduce MonoFusion-R, which further enriches the iteration process of texture-scale feature fusion with a specially designed recurrent framework. Our method achieves state-of-the-art (SOTA) monocular performance on four KITTI tasks, including optical flow, optical expansion, time-to-collision, and scene flow, and also shows strong optical-flow generalization on FlyingThings3D.