TraTeTrack: historical trajectory-guided temporal modeling for visual object tracking

Abstract

Currently, prevalent object-tracking methods are mostly trained using image pairs (i.e. a template image and a search image), and this training paradigm makes it difficult to capture temporal correlations in the object’s continuous motion. Meanwhile, methods that depend on consecutive video frame training incur a drastic surge in hardware resource consumption. To overcome the above issues, an object tracking network model called TraTeTrack, based on historical motion trajectory modeling and multi-feature fusion, is proposed in this paper. Specifically, the visual features of the template and search images are extracted by the vision transformer module, and the TimesNet module is utilized to obtain the cross-frame temporal correlation features from historical motion trajectories. Then, the position of the object in the next frame is predicted through the detection head after the visual features and temporal features are fused. Moreover, a multi-strategy collaborative mechanism is designed to dynamically adjust the search range and update the object template, thereby effectively addressing challenges such as object occlusion. The core innovation of TraTeTrack is to introduce historical trajectory guidance into the conventional template-search image-pair tracking framework. Instead of replacing visual image pairs, the proposed trajectory branch complements them by modeling low-dimensional historical bounding-box sequences. To evaluate the tracking performance, the proposed TraTeTrack is compared with the state-of-the-art models on multiple datasets, and ablation experiments are conducted to validate the effectiveness of each module in the proposed model. Experimental results reflect that the proposed model achieves an area under curve of 70.5% on the LaSOT dataset with a frame rate of 90 FPS, which satisfies the requirements of real-time tracking.