Prompt learning with joint spatial-frequency features for RGB-T tracking

Abstract

In recent years, prompt learning-based RGB-T tracking methods have attracted growing attention due to their ability to transfer knowledge from pre-trained vision models to tracking tasks, thereby significantly improving tracking accuracy. However, most of these methods predominantly focus on spatial-domain feature extraction, often overlooking the potential benefits offered by frequency-domain representations. To address this issue, we propose a novel prompt-based RGB-T tracking framework named the Spatial-Frequency Prompt Network (SFPNet), which is specifically designed to jointly model high-quality features from both spatial and frequency domains. In particular, we introduce a Fast Fourier Transform (FFT)-based feature extraction component, referred to as Phase-Guide-Amplitude (PGA), to extract essential frequency characteristics from RGB and thermal infrared (TIR) images. Additionally, a Joint Spatial-Frequency (JSF) module is developed to effectively fuse spatial and frequency information, leading to the generation of more discriminative and robust tracking features. Comprehensive experiments on two widely adopted benchmark datasets—LasHeR and RGBT234—demonstrate that the proposed approach outperforms several existing state-of-the-art methods in terms of tracking performance.