Video Person Re-Identification via RGB-D Triple Pyramid Model with Transferable Feature Fusion

Abstract

Existing video person re-identification (ReID) methods still face three major challenges: limited spatial perception from RGB-only inputs, equal-weight local feature extraction under occlusion, and ineffective fusion of heterogeneous motion and appearance features. To address these issues, we propose an RGB-D Triple Pyramid Model (TPM) for robust spatiotemporal feature learning. The proposed framework can utilize conventional RGB surveillance videos without requiring additional depth sensors. Specifically, TPM first generates pseudo RGB-D representations from RGB frames through monocular depth estimation. This process introduces additional spatial cues, such as pedestrian scale and relative distance, to improve spatial perception. To capture temporal dynamics, frame differences from consecutive 5-frame clips are extracted and aggregated into compact motion representations, which effectively describe short-term pedestrian movements. For appearance representation, each pedestrian frame is divided into dense uniform grids to extract fine-grained local features. The proposed method automatically identifies visible and informative regions and assigns adaptive weights to different grid cells. These weighted local features are aggregated to construct an occlusion-robust appearance representation, which emphasizes discriminative regions while reducing the influence of unreliable areas. Furthermore, a Transferable Feature Fusion (TFF) module is introduced to integrate motion and appearance representations. Instead of directly combining heterogeneous features, TFF transfers discriminative information from motion features to appearance features by exploiting the regression residuals learned from the motion branch. This process refines appearance representations and improves the complementary interaction between motion and appearance features. Extensive experiments on three widely used video ReID benchmarks, including MARS, iLIDS-VID, and PRID-2011, demonstrate the effectiveness of the proposed method. Specifically, TPM achieves improvements of 4.7%, 4.8%, and 1.7% in Rank-1, Rank-5, and Rank-20 accuracy, respectively, on the MARS dataset, validating the effectiveness of the proposed framework.