MSDR-Mamba: A Multi-Scale Branch-Decoupled Routing State-Space Detector for Temporal Action Localization

Abstract

Temporal action localization (TAL) requires a detector to recognize action categories and estimate temporal boundaries in untrimmed videos. Mamba supports linear-complexity long-sequence modeling, yet a uniform allocation of state-space operators does not explicitly differentiate the context requirements associated with temporal scales and prediction branches. We present Multi-Scale Decoupled Routing Mamba (MSDR-Mamba), a multi-scale branch-decoupled routing state-space detector. The method combines a phase-dilated multi-rate Mamba temporal pyramid, multi-band state-time initialization, level-wise local–global gating guided by duration priors, and a branch role-decoupled head. The final head uses CNNs for classification and center-offset estimation, with Mamba used for class-specific start/end boundary neighbor modeling. With frozen InternVideo2-6B features on THUMOS14, MSDR-Mamba achieves a five-threshold mAP of 73.09%, exceeding TriDet by 0.43 percentage points. Supplementary experiments on ActivityNet-1.3 and P2ANet further evaluate the complete configuration under longer-duration and dense short action distributions. The results support scale- and branch-aware state-space modeling as a practical design strategy for TAL.