S4oP: Operator-level Pruning of Structured State Space Models for Resource-Constrained Devices

Abstract

Structured State Space Models (SSMs), including the S4 and S4D architectures, have recently emerged as powerful alternatives to attention-based models for capturing long-range dependencies in sequential data. Despite their strong empirical performance, deploying these models in time- and resource-constrained settings remains challenging due to their computational and memory demands. In this paper, we propose a novel incremental, operator-level pruning approach for S4- and S4D-based models that significantly reduces inference cost while preserving predictive performance.To the best of our knowledge, this is the first work to systematically investigate deployment-oriented operator-level pruning of parallel SSM channels in S4 and S4D architectures, explicitly targeting the trade-off between predictive performance and real inference latency on resource-constrained devices. Our method progressively prunes model operators by interleaving structured masking with fine-tuning, while jointly monitoring accuracy and inference latency. We implement this approach within a unified training and evaluation framework that enables systematic exploration of efficiency-accuracy trade-offs.Experiments across multiple datasets show that pruning up to 30% of the model operators generally preserves the predictive performance of the original models, with only marginal accuracy degradation. In some cases, the pruning ratio can be increased up to 70% while still maintaining comparable performance in predictive quality, while substantially reducing inference latency. These results demonstrate that structured operator pruning is an effective and previously unexplored strategy for improving the efficiency of SSMs and facilitate their deployment in practical, resource-constrained scenarios.