A Multimodal Fusion Framework For Abnormal Posture Recognition Based on Plantar Pressure and Kinematic Sensing

Abstract

In response to the limitations of traditional posture assessment (subjective, low quantification) and single sensing modalities (vision, pressure, IMU), this paper proposes a multimodal fusion framework integrating plantar pressure and kinematic sensing. Using the MovePort public dataset and a clinical dataset, sliding window augmentation generated 1,216 labeled samples. Five statistical features were extracted from COP and IMU signals. A random forest classifier was adopted with GroupKFold (public dataset) and LOOCV (clinical dataset). Unlike previous work, this study has innovated in the following areas. Firstly, the modal level ablation experiment quantified the fusion gain: Full fusion achieved an accuracy of 85.32% on the MovePort public dataset, outperforming only COP (70.29%) and only IMU (76.83%), with a quantization gain that was+8.49 percentage points higher than the optimal single modality. Secondly, feature importance analysis revealed for the first time the relative contribution of each modality: IMU accounted for 72.6%, COP accounted for 27.4%, providing quantitative basis for sensor selection. Thirdly, strictly adopt topic independent validation (GroupKFold) to avoid overly optimistic generalization estimates commonly seen in random partitioning. On a clinical small sample dataset (N=37, three types of tasks), the framework achieved an accuracy of 56.76%, establishing a baseline for future clinical studies.