Upper-Limb Pose Estimation From Sparse Body-Worn Visual-Inertial Sensors

Abstract

Estimating the upper-body pose is essential for a wide range of applications, from rehabilitation therapy to industrial injury prevention. Current solutions face several limitations, such as heavy setup requirements, and often suffer from occlusion and drift issues. We propose a novel, sparse sensor fusion framework that estimates upper-limb kinematics using two shoulder-mounted inertial measurement units (IMUs) and a chest-mounted camera with an integrated IMU, without visual markers. The proposed pipeline integrates an orientation estimation module with a quadratic programming inverse kinematics (QPIK) solver. This formulation enables the system to enforce biomechanical constraints and utilize low-frequency visual input to correct inertial drift and recover joint positions that are unobservable by sparse IMUs alone. We evaluate the framework on the Total Capture dataset, where egocentric targets are simulated from motion-capture keypoints masked by a virtual chest-camera field of view, allowing the fusion layer to be assessed under controlled visual rate and accuracy. In this setting, our method achieves lower tracking error than adapted EKF-based visual-inertial baselines while using fewer wearable sensors. Compared with an IMU-only data-driven method, the proposed fusion pipeline achieves lower error with sufficiently frequent and accurate visual updates, demonstrating the benefit of exploiting intermittent egocentric constraints.