Perception Delay-Aware Predictive Visual Servoing for Quadruped Robot Person Following

Abstract

Deploying pure-vision person-following systems on Size-, Weight-, and Power- (SWaP) constrained quadruped robots is challenging because onboard visual perception is generally computationally demanding. In practice, the perception pipeline often runs asynchronously at only 5–10 Hz, while its measurements are further corrupted by apparent jitter caused by human walking gait and robot trotting. The resulting delayed and noisy feedback can substantially degrade the stability and tracking performance of conventional closed-loop controllers. To address this issue, this paper presents a fully onboard, multi-rate, delay-aware predictive visual servoing framework. A discrete-time Kalman filter is introduced to smooth visual measurements and bridge asynchronous perception updates, thereby providing state estimates for high-frequency control. Based on these estimation and prediction, an image-based visual servoing model predictive controller generates smooth and deterministic 50 Hz locomotion commands while explicitly enforcing field-of-view and safety-distance constraints. The proposed method is validated through extensive high-fidelity Gazebo simulations and real-world deployments on a Unitree Go2 robot equipped with an NVIDIA Jetson Orin NX and Intel RealSense D435i camera. In highly dynamic maneuvers, ablation studies show that the proposed delay compensation reduces the peak lateral tracking error by 57.0% and the overall root mean square error by 32.8% compared with an uncompensated baseline, enabling reliable person following without auxiliary non-visual sensors.