Edge-FPGA Deployment of DeepLabCut for Real-Time Human Pose Estimation: A Distributed INT8/FP32 Hybrid Inference Architecture
Due to the high cost of commercial motion capture systems, their widespread utilization in research and teaching is limited. To the best of our knowledge, this paper presents the first successful deployment of the DeepLabCut ResNet-101 backbone to human pose estimation on the AMD Kria KV260 using a Xilinx DPUCZDX8G Deep Processing Unit (DPU). Since the transposed convolution prediction head of DeepLabCut is incompatible with the instruction set available on the DPU, the model is divided into two parts, with the first part running on the DPU with quantization to INT8 and executed at the edge of the network, with the transposed convolution prediction head running on a host PC in FP32, creating a hybrid inference pipeline via Gigabit Ethernet. The developed system operates at 13.1 FPS on a $250 KV260 board, with a peak joint-detection confidence of 0.92 on visible keypoints, which is significantly less expensive than compared embedded GPU platforms for this operation.