Graph Topology-Guided Multi-Task Learning for Fall Detection
TL;DR
The proposed MTC-DETR framework outperforms 14 state-of-the-art methods, proving its robustness and potential for real-world fall detection applications, and is deployed on the Leju Kuavo 5 Robot.
Abstract
Fall detection faces challenges of visual occlusions and scale variations in complex multi-person scenarios. To address these issues, this paper proposes MTC-DETR, an end-to-end multi-task collaborative detection framework. After a feature extraction backbone, MTC-DETR builds a Dynamic Scale-Routing and Task-Tuning Encoder to adapt to scale variations, which integrates Deformable Large-Kernel Attention for intra-scale long-range feature enhancement and Dynamic Cross-Scale Fusion and Task-Aware Dispatch for adaptive multi-scale aggregation and task-specific feature calibration. Then, a Topology-Guided Dual-Branch Decoder is designed for object detection and keypoint detection. The Topology-Guided Local–Global Synergistic Attention reconstructs occluded keypoints via multi-hop graph convolutions, and the cross-branch pose prior guides Deformable Cross-Attention sampling in the object detection branch to enhance the localization robustness. Finally, a Homoscedastic Uncertainty Dynamic Joint Loss is introduced to resolve gradient conflicts during multi-task optimization. Experimental results show that MTC-DETR achieves mAP@0.5 scores of 92.7%, 88.9%, and 90.4% on the CAUCAFall, DiverseFall10500, and MT-Fall datasets, respectively. The proposed framework outperforms 14 state-of-the-art methods, proving its robustness and potential for real-world fall detection applications. The model was deployed on the Leju Kuavo 5 Robot. Experimental results from the deployment demonstrate that the model achieves an inference speed of 44.2 FPS (INT8) on an edge computing platform, meeting the real-time requirements for fall detection.