SmartFallMM: A Multimodal Dataset for Fall Detection Using Commodity Devices

Abstract

Wearable sensor-based fall detection systems that rely solely on inertial measurement unit (IMU) data are prone to high false-positive rates, particularly for subtle ”soft falls.” While incorporating multimodal data can substantially improve detection accuracy, existing multimodal datasets typically depend on specialized, lab-grade IMU sensors and often lack demographic diversity, limiting their applicability to real-world deployment in commodity wearable devices and generalizability across age groups. To bridge this gap, we present SmartFallMM, a multimodal dataset collected using a combination of commodity and specialized hardware: smartwatches, smartphones, Kinect motion cameras, and Meta sensors. The dataset captures synchronized skeletal motion (32 joint positions), accelerometer and gyroscope readings from 51 participants spanning two distinct age groups (20–30 and 60–93), performing 14 activities encompassing both falls and activities of daily living (ADLs). Since multimodal models are too computationally demanding for edge devices, we propose modality-aware knowledge distillation (MAKD), a cross-modal approach that distills multimodal knowledge into an IMU-only model. MAKD improves F1-score by up to 7.85% on SmartFallMM and 8.65% on the UP-Fall dataset over IMU-only baselines, showing the practical value of SmartFallMM and the effectiveness of cross-modal knowledge transfer.