Explainable Early Activity Recognition via Wearable Inertial Sensors for Human–Robot Collaboration in Agriculture

Abstract

In open-field agriculture, timely human activity recognition (HAR) is critical for anticipating worker actions and enabling proactive human–robot collaboration. This study developed an offline early HAR framework based on pre-segmented activity sequences. Long Short-Term Memory (LSTM) networks were used in conjunction with wearable inertial sensors mounted on the chest, cervical region, lumbar region, and right and left wrists. Accelerometer, gyroscope, and magnetometer signals were collected from 20 participants during an outdoor agricultural material-handling task, with a mobile ground robot serving as the receiving platform for the crate. Each activity was represented by cumulative prefixes from 30% to 100% of its duration. At the 30% observation ratio, the model achieved a macro-F1 score of 0.9314, compared with 0.9481 for the full sequence, corresponding to an absolute difference of 0.0167. Shapley Additive Explanations (SHAP) were also used to identify the body locations, sensor modalities, and signal channels that contributed most. The early decisions were mainly supported by sensors placed on the trunk, with wrist sensors providing complementary information, particularly for standing classification. Multimodal inertial information was also important. The most influential inputs were mainly gyroscope and magnetometer channels from the chest, cervical region, and lumbar region. In conclusion, the high performance at early observation ratios highlights the potential to support more adaptive and better-coordinated robot-assistance strategies.