A Systematic Analysis of In-Domain and Out-of-Domain Strategies for Self-Supervised Learning in HAR
Self-Supervised Learning (SSL) has emerged as a promising approach in Human Activity Recognition (HAR) by mitigating the need for large volumes of labeled data. However, the impact of pre-training data composition still requires in-depth investigation. This work analyzes data composition strategies in SSL, comparing the inclusion of the target dataset (IN) versus the exclusive use of external data (OUT). We evaluated four pretext tasks (Contrastive Predictive Coding (CPC), Learning from Randomness (LFR), Time-Frequency Consistency (TFC), and Temporal Neighborhood Coding (TNC)) on five public datasets (KU-HAR, HAPT, MotionSense, RealWorld, and UCI-HAR), considering two fine-tuning strategies and multiple sample budgets, totaling 26,040 experiments. The results indicate that the advantage of the IN strategy is not consistent within the evaluated context, depending on the technique, its backbone architecture, and the target dataset: while the IN strategy presents consistent improvements on CPC, methods such as TNC seem to be indifferent to the presence of target data. Regarding the OUT strategy, it was observed that the simple increase in data volume does not guarantee a proportional performance gain in the downstream task. Factors such as source variety and domain similarity (e.g., HAPT and UCI) showed to be more critical than the quantity of samples. Additionally, we validated that SSL outperforms supervised learning in data-scarce scenarios (few-shot), even without the target dataset in pre-training. Finally, our results indicate that in the absence of target data, one should prioritize source variety, ensuring that aiming to provide synergy between the data composition and the feature representation explored by the pretext task.