A Cross-Modal Consistency Constraint-Based Method for Adversarial Attack Detection and Robust Representation Learning in Embodied Perception

Abstract

Non-visual multimodal embodied perception relies on heterogeneous sensory streams such as inertial signals, physiological measurements, body-worn sensors, ambient sensors, and structured temporal data. Although these modalities provide complementary information for embodied intelligence systems, they also introduce new security risks because adversarial perturbations may corrupt one modality, multiple modalities, or local temporal segments while remaining difficult to identify from single-signal statistics. This paper studies adversarial attack detection and robust representation learning for non-visual multimodal embodied perception from the perspective of cross-modal consistency. We propose a consistencyconstrained framework that first extracts modality-specific temporal representations and then estimates their agreement in a shared latent space. To capture attack evidence more comprehensively, the framework further introduces embodied temporal-dynamical consistency and cross-modal reconstruction, which reveal abnormal temporal transitions and modality-level inconsistency. A reliability-aware fusion module uses consistency scores and reconstruction residuals to reduce the influence of suspicious modalities and generate a robust fused representation for both attack detection and downstream perception. Experiments are designed on UCI-HAR, PAMAP2, OPPORTUNITY, and MHEALTH to evaluate detection effectiveness, downstream robustness, ablation behavior, generalization, and interpretability. The proposed method provides a unified mechanism for detecting adversarially induced relational disruption while maintaining task-relevant multimodal representations.