Skip to content

Deep Learning for Real-Time Sound Order Recognition in Human-Robot Interaction

Aug 2026 · arXiv.org · Vol abs/2608.04072 · 0 citations · 16 references
Computer Science

TL;DR

It is demonstrated that deep learning can reliably recognize sound order in overlapping conditions, supporting practical HRI scenarios and providing an extended discussion on generalization, ecological validity, and deployment challenges in real-room environments.

Abstract

Recognizing the temporal order of overlapping sounds is an underexplored challenge in human-robot interaction (HRI), with direct relevance to applications such as first responder detection systems. This paper presents a deep learning framework for real-time sound order recognition using recordable buzzers that emit distinct non-verbal sounds (cat meows, dog barks, helicopter noises). A multi-branch convolutional neural network (CNN) processes Mel spectrograms, Mel-frequency cepstral coefficients (MFCCs), and short-time Fourier transform (STFT) features, with an attention-based fusion mechanism to emphasize critical temporal cues. Experiments were conducted under same-amplitude, varied-amplitude, and unseen sound conditions. The proposed system achieved 99% accuracy in balanced overlaps, 91% under amplitude variation, and 74% on unseen test data with normalization. These results demonstrate that deep learning can reliably recognize sound order in overlapping conditions, supporting practical HRI scenarios. While experiments were conducted on carefully controlled synthetic overlaps, we additionally report latency benchmarks demonstrating real-time feasibility and provide an extended discussion on generalization, ecological validity, and deployment challenges in real-room environments.

View source

Similar papers

NAVIR: Neuromorphic Audio-Visual Speech Recognition for Robust Human-Robot Interaction on Edge Hardware

Voice-controlled interaction in industrial settings is hampered by acoustic noise, which severely degrades audio-only speech recognition. Audio-visual speech recognition (AVSR) addresses this by fusing lip-motion cues with the audio stream, but state-of-the-art pipelines rely on three-dimensional convolutions, recurren...

Leonidas Delimpasis, Panagiota Moraiti, Antonis Porichis et al. · 0 citations
Open access 2026

Time-Frequency Feature Fusion with Multi-Scale Attention for Underwater Acoustic Target Recognition

Traditional convolutional neural networks (CNNs) for underwater acoustic target recognition (UATR) are limited by restricted local receptive fields, and reliance on frequency-domain inputs alone can discard temporal information present in the raw waveform. This study proposes TFMSUATR, a model that fuses a raw-waveform...

Gang Hu, Yi-Fei Song, Mohamad Farhan Mohamad Mohsin · 0 citations
Conference Open access Oct 2026

Audio Signal Classification using MFCC-based Deep Neural Network

It has long been recognized that the accurate classification of audio signals is a major challenge in the field of speech and signal processing research. Applications range from speaker recognition, speech emotion detection, and environmental sound categorization. This is especially challenging due to the dynamic natur...

Jayapadmini Kanchan, K. S. Pradyuthi, C. V. Swaraj et al. · 0 citations
Conference Oct 2026

A learning method for multitask recognition of piano playing dynamics and articulation

This paper presents a multitask learning framework for the simultaneous recognition of piano playing dynamics and articulation from audio recordings. The proposed system integrates advanced audio feature extraction methods, including mel-frequency cepstral coefficients and constant-Q transforms, with precise onset and...

Si-Yuan He, Si-Xian He, Xin Gao · 0 citations
Preprint Oct 2026

Unheard but Recognizable: Ultrasonic Signatures for Musical Instrument Recognition

Musical-instrument recognition is a fundamental music information retrieval (MIR) task supporting transcription, source separation, and audio understanding. Despite significant algorithmic progress, there is still room for improvement in recognizing acoustically similar instruments, particularly in polyphonic mixtures....

Izhak Kapash, Uri B. Rom, Ram Zamir · 0 citations
Open access 2026

Performance Evaluation of DenseNet-121 and Bi-LSTM with Attention Mechanism based Vocal Signal Separation

This paper presents a supervised attention-based deep learning approach for separating single-channel audio signals that establishes a low complexity, scalable, and interpretable solution to realistic problems in source separation.

Arfana Chowdhary, Parveen Kumar Lehana · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.