Lightweight Neuromorphic Perception: Porting Low-Latency Privacy-Responsive Human Motion Analysis to Constrained Edge Architectures
Neuromorphic vision sensors offer significant advantages for real-time embedded perception due to their microsecond latency, high temporal resolution, wide dynamic range, and low power consumption. However, deploying multi-stage event perception pipelines onto edge hardware remains fundamentally constrained by memory, throughput, and execution bottlenecks on resource-limited silicon. In this work, we introduce an end-to-end, real-time perception pipeline deployed on a Raspberry Pi 5 platform, establishing a hardware-aware reference baseline for future Neural Processing Unit (NPU) and neuromorphic architectures. Our system integrates a custom attention-enhanced YOLOv8-nano model for joint person and face detection, a parameter-free ByteTrack multi-object tracking framework, and a lightweight kinematic feature extraction engine for behavioral triage and velocity-based activity classification. To evaluate localized spatial trade-offs—particularly critical for in-cabin automotive application domains such as Driver and Occupant Monitoring Systems (DMS/OMS) we systematically quantify the sensor bias configurations and optical geometries (76° vs. 104° Fields of View) across detection accuracy, temporal consistency, face region extraction, and kinematic activity mapping. The detection backbones are fine-tuned on a custom indoor event dataset and exported using static ONNX execution graphs optimized for edge deployment. Operating strictly on sparse spatiotemporal event representations without intermediate RGB frame reconstruction, our pipeline maintains robust, privacy-responsive performance under extreme lighting and motion dynamics, sustaining 19–24 FPS end-to-end throughput. By providing complete quantitative and qualitative benchmarks, this framework establishes a reproducible baseline for low-latency edge-AI deployment in automotive cabin monitoring, autonomous robotics, and assistive healthcare. The project details, fine-tuned models, empirical results, and curated event datasets are publicly available at http://mali-farooq.github.io/NeuroVision