Exploiting Model Replication and Heterogeneous Accelerator Parallelism for Low-Latency Multi-View Perception

Abstract

Multi-view perception is a fundamental requirement for autonomous driving systems. While state-of-the-art approaches often rely on bird’s-eye-view networks executed on high-performance monolithic accelerators, deploying such power-hungry hardware is economically and thermally infeasible for resource-constrained edge platforms. Consequently, cost-effective edge platforms are increasingly adopting distributed, heterogeneous multi-accelerator architectures. However, orchestrating multi-view workloads across these platforms presents significant challenges due to severe performance heterogeneity and incompatibility across disparate instruction set architectures and proprietary toolchains. To address this, we propose a view-independent inference architecture that leverages model replication and view-level parallelism, enabling independent frames to be processed concurrently across disparate accelerators. To dynamically distribute these independent frames across performance-asymmetric accelerators, we introduce a lightweight, capacity-aware scheduling policy. By employing an exponential moving average filter to capture dynamically changing backend computing capacities, our scheduler adapts to macroscopic hardware state transitions without relying on offline profiling. We evaluate our approach on a fully pipelined 3D tracking application, driven by the 6-camera nuScenes dataset and deployed on edge platforms featuring disparate computing units (GPUs, DSPs, and NPUs). Empirical results confirm that our framework effectively mitigates straggler bottlenecks, reducing maximum processing latency by up to 80.7% and improving end-to-end throughput by up to $5.05\times $ .