Dynamic Scene Reconstruction for Spike Camera via Bayesian Imaging and Correlation Modeling

Abstract

Spike camera is a kind of bio-inspired neuromorphic camera which is designed for capturing dynamic scenes of ultra-high speed motion at extremely high temporal resolution. It adopts an “integrate-and-fire” mechanism to convert the dynamic arrivals of photons at each sensor pixel to a stream of asynchronously fired spikes. The occurrence of spike firing may be disturbed by multiple factors, including the Poisson nature of photon arrivals and the quantization effect in spike readout. Therefore, recovering high-quality visual images from the recorded spike stream is an important yet challenging problem. This paper presents a reconstruction scheme for spike camera based on Bayesian imaging framework. We discuss the probability model of photon arrival in the imaging process and derive the likelihood model of the Bayesian framework. To fully exploit the dynamic information of spike streams, temporal correlation is used to model the image prior of reconstructed scenes. To be specific, we propose an auto-regressive model based on a non-stationary Laplacian distribution to model the temporal correlation. An efficient way to solve the optimization problem of the proposed Bayesian framework is further given. Experimental results show that the proposed method improves the reconstruction quality on both real-captured and synthesized spike datasets.