High-speed quantitative laser speckle computational imaging via event camera and SNN-CNN hybrid architecture

Physics-guided neural network for single-shot real-time dynamic speckle imaging

Laser speckle imaging is essential for continuous hemodynamic monitoring, but emerging high-speed modalities such as rolling shutter speckle imaging (RSSI) are severely bottlenecked by intensive computational demands and high susceptibility to noise caused by statistical undersampling. To address these fundamental limitations, we introduce the Bifurcated Speckle Network (BiSNet), a hybrid deep learning architecture that merges empirical data training with analytical physical models to mitigate noise artifacts of undersampled speckle measurements in RSSI. To realize this hybrid training framework, BiSNet links localized architectural constraints with physics-guided optimization. By bounding the network's receptive field, the model is forced to evaluate flow dynamics based on localized speckle fluctuations. This prevents the network from memorizing the macroscopic, uniform structures of physical phantoms. Also, a physics-guided loss embeds the RSSI forward model directly into the training process, enabling the network to learn the governing physics. This approach improves data efficiency, allowing the model to generalize across diverse flow conditions without requiring an exhaustive library of experimental phantom datasets. Validated using controlled microfluidic setups and in vivo cranial window models, BiSNet delivers high-fidelity, real-time hemodynamic mapping. By effectively suppressing undersampling noise and bypassing the latency of traditional iterative solvers, our proposed framework accelerates accurate flow parameter estimation for continuous biological monitoring.

Efficient High-Frequency Event-Based Optical Flow with Temporal Iterative Refinement and Spiking Neural Networks.

Existing event-based optical flow approaches often build on frame-based counterparts, failing to deliver high-frequency flow estimation. Methods that specifically address this issue fail to achieve comparable performance or the desired computational efficiency. In this work, we introduce a novel temporal iterative refinement (TIR) framework to obtain low-latency flow updates at high frequency. The TIR module incorporates the previous flow estimate along with the updated feature maps to simultaneously update and refine the flow estimate at each time step, thereby predicting accurate nonlinear pixel trajectories. However, updating the feature space at high frequency with conventional CNNs may lead to the temporal aperture problem, as the small temporal receptive field may not be enough to capture the necessary spatial context. We introduce SNN-based feature encoders to efficiently address this problem. The temporal dynamics of the SNNs provide an increased temporal receptive field, while their deployment on neuromorphic hardware offers a promising path toward additional energy efficiency. The results obtained on the real-world MVSEC dataset show that our network achieves 17× and 33× lower computations than the state-of-the-art E-RAFT and TMA, respectively, while maintaining similar accuracy performance. Compared to other supervised learning-based approaches, our network exhibits better cross-domain generalization, hinting toward the strong inductive biases of the network. To demonstrate the remarkable potential of our approach, we also provide results in extremely challenging scenarios with highly nonlinear pixel trajectories from the MultiFlow dataset, which also features high-frequency ground truth. Our code will be available at https://github.com/AhmedHumais/STIRFlow.

Optical Flow from Photons

Optical flow remains challenging in high-speed and low-light scenes, where the limited frame rate and sensitivity of conventional cameras lead to motion blur and underexposure. Single-photon avalanche diode (SPAD) cameras offer single-photon sensitivity and extremely fine temporal sampling. However, individual slices in these high FPS binary photon streams are too sparse for dense correspondence. Temporal aggregation can provide the spatial cues required by optical flow, but accumulating photons at fixed coordinates blurs moving structures. Motion-aware aggregation can reduce this blur, yet it depends on the flow being estimated. To address this dependency, we propose QuantaFlow, the first method for dense optical flow directly from SPAD streams. Instead of constructing a fixed input representation, QuantaFlow embeds SPAD representation construction into iterative flow refinement. At each iteration, the current flow coarsely aligns the slices within the source and target sub-streams. A photon-flux transformation then constructs multi-scale representations containing intensity and structural cues, while adaptive multi-scale fusion balances photon noise and residual motion blur at each pixel. The fused representations drive a feature-warping flow update, and the refined flow guides representation construction in the next iteration. We further construct a synthetic dataset for SPAD optical-flow training and evaluation. Experiments on the synthetic dataset and real-world SPAD data demonstrate the effectiveness and generalization of QuantaFlow.

FlowRoI: fast optical-flow-based Roi extraction for high-throughput immune cell image compression

Autonomous migration is central to neutrophil function and diverse disease processes. ComplexEye, our recently introduced multi-lens array microscope, enables high-throughput live-cell video acquisition for routine quantification of autonomous motility. However, such platforms generate data at extreme scale, creating substantial challenges for storage and transmission. Here we present FlowRoI, a fast, training-free framework for region-of-interest (RoI) extraction and RoI-aware compression in immune cell migration studies. FlowRoI computes optical flow between consecutive frames and derives RoI masks that effectively capture migrating cells in the evaluated dataset. Each frame and its RoI mask are then jointly encoded using JPEG2000 to achieve efficient, cell-focused compression. FlowRoI is computationally lightweight, operating at approximately 30 frames per second on a laptop with an Intel i7-1255U CPU—comparable to standard JPEG2000. At matched peak signal-to-noise ratio, FlowRoI achieves approximately 2 × higher compression rates than standard JPEG2000 in the evaluated dataset while preserving higher image quality in cellular regions. To assess downstream impact, we evaluated cell instance segmentation as a representative task. At comparable segmentation accuracy in our experiments, FlowRoI provides approximately 2 × higher compression efficiency. FlowRoI requires only a few hyperparameters and shows stable performance across the tested settings, facilitating practical parameter selection. Together, these findings demonstrate the feasibility of FlowRoI as a computationally efficient, domain-oriented approach for task-aware compression in bright-field neutrophil migration imaging. Its applicability to other cell types, imaging modalities, and experimental settings remains to be systematically evaluated.

High-Speed Imaging with Light-Deserializing Coded-Exposure-Pixel CMOS Image Sensor

High-speed volumetric amplitude-spectrum dynamic optical coherence tomography by neural network with multi-burst scanning

Dynamic optical coherence tomography (DOCT) enables label-free, three-dimensional (3D) assessment of tissue dynamics. However, it suffers from long acquisition times because conventional time-spectrum DOCT requires hundreds of repeated OCT frames per location. Here we present a neural network (NN) framework integrated with a non-uniform-time scanning protocol (multi-burst scan) to accelerate amplitude-spectrum DOCT (AS-DOCT). Combining 3D convolutional and long-short term memory (LSTM) layers with dual inputs (the temporal OCT sequence and its pseudo-amplitude spectrum), the model generates AS-DOCT images from only 16 frames per location. Validated on 29 cancer spheroids, the proposed method resolved distinct functional domain structures with high fidelity (structural similarity index metric (SSIM)>0.8) and enabled full volumetric AS-DOCT acquisition in 26.2 seconds. This method will enable high-throughput 3D dynamic tissue screening.