Learning Event Perturbation Field for Continuous Event Stream-Based Object Tracking

Abstract

Event-based cameras offer promising potential for object tracking due to their high temporal resolution and low latency. However, the sparse and asynchronous nature of event streams makes it nontrivial to construct effective representations for event-only tracking. Existing methods predominantly rely on handcrafted event encodings, which are agnostic to downstream tracking objectives and often fail to fully exploit the temporal structure of events. In this work, we show that variations in tracking objectives fundamentally affect how event information should be represented, motivating the need for a task-adaptive event representation. We propose a novel Event Perturbation Field (EPF), a learnable and stackable event representation that enables end-to-end optimization of event features for object tracking. EPF models raw events as a learnable event vector field, capturing temporal perturbations in a flexible and objective-aware manner. To demonstrate the effectiveness of EPF for continuous event stream-based object tracking, we develop a transformer-based tracking framework that directly bridges raw event streams and target state prediction. Comprehensive experiments validate the advantages of EPF, and our tracker achieves state-of-the-art performance on multiple challenging event-based benchmarks.