Learning From Past and Future: A Unified Instantaneous Pedestrian Intent Prediction Framework Based on Privileged Knowledge Distillation for Autonomous Driving

Abstract

Accurately predicting pedestrian crossing intent is paramount for the safety and reliability of autonomous driving systems in complex urban environments. Despite tremendous progress in the past decade, existing methods often fail to make reliable predictions from instantaneous observations. To address this limitation, we propose a novel privileged knowledge distillation (PKD) framework that can be seamlessly integrated into existing prediction models to improve their performance under instantaneous observation scenarios. Specifically, we propose a unique training paradigm where a privileged branch (teacher) learns temporal trend features from bidirectional privileged sequences, encompassing both extended historical observation sequences and future behavioral sequences in the time-to-event (TTE) period. This bidirectional trend knowledge is then distilled into the instantaneous branch (student) via a meticulously designed composite loss, enabling the student to effectively simulate and infer complex pedestrian intent from only a few frames of input at test time. The framework exploits specialized components, including a Gated Multi-scale Trend Module (GMTM) and a bidirectional trend simulation module (BTSM), to capture and mimic privileged knowledge explicitly. Extensive experiments on the public PIE and JAAD datasets demonstrate that our PKD framework significantly enhances the performance of multiple state-of-the-art baseline models, confirming its effectiveness and broad applicability. Note to Practitioners—In real-world autonomous driving, vehicles often need to predict a pedestrian’s intent to cross the road using only a few visual frames, especially in sudden or complex traffic situations. However, short and incomplete observations frequently lead to unreliable intent predictions. This study aims to improve the reliability of such instantaneous pedestrian intent prediction when limited sensory data are available. We propose a Privileged Knowledge Distillation (PKD) framework that is flexible and can be plugged into existing prediction systems without additional sensing hardware. During training, the privileged branch learns from both extended past and future sequences to form a deeper understanding of how pedestrians’ intent evolves over time. The knowledge of the privileged branch is distilled into an instantaneous branch that only relies on short observations. At runtime, the instantaneous branch can reason about pedestrians’ intent using only brief observations. In practice, this approach can enhance the safety and responsiveness of autonomous vehicles by allowing them to make early and reliable decisions even with minimal visual input. Therefore, our approach has broad application prospects in autonomous driving and automatic robotics.