Yuxuan Jiang

Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation

This work introduces TD-V2A, which leverages temporal differences (TD) as the key representation that distinguishes V2A from I2A, enriching visual conditioning with minimal architectural modification and significantly improves end-to-end V2A generation quality, even outperforming dedicated V2A representations such as contrastive audio-visual pretraining.