TextZGCA: Text-Guided Zero-Gated Cross-Attention for 3D Human Motion Prediction
Abstract
3D human motion prediction (HMP) forecasts future joint trajectories from a short observed pose sequence, but pose alone is often a narrow description of human behavior. A person who is turning, sitting down, or preparing to reach can have similar short-term pose dynamics while implying different futures. Modern vision-language models can turn simple stick-figure renders into natural-language descriptions, making semantic context available without changing the motion input itself. The challenge is to use this context without overwhelming a strong pose-only predictor with heavy language-conditioning machinery. We propose TextZGCA, a text-guided zero-gated cross-attention recipe for 3D HMP. Starting from a DCT-based spatial-temporal Transformer backbone, we encode each VLM-generated caption with a frozen plain CLIP-L text tower, project it into a small set of conditioning tokens, and insert a zero-gated cross-attention adapter after every motion encoder block. Because each adapter gate is initialized to zero, the model is initially identical to the no-text baseline and can learn to use text only where it improves prediction. On Human3.6M, TextZGCA achieves the best short-horizon and overall MPJPE among representative state-of-the-art methods in our comparison, while avoiding FiLM branches, auxiliary heads, and retrieval banks. The same recipe also achieves competitive results on CMU-Mocap, supporting a simple view of language in HMP: captions are useful as a semantic prior attached to a strong motion model.