Beyond Index-Only RoPE: Integrating Time and Order for Generative Recommendation

Abstract

In generative recommendation, position has two distinct meanings: an event’s order in the sequence and its wall-clock time. Existing approaches usually inject temporal information through auxiliary embeddings or relative attention biases, while vanilla RoPE captures only order. We revisit RoPE for generative recommendation and ask how to encode both time and order without losing the benefits of rotary attention. We present Time-and-Order RoPE (TO-RoPE), a simple family of designs that uses both signals in rotary embeddings with three lightweight variants: early fusion, split-by-dimension, and split-by-head. The key insight is that early fusion can cause interference between time and order inside the same rotary plane, whereas split allocation provides a cleaner inductive bias. Across a public dataset and a proprietary industrial dataset, TO-RoPE consistently outperforms absolute, relative-bias, and single-source RoPE baselines.