A. Bose

Learned Relay Representations for Forward-Thinking Discrete Diffusion Models

Relay introduces a differentiable per-token channel that passes information between forward passes and is trained via truncated backpropagation through time (BPTT), demonstrating that state-of-the-art DLMs can be explicitly trained to relay latent information forward across decoding steps, advancing the performance-latency Pareto frontier.