We propose a conditioning mechanism for diffusion models based on multi-speed joint diffusion of the target and the condition. The mechanism learns an unconditional joint score network and enforces conditioning at inference via a plug-in correction term. The plug-in term separates the conditioning contribution from the learned unconditional dynamics, offering a transparent view of how the condition steers generation of the target distribution. Building on this, we derive explicit conditional reverse-time SDEs and approximate probability-flow ODEs, enabling principled and directly comparable conditional samplers. To reduce the induced ODE--SDE discrepancy, we introduce a log-Fokker--Planck residual regularization that improves ODE sampling quality. Experiments on conditional image generation tasks demonstrate competitive performance and support the effectiveness of the plug-in conditioning view. Additional ODE--SDE comparison experiments show that the log-Fokker--Planck residual regularization improves deterministic ODE sampling.
Libo Chen, Souvik Ghosh, Teo Deveney et al.· 0 citations
Continuous robotic control requires policies that execute with low latency and modest computational cost during deployment. Foundation models provide strong semantic and visual reasoning, but repeatedly querying a large model throughout deployment incurs substantial inference latency and compute requirements. Language-to-Reward (L2R) methods avoid this deployment-time cost by using large language models (LLMs) to synthesize rewards for training lightweight policies, but these rewards are generated without visually analyzing how the learned policy physically fails, and thus often lack physical grounding. We propose Visually-Grounded Reward Synthesis (VGRS), which uses slow foundation models during training to produce fast robotic control policies. An LLM first synthesizes executable reward code from a natural-language instruction to train a lightweight hierarchical policy. When learning stalls, a frozen vision-language model (VLM) analyzes failed trajectories to provide failure mode diagnosis, which the LLM uses to rewrite and densify the reward. Since foundation models are used only during training, deployment requires only the learned policy. We perform experiments on simulated and real-world navigation and manipulation tasks, and show that VGRS achieves success rates above 55% on challenging long-horizon tasks while deploying successfully to real robots.
Utsav Singh, Pramit Bhattacharyya, Vinay P. Namboodiri· 3 citations