Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltoni...
Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan at a single timescale or with multiple horizons in one shared latent space. We introduce H-JEPA, an end-to-end recipe for training a hierarchy of actio...
Wan-Cong Zhang, Basile Terver, Michael G. Rabbat et al.· 0 citations
Autonomous driving requires \textit{world models} that can understand the physical world, reason and plan, and operate safely. In this paper, we first systematically evaluate existing action-conditioned joint-embedding predictive architecture (JEPA) world models, including LeWM, DINO-WM, and JEPA-WM for end-to-end auto...
Hao-Ran Zhu, Wan-Cong Zhang, Yann LeCun et al.· 0 citations
This work believes Patch Policy provides a pipeline for the robotics community to readily leverage continuing progress in visual representation learning, without sacrificing the training efficiency or inference speed required for high-frequency, reactive control.
Gaoyue Zhou, Zichen Jeff Cui, Ada Langford et al.· arXiv.org· 0 citations
This work characterize models from this family of methods as JEPA-WMs and investigates the technical choices that make algorithms from this class work, and proposes a model that outperforms two established baselines, DINO-WM and V-JEPA-2-AC, in both navigation and manipulation tasks.
Basile Terver, Tsung-Yen Yang, Jean Ponce et al.· Trans. Mach. Learn. Res.· 49 citations· ⚡7
LeVJEPA is introduced, the first video encoder trained under LeJEPA's collapse-free objective, which indicates that video becomes a viable and in several respects preferable substrate for general-purpose visual pretraining.
Lukas Kuhn, Lucas Maes, Giuseppe Serra et al.· 3 citations· ⚡1
This work introduces LpWorldModel, a JEPA model regularized with Rectified Distribution Matching Regularization to match encoder features to a Rectified Generalized Gaussian distribution, yielding non-negative sparse codes, and finds that the learned sparse representations are mode-factored.
Yilun Kuang, Yash Dagade, Quentin Le Lidec et al.· 5 citations
This work proposes to learn a world model of piano sound using JEPA by framing music as an action-conditioned system, and shows that the learned model captures the relationships between musical actions and their resulting sound.
Ziyu Wang, Kun Fang, Yann LeCun· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.