How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those...
Holger Molin, William Peng, M. Bagatella et al.· 0 citations
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltoni...
Modeling human reactions to spatial audio requires understanding both what is heard and where it comes from. Prior work generates motion offline from complete acoustic and spatial input sequences. Humans, however, react to sounds without knowing what they will hear next. We present Sonic-AMG, to our knowledge the first...
Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint t...
Ming-Hao Fu, Tavis Siebert, Eryk Halicki et al.· 0 citations
DSReg (Dependency-Sparsity Regularization) recovers individual world latents up to signed permutation, without reconstruction or a decoder, without reconstruction or a decoder, and establishes the first fully identifiable JEPA that recovers every world latent.
Yujia Zheng, David A. Klindt, Randall Balestriero et al.· 0 citations
Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan at a single timescale or with multiple horizons in one shared latent space. We introduce H-JEPA, an end-to-end recipe for training a hierarchy of actio...
Wan-Cong Zhang, Basile Terver, Michael G. Rabbat et al.· 0 citations
SemiGroup-JEPA (SG-JEPA), which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model via action-conditioning and jointly training an encoder and predictor through an autoregressive latent rollout, is introduced.
LeVJEPA is introduced, the first video encoder trained under LeJEPA's collapse-free objective, which indicates that video becomes a viable and in several respects preferable substrate for general-purpose visual pretraining.
Lukas Kuhn, Lucas Maes, Giuseppe Serra et al.· 3 citations· ⚡1
This work introduces LpWorldModel, a JEPA model regularized with Rectified Distribution Matching Regularization to match encoder features to a Rectified Generalized Gaussian distribution, yielding non-negative sparse codes, and finds that the learned sparse representations are mode-factored.
Yilun Kuang, Yash Dagade, Quentin Le Lidec et al.· 5 citations
The Veracity-Influence-Sobriety score (VIScore), a metric that quantifies the reachability and capacity of a predictor given the encoded feature, and the hallucination of the searching-based planner, is proposed, showcasing the importance of these three aspects in planning success.
Hai-Yu Wu, Randall Balestriero, Morgan E. Levine· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.