Motion editing is central to VFX and game development, where it is used extensively to modify and augment existing movements to conform to new environments or changes in artistic direction. While traditional motion editing can do small modifications, it cannot accommodate larger structural edits, resulting in visual warping artifacts that require authoring new motion. Conversely, recent advances in large-scale generative modeling have unlocked newfound capabilities for authoring entire movements by directly manipulating sparse spatial constraints. While impressive at creating new movements, these methods lack the capability to preserve and edit existing motion interactively. In this work, we introduce scheduled inpainting, a method that enables interactive generative motion editing, a novel paradigm unifying motion synthesis and editing by leveraging generative models. Scheduled inpainting is a simple yet powerful inference-based technique that enables fine-grained spatiotemporal control over the balance between preserving the original motion and generating new content. By building atop generative models that support direct manipulation, our system allows artists to interactively refine existing animations while ensuring results remain natural and consistent with the learned motion distribution. Scheduled inpainting is versatile and supports many editing applications, such as extending, stitching, and compositing different clips. Finally, we extensively validate our approach by comparing with four baselines, conducting ablations of our design, and reporting user feedback.
Dhruv Agrawal, D. Borer, Luca Vögeli et al.· 0 citations
Realistic animal motion for virtual production is typically obtained either through motion capture of highly trained performers who accurately mimic animal behavior, or by retargeting ordinary human motion using complex control setups. Both approaches are challenging and often fail to fully reproduce the nuances of natural animal motion, motivating data-driven alternatives. We present an automatic human-to-quadruped puppeteering framework that produces plausible and controllable quadruped motions from ordinary human motion data. Our approach employs a two-stage generative diffusion model trained purely on quadruped motion data. By introducing a structured conditioning and inpainting strategy, our method supports a wide range of actions, including walking, running, jumping, sitting, and lying. Furthermore, we enable fine-grained intuitive control of the quadruped motion such as head movement control and individual limb puppeteering. Experimental results demonstrate improved motion realism and controllability compared to existing retargeting approaches, highlighting the effectiveness of our framework as a tool for animation and virtual production applications.
Fatemeh Zargarbashi, Zehong Qiu, Dhruv Agrawal et al.· 0 citations
The recent success of generative modeling has led to entirely new capabilities to author 3D motion. By manipulating only a few sparse handles and poses, it is now possible to generate entire motion sequences, at scale and with many variations. To bridge the gap between model research and real-world integration, we developed a Generative Motion Rig as a Blender plugin, built atop a general motion model. Our rig supports a new “generative keyframing” workflow, where artists author movements by manipulating sparse poses, handles, window lengths, and noise sampling. We show in our accompanying video how artists use these controls to rapidly make a short animation. We also show how the same rig can support generative motion editing, allowing one to edit and extend mocap clips. Finally, we share insights and future challenges to help close the gap between generative and traditional animation workflows.
Jakob Buhmann, Dhruv Agrawal, D. Borer et al.· Proceedings of the Special I...· 0 citations