Jul 2026· IEEE Transactions on Pattern Analysis and Machine Intelligence· Vol PP· 0 citations
Medicine
TL;DR
Flow2Act is presented, a unified framework that integrates agglomerative perception with a deterministic one-step generative policy and devise a curriculum region-aware mechanism via a Spatial-Grounded State Space Duality architecture, demonstrating significant gains in policy performance, robustness to environmental perturbations, and cross-task real-world applicability.
Abstract
Developing generalizable robotic policies that balance inference efficiency, manipulation accuracy, and robustness remains a formidable challenge. Existing Vision-Language-Action models demand prohibitive data scales, while keyframe-based approaches struggle to reconcile the expressivity of generative models with the latency of iterative sampling. To address this trilemma, we present Flow2Act, a unified framework that integrates agglomerative perception with a deterministic one-step generative policy. Unlike prior methods relying on separate semantic encoders or iterative diffusion processes, our approach introduces three key innovations. First, we employ an agglomerative multi-teacher visual backbone that distills complementary strengths from diverse foundation models, capturing semantics, spatial structure, and segmentation to yield robust representations without task-specific pretraining. Second, we propose a conditional MeanFlow policy that parameterizes the interval-averaged velocity field. This formulation enables genuine single-step action generation, eliminating the discretization errors and computational overhead inherent in ODE-based flow matching. Third, we devise a curriculum region-aware mechanism via a Spatial-Grounded State Space Duality architecture, which progressively shifts attention from global flow stability to fine-grained contact precision. We evaluate Flow2Act on challenging simulation benchmarks and real-world robotic tasks, demonstrating significant gains in policy performance, robustness to environmental perturbations, and cross-task real-world applicability. Videos, code, and more details are available at project page.
This work proposes a generative learning-from-demonstration framework that enables compositional modeling of robotic behavior by connecting perception and motion through shared object-level representations, and renders scenes from object-centric neural representations that integrate canonical neural fields with latent-conditioned deformations.
A. Tekden, Yasemin Bekiroglu· IEEE Robotics and Automation...· 0 citations
The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.
Qihang Zhang, Lin Li, Luyao Zhang et al.· 10 citations· ⚡3
Cortex is introduced, a bidirectionally aligned embodied agent framework with a customized planning interface that conveys executable and tractable subtask plans from high-level VLM to low-level VLA and enables zero-shot completion of unseen real-world long-horizon tasks.
Jiaqi Peng, Xiqian Yu, Delin Feng et al.· 1 citation
By relying on independent couplings from uninformative Gaussian priors, standard diffusion and flow matching models are forced to learn complex, high-cost vector fields to reach the physical action space. Generative models excel at capturing multimodal behaviors for robotic Learning from Demonstration (LfD), but often suffer from high inference cost. This paper introduces Temporal Policy, a generative framework based on stochastic interpolants that formulates action generation as a temporally coupled transport problem. By initializing the generative flow at the robot's recent history, we explicitly couple past states to future action sequences. This data-dependent coupling reduces transport cost and produces straight vector fields. We validate Temporal Policy across visuomotor simulation benchmarks and on a physical Barrett WAM 2x 7DoF teleoperation platform. Our approach reduces transport costs by nearly an order of magnitude compared to noise-initialized baselines, achieving a 19.1 ms inference latency on a single NVIDIA RTX 4080. Crucially, these geometric and computational efficiencies are achieved while matching the success rates of state-of-the-art baselines. This simplified transport geometry bypasses the computational bottleneck of independent Gaussian priors, helping enable high-frequency, closed-loop control. The code is publicly available at https://github.com/dmiller12/TemporalPolicy.
Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment. We identify two primary sources of temporal redundancy in existing VLA pipelines: repeated visual encoding of highly similar consecutive frames and multi step iterative sampling in diffusion based policies. To address this, we propose a system level acceleration strategy that reduces computation in both perception and action generation. On the perception side, we incrementally update only tokens corresponding to dynamic scene regions instead of re-encoding entire frames. On the policy side, we compress diffusion sampling into a compact 2-step schedule through efficiency oriented training while preserving action precision. Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance, achieving up to 98% success rate on general manipulation benchmarks. Our codes will be released on Github.
Yuzhou Wu, Yuxin Zheng, Muchun Niu et al.· 0 citations