Deltoris is presented, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference that exploits the temporal similarity of consecutive inputs and proposes a temporal-aware bit-sparsity algorithm that computes only the differences between consecutive inputs, eliminating redundant bit-level operations.
Abstract
Vision-language-action (VLA) models have emerged as a key component in embodied AI. Among existing approaches, diffusion-based VLA models achieve superior motion quality and generalization. However, diffusion-based VLA models are compute-intensive and must run at high control frequency, e.g., 50-200 Hz. Thus, it imposes strict latency and energy constraints on edge devices. In this work, we present Deltoris, an algorithm-hardware co-design framework for efficient diffusion-based VLA inference. First, we exploit the temporal similarity of consecutive inputs and propose a \textit{temporal-aware bit-sparsity} algorithm that computes only the differences between consecutive inputs, eliminating redundant bit-level operations. To further address the extra off-chip traffic introduced by our algorithm, we propose a \textit{speculative inference} technique, which amortizes data loading across multiple control steps. Lastly, to support these techniques, we co-design a dedicated accelerator with customized 1D systolic bit-serial PE arrays that eliminate PE workload imbalance. Our evaluation shows that Deltoris achieves up to 34.2$\times$ speedup over mobile GPUs and 6.1$\times$ over prior accelerators, while maintaining comparable accuracy.
Combining two-step denoising with representative execution methods substantially reduces inference cost with a small reduction in task performance, motivating joint optimization of model-inference efficiency and robot-system timing.
Di Wu, Rong-Tian Shen, Ping Liu et al.· 0 citations
Diffusion-based Vision-Language-Action (VLA) models achieve strong performance in embodied tasks, but their iterative sampling imposes heavy computational and memory-access cost, blocking real-time deployment on edge platforms. Existing acceleration methods either require expensive training (e.g., distillation, flow ma...
Yu-Cheng Han, Jianhan Wu, Xiaoyang Qu et al.· 0 citations
FlashVLA is introduced, a streaming action decoding framework that addresses both challenges in a unified formulation and can achieve $\geq$30\,Hz control frequency on a single GPU with smooth asynchronous inference in real-world deployment.
This work describes four representative VLA models on an edge GPU server and two onboard SoCs, using single-inference profiling and 43,200 closed-loop episodes, and guides joint design of VLA model architectures, hardware, and runtime policies.
Seonghun Jung, Sieun Moon, Jiyoung Jeong et al.· 0 citations
Recurrent Action Memory (RAM) is proposed, a lightweight module that conditions action prediction on previous action tokens, providing temporal context critical for manoeuvres such as overtaking and emergency braking.
Kemal Oksuz, Alexandru Buburuzan, Yu-Han Yao et al.· arXiv.org· 0 citations
ComVLA is proposed, a framework that uses the dense semantic information contained in the language guidance to adapt the VLA token budget to the channel capacity, and demonstrates that co-designing VLA inference and wireless communication is a practical direction for 6G-connected robotics.
Bo-Liang Liu, Wint Yi Poe, Jing-Yun Di et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.