TL;DR
This work investigates whether Cosmos Policy can serve as a teacher to train IL and RL policies on specialized tasks, which would remove all human intervention from the loop and simplify robotics deployment.
Desc descriptive evidence is provided that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain, and both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks.
Despite rapid advances in artificial intelligence, robotic systems remain limited by poor generalisation across unstructured environments and fragile training pipelines. Reinforcement learning (RL) has shown promise in training robotics, yet its effectiveness is often constrained by manually engineered reward mechanisms. In parallel, large language models (LLMs) demonstrate strong reasoning and evaluation capabilities that remain underutilised in robotic training pipelines. This paper proposes a hybrid LLM-RL framework in which an LLM dynamically evaluates robot performance during simulation training and adaptively modifies the reward weights to improve learning stability, accuracy of task completion, and policy convergence. Unlike existing work that focuses on natural language control at inference time, the proposed method leverages the LLM during training, acting as a high-level reward critic. We implemented this framework using an open-source robotic arm trained in simulation to demonstrate improved task success rates and learning efficiency compared to static reward mechanisms. This work highlights a scalable pathway toward more adaptive and generalisable robotic training systems for advanced robotics.
This work proposes QWM, a framework that leverages world models to perform test-time search over imagined trajectories on top of Q-learning to select high-value actions during both online rollouts and evaluation, and significantly outperforms strong prior state-of-the-art methods on both sample efficiency and performance.
Continuous robotic control requires policies that execute with low latency and modest computational cost during deployment. Foundation models provide strong semantic and visual reasoning, but repeatedly querying a large model throughout deployment incurs substantial inference latency and compute requirements. Language-to-Reward (L2R) methods avoid this deployment-time cost by using large language models (LLMs) to synthesize rewards for training lightweight policies, but these rewards are generated without visually analyzing how the learned policy physically fails, and thus often lack physical grounding. We propose Visually-Grounded Reward Synthesis (VGRS), which uses slow foundation models during training to produce fast robotic control policies. An LLM first synthesizes executable reward code from a natural-language instruction to train a lightweight hierarchical policy. When learning stalls, a frozen vision-language model (VLM) analyzes failed trajectories to provide failure mode diagnosis, which the LLM uses to rewrite and densify the reward. Since foundation models are used only during training, deployment requires only the learned policy. We perform experiments on simulated and real-world navigation and manipulation tasks, and show that VGRS achieves success rates above 55% on challenging long-horizon tasks while deploying successfully to real robots.
A unified framework that combines centralized training with decentralized execution (CTDE) and a Hybrid Reward Architecture (HRA) is introduced that enables multiple actors to share a centralized multi-head critic and substantially improves both sample efficiency and policy performance.
Overall, the findings indicate that VLA offers practical advantages for changing tasks by reducing programming effort and enabling useful performance with minimal data, while DQN provides a viable baseline when sufficient tuning effort is acceptable.