Jul 2026· 2026 6th International Conference on Electrical, Computer and Energy Technologies (ICECET)· pp. 1-6· 0 citations· 10 references
Abstract
Despite rapid advances in artificial intelligence, robotic systems remain limited by poor generalisation across unstructured environments and fragile training pipelines. Reinforcement learning (RL) has shown promise in training robotics, yet its effectiveness is often constrained by manually engineered reward mechanisms. In parallel, large language models (LLMs) demonstrate strong reasoning and evaluation capabilities that remain underutilised in robotic training pipelines. This paper proposes a hybrid LLM-RL framework in which an LLM dynamically evaluates robot performance during simulation training and adaptively modifies the reward weights to improve learning stability, accuracy of task completion, and policy convergence. Unlike existing work that focuses on natural language control at inference time, the proposed method leverages the LLM during training, acting as a high-level reward critic. We implemented this framework using an open-source robotic arm trained in simulation to demonstrate improved task success rates and learning efficiency compared to static reward mechanisms. This work highlights a scalable pathway toward more adaptive and generalisable robotic training systems for advanced robotics.
Vision-language-action (VLA) models have demonstrated remarkable capabilities in robotic manipulation by leveraging pretrained vision-language models. However, existing post-training methods predominantly optimize VLA models as flat policies, making it difficult to explicitly model task progression and perform robust long-horizon manipulation. Although hierarchical approaches introduce task decomposition, they mainly rely on supervised learning from offline demonstrations and cannot effectively improve execution through online interaction. To address this limitation, we propose Hierarchical Robotic Control (HiRoC), a hierarchical post-training framework that decouples high-level task planning from low-level action execution. The planner decomposes complex tasks into executable subgoals to provide explicit semantic guidance, while the executor continuously improves subgoal-conditioned action generation through reinforcement learning. To enable effective collaboration between the two modules, we further align the executor with planner-generated subgoals before reinforcement learning, mitigating the distribution misalignment between planning and execution. Extensive experiments across diverse robotic manipulation benchmarks demonstrate that HiRoC consistently outperforms strong baselines. Comprehensive analyses further validate the effectiveness of hierarchical post-training and the contribution of each key component.
This paper introduces an LLM-Augmented Reinforcement Learning Agent that integrates LLM-driven planning with RL-based action optimization, and highlights a promising direction for building more capable autonomous systems.
Christophe D. Hounwanou, John Emeka Eze, Yaé Ulrich Gaba· 0 citations
Continuous robotic control requires policies that execute with low latency and modest computational cost during deployment. Foundation models provide strong semantic and visual reasoning, but repeatedly querying a large model throughout deployment incurs substantial inference latency and compute requirements. Language-to-Reward (L2R) methods avoid this deployment-time cost by using large language models (LLMs) to synthesize rewards for training lightweight policies, but these rewards are generated without visually analyzing how the learned policy physically fails, and thus often lack physical grounding. We propose Visually-Grounded Reward Synthesis (VGRS), which uses slow foundation models during training to produce fast robotic control policies. An LLM first synthesizes executable reward code from a natural-language instruction to train a lightweight hierarchical policy. When learning stalls, a frozen vision-language model (VLM) analyzes failed trajectories to provide failure mode diagnosis, which the LLM uses to rewrite and densify the reward. Since foundation models are used only during training, deployment requires only the learned policy. We perform experiments on simulated and real-world navigation and manipulation tasks, and show that VGRS achieves success rates above 55% on challenging long-horizon tasks while deploying successfully to real robots.
Utsav Singh, Pramit Bhattacharyya, Vinay P. Namboodiri· 3 citations
Actor-critic models are a class of model-free deep
reinforcement learning (RL) algorithms that have
demonstrated effectiveness across various robot learning
tasks. While considerable research has focused on improving
training stability and data sampling efficiency, most
deployment strategies have remained relatively simplistic,
typically relying on direct actor policy rollouts. In
contrast, we propose PACHS (Parallel Actor-Critic Heuristic
Search), an efficient parallel best-first search algorithm
for inference that leverages both components of the
actor-critic architecture: the actor network generates
actions, while the critic network provides cost-to-go
estimates to guide the search. Two levels of parallelism
are employed within the search---actions and cost-to-go
estimates are generated in batches by the actor and critic
networks respectively, and graph expansion is distributed
across multiple threads. We demonstrate the effectiveness
of our approach in robotic manipulation tasks, including
collision-free motion planning and contact-rich
interactions such as non-prehensile pushing. Visit
https://p-achs.github.io for demonstrations and examples.
Itamar Mishani, Hanlan Yang, Luca Pivetti et al.· Proceedings of the Internati...· 0 citations
In the quantitative finance area, particularly in order execution, reinforcement learning (RL) has shown great promise due to its ability to interact with market environments based on real data. However, traditional RL methods suffer from slow research speed and rely on static market assumptions, which do not consider the impact of the agent's execution action on the environment. To address these, we propose a Self-Evolutional single-agent/multi-agent Reinforcement Learning (SE-RL) framework. The framework utilizes a Large Language Model (LLM) to design various RL algorithm modules, such as agent model design, reward function, profiling, communication, and state imagination, by leveraging the LLM generating module output or code. SE-RL could continuously improve the accuracy of LLM-generated RL algorithms through a dual-enhancement kit at both high-level (prompt refinement) and low-level (parameter fine-tuning). Additionally, we use a multi-agent system to simulate dynamic financial markets, accounting for the impact of order executions on market dynamics. To further enhance training in such a dynamic market, we develop a hybrid environment training method that could rebalance each environment's loss weight. Comprehensive experiments on 200 realistic stock datasets demonstrate that our proposed framework outperforms current state-of-the-art baselines. Project page: https://kdd2026-se-rl.github.io/.
Vincent Fu, Xinxin Xu, Weichen Xu et al.· Proceedings of the 32nd ACM...· 0 citations
Sample effective and stable training remains a key challenge in reinforcement learning (RL), especially for real-world applications such as mobile robot control where data collection is time-consuming and failures may be hazardous.Building on the residual reinforcement learning paradigm, this work presents, to the best of our knowledge, one of the first detailed physical studies of a residual Soft Actor-Critic (SAC) controller for camera-based lane following on a mobile robot. We combine an established stable, but sub-optimal lateral P-controller with a regularized SAC agent in a hybrid architecture. The classical controller provides baseline stability and rapid initial learning, while the RL agent learns residual corrections to improve performance. We employ a PID-inspired reward function and quadratic policy output regularization to ensure smooth control actions and effective sim-to-real transfer.The hybrid controller design enables rapid training convergence, requiring only a few epochs and outperforming the pure RL approach by two orders of magnitude in sample efficiency. This enables efficient hyperparameter tuning in simulation and opens the door to future learning directly on physical robots. Fine-tuning with only a few dozen real-world laps achieved robust transfer to the physical robot, maintaining the same architecture and hyperparameters. The method generalized effectively to new scenarios, such as lane changes.
Fedi Boukhris, J. Will, Timo von Marcard et al.· International Conference on...· 0 citations