Automated Dense Reward Function Design for Reinforcement Learning in Autonomous Driving

Abstract

Crafting reward functions remains a challenge in reinforcement learning, particularly in autonomous driving (AD), where safety, efficiency, and task completion must be balanced. We introduce a GPT-5.6-based revision of AD Reward Designer, a framework for generating interpretable dense reward functions from natural language goals and concise Pythonic environment descriptions. The initial reward generation requires no expert demonstrations or model fine-tuning. GPT-5.6 translates task requirements into executable reward code, which is checked before being used by a reinforcement learning algorithm. Policy rollout summaries and human feedback then guide iterative refinement of the reward terms and their weights. An optional demonstration-conditioned extension supplies expert and policy trajectories as additional context for reward design. We retain Highway-env and MetaDrive as the evaluation environments and PPO as the policy learning algorithm. The original experiments provide evidence for the feasibility of language-model-generated rewards; their numerical results are retained as historical evidence and are not reported as GPT-5.6 measurements. The revised GPT-5.6 configuration requires a new evaluation under the same task settings.