Hybrid Behavior Tree and Reinforcement Learning for Adaptive Combat Agents in Real-Time 3D Game

Abstract

The design of fully autonomous agents that are capable of adaptive and realistic combat behavior in a 3D environment is still a big challenge in developing game AI. Reinforcement Learning agents pose issues like erratic behaviour outside of their training distribution. At the same time, rule-based agents are easy to understand and can be exploited by the human players of the game. This paper presents a hybrid method that utilizes both structures and combines a Behavior Tree model for higher level decision making with a Proximal Policy Optimization (PPO) policy, which is implemented using Unity ML-Agents. The setup is trained and tested in a Unity environment that features sword based and hand to hand combat, with mechanisms such as blocking, parrying and stamina management. The paper also compares three configurations, Behaviour Tree only, ML only, and the hybrid architecture that is described. The hybrid agent achieves higher performance statistics across various evaluation metrics like win rate, parry accuracy.