Skip to content
Open access

A Comparative Analysis of Reinforcement Learning Training Strategies for Agents on Competitive 2v2 Soccer

Sep 2026 · Brazilian Symposium on Games and Digital Entertainment · 0 citations · 13 references

TL;DR

The results indicate that higher competitive performance does not necessarily lead to better player experience, and models trained against deterministic opponents with difficulty scaling were rated as more natural and more enjoyable.

Abstract

Introduction: The use of Reinforcement Learning (RL) in digital games has grown significantly, enabling agents to learn complex behaviors through interaction, with many approaches emphasizing improvements in agent performance. However, in game development, aspects such as perceived difficulty, behavioral naturalness, and player enjoyment are equally important. Objective: This study aimed to analyze the impact of different RL training strategies on both objective performance and subjective player experience in a competitive game scenario. Methodology or Steps: A 2v2 soccer environment was developed in Unity using the ML-Agents toolkit. The original scene was modified to include deterministic agents with scalable difficulty. Three models were trained using the multi-agent Posthumous Credit Assignment (POCA) algorithm: one with self-play and two against deterministic opponents with difficulty adjustment based on win streak and win rate. For evaluation, pairs of human players participated in sessions of three matches, each against a different model, followed by a questionnaire assessing perceived difficulty, movement naturalness, and enjoyment. Objective gameplay metrics, including match results and movement heatmaps, were also collected. Results: The self-play model was consistently perceived as the most difficult. In contrast, models trained against deterministic opponents with difficulty scaling were rated as more natural and more enjoyable. These results indicate that higher competitive performance does not necessarily lead to better player experience. Limitations include the small sample size and evaluation in a single scenario. Future work involves testing additional RL algorithms and extending the environment to more complex or varied game contexts.

Read PDF

Similar papers

Conference Open access Sep 2026

Starting Position Bias in Multiplayer Games Through Reinforcement Learning

Ensuring fairness in multiplayer games is one of the most important elements. To achieve fairness, the game needs to be well-balanced, and this has become a serious challenge for game developers in recent times. In this study, a multi agent PPO based testing method was developed to automatically detect and reduce posit...

Beyza Akbulut, Ayşe Tosun · 0 citations
#artificial intelligence Preprint Oct 2026

Reinforcement Learning for the Game of Balatro: A Fast Simulator and Two-Policy Training Approach

Balatro, a 2024 roguelike deckbuilder, presents three difficulties that have historically derailed reinforcement learning methods: a 3,000-action state-dependent action space, horizons of several hundred decisions with sparse rewards, and pervasive stochasticity in both card draws and shop generation. Published RL resu...

Ajay Vikram Periasami, Anjaneya Gupta, Ethan Cheng et al. · 0 citations

GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games

GameBoyWorlds is introduced, a testbed for agentic self-improvement in video games and it is shown that contemporary approaches to self-improvement are lacking, with world modelling and autonomous skill discovery failing, and a novel strategy that uses curiosity-based exploration to write guides achieving only partial...

Dhananjay Ashok, Adam Shen, Aslan H. Feng et al. · 1 citation
Conference Open access Sep 2026

Reinforcement Learning: A Comprehensive Survey of Foundational Algorithms, Recent Advances, and Future Directions

Reinforcement Learning (RL) has emerged to be one of the most radical ideas in artificial intelligence since it enables the learner to learn the desired behavior through a response to the environment. The survey provides the overall picture of reinforcement learning, its development history beginning with the classical...

N. Ramshankar, D.Elangovan, S. S et al. · 0 citations
Open access Sep 2026

Comparative Analysis of Proximal Policy Optimization and Soft Actor-Critic Algorithms in a Procedurally Generated Dynamic Environment

The development and evaluation of an autonomous agent based on reinforcement learning (RL) that learns to play a 2D platformer game is presented and potential future extensions (such as procedural level generation and combined learning methods) are discussed.

Máté Szabolcs Józsa, Balázs Gáspár · 0 citations
#machine learning Preprint Oct 2026

Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football

Valuing player actions in football, a complex multi-agent system, requires accounting for strategic interactions among 22 players, including off-ball movements and defensive positioning. Existing reinforcement-learning-based valuation methods commonly incorporate other players through the observed state while retaining...

Kenjiro Ide, Taiga Someya, Kohei Kawaguchi et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.