Skip to content
Open access

Comparative Analysis of Proximal Policy Optimization and Soft Actor-Critic Algorithms in a Procedurally Generated Dynamic Environment

Sep 2026 · Symposium on Intelligent Systems and Informatics · pp. 779-784 · 0 citations · 16 references

TL;DR

The development and evaluation of an autonomous agent based on reinforcement learning (RL) that learns to play a 2D platformer game is presented and potential future extensions (such as procedural level generation and combined learning methods) are discussed.

Abstract

This paper presents the development and evaluation of an autonomous agent based on reinforcement learning (RL) that learns to play a 2D platformer game. Designed in Unity and trained using the Unity ML-Agents Toolkit, the environment integrates a variety of procedural obstacles and traps to provide realistic challenges and measure learning adaptability. Two RL algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC), were trained using identical conditions for fair comparison. Quantitative results show that SAC converged faster and achieved higher average rewards early on, whereas PPO learned more slowly but with a steadier improvement in performance. These findings are evaluated statistically, and potential future extensions (such as procedural level generation and combined learning methods) are discussed.

Read PDF

Similar papers

#reinforcement learning Open access Oct 2026

Optimizing and evaluating proximal policy optimization and soft actor-critic agents in Unity ML-Agents-based games

This study presents a comparative analysis of proximal policy optimization (PPO) and soft actor-critic (SAC) for training autonomous delivery agents in high-fidelity 3D environments using Unity ML-Agents. Both algorithms were evaluated with identical hyperparameters and reward functions across five independent runs to...

Muhammad Fakhri Loebis, Andry Chowanda · 0 citations
Conference Open access Sep 2026

Reinforcement Learning: A Comprehensive Survey of Foundational Algorithms, Recent Advances, and Future Directions

Reinforcement Learning (RL) has emerged to be one of the most radical ideas in artificial intelligence since it enables the learner to learn the desired behavior through a response to the environment. The survey provides the overall picture of reinforcement learning, its development history beginning with the classical...

N. Ramshankar, D.Elangovan, S. S et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Adaptive Expert Guidance for Efficient On-Policy Reinforcement Learning

With massively parallel simulation, on-policy Reinforcement Learning methods such as PPO have become standard in many domains. However, learning from scratch is sample-inefficient and fails to exploit the potential existence of a suboptimal expert, such as a heuristic, a model-based controller, or a policy trained on a...

D. Affinita, Ming-Jing Xu, Rudolf Reiter et al. · 0 citations

Bellman Policy Optimization

Bellman Policy Optimization (BPO), a critic-free method derived from Policy Mirror Descent, is introduced and it is proved that it has the same unique optimal solution as the original PMD objective.

Zhuo-Qing Song, Hao-Tian Xu, Xi-Kun Zhang et al. · 0 citations
Open access Sep 2026

A Comparative Analysis of Reinforcement Learning Training Strategies for Agents on Competitive 2v2 Soccer

The results indicate that higher competitive performance does not necessarily lead to better player experience, and models trained against deterministic opponents with difficulty scaling were rated as more natural and more enjoyable.

M. P. Rezende, R. Torchelsen · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.