Skip to content

An Evolutionary Computation Framework for Multi-Agent Q-Learning with Mean-Field Environmental Feedback

Sep 2026 · 0 citations · 55 references
Computer Science

TL;DR

A coupled learning--environment model in which agents update stateless $Q-values on a fixed graph, while their population-average behavior drives an environmental variable that dynamically modifies the payoff matrix is formulated.

Abstract

Multi-agent reinforcement learning in networked populations is governed by the interaction between individual adaptation, local encounters, and changing environmental conditions. To study this interaction, we formulate a coupled learning--environment model in which agents update stateless $Q$-values on a fixed graph, while their population-average behavior drives an environmental variable that dynamically modifies the payoff matrix. Under a first-order mean-field closure, we derive a deterministic transport equation for the population distribution of $Q$-values and couple it with a projected discrete update for the environmental state. The resulting model is evaluated against finite-network Monte Carlo simulations on random regular, Erd\H{o}s--R\'enyi, Barab\'asi--Albert, and random geometric graphs. Across the tested parameter ranges, the mean-field system reproduces the main macroscopic cooperation and environmental trajectories, and the trajectory-level root-mean-square error generally decreases with population size and average degree. The analysis further shows that environmental feedback reshapes the learned action-value ordering, while reinforcing feedback can produce pronounced dependence on the initial learning bias and resource level. The environmental timescale also plays an important role: a rapid response can drive the resource state to a boundary before learning adapts, whereas a slower response preserves the interaction between behavioral learning and environmental recovery. These results provide a population-level description of coupled reinforcement learning and environmental dynamics and characterize the performance of the mean-field approximation within the tested network and parameter ranges.

View source

Similar papers

#machine learning Preprint Aug 2026

Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representations

A mean-field RL framework is introduced in which the rewards and transition dynamics depend on the population only through an unknown low-dimensional aggregate statistic, and it is shown that under a fixed neural-network parameter count and optimization budget, learning a low-dimensional population representation impro...

Aditya Makkar, B. Unger, Jeongyeol Kwon et al. · 0 citations
Preprint Oct 2026

Interacting Stochastic Systems with Agent-Based Sampling and Connection-Based Reinforcement

We introduce a general class of interacting reinforced stochastic systems evolving on a network. The dynamics are governed by two distinct interaction structures: a sampling network, which determines how information is acquired, and a reinforcement network, which determines how the sampled information influences future...

Yogesh Dahiya, G. Kaur, Neeraja Sahasrabudhe · 0 citations

Evolutionary Stability Does Not Guarantee Learning Accessibility: A Multi-Agent Reinforcement Learning Perspective on Cooperation Emergence

The results indicate that evolutionary stability and learning accessibility are distinct properties of a coupled game--learning system in a transparent three-agent governance-motivated game involving a government, a platform firm, and users.

Yi-Jie Wang · 0 citations
#machine learning Preprint Oct 2026

Beyond Nominal Equilibria: Risk-Averse Multi-Population Mean-Field Games

Recent advances in mean-field games and its multi-population variants enable large-scale heterogeneous multi-agent systems to be modeled through representative agents and their associated mean-field distributions. However, existing approaches do not explicitly account for uncertainty in the behavior of other population...

Bhavini Jeloka, Siddhartha Ganguly, Panagiotis Tsiotras · 0 citations
Preprint Sep 2026

Collective adaptation to time-dependent targets for active Brownian particles

This work considers a minimal model of active agents which collectively learn to reach a target macroscopic state via a decentralized adaptation process via a decentralized adaptation process, and describes the evolution of the policy distribution across the agent population with a kinetic theory.

Gerhard Jung, Eric Bertin · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.