A coupled learning--environment model in which agents update stateless $Q-values on a fixed graph, while their population-average behavior drives an environmental variable that dynamically modifies the payoff matrix is formulated.
Abstract
Multi-agent reinforcement learning in networked populations is governed by the interaction between individual adaptation, local encounters, and changing environmental conditions. To study this interaction, we formulate a coupled learning--environment model in which agents update stateless $Q$-values on a fixed graph, while their population-average behavior drives an environmental variable that dynamically modifies the payoff matrix. Under a first-order mean-field closure, we derive a deterministic transport equation for the population distribution of $Q$-values and couple it with a projected discrete update for the environmental state. The resulting model is evaluated against finite-network Monte Carlo simulations on random regular, Erd\H{o}s--R\'enyi, Barab\'asi--Albert, and random geometric graphs. Across the tested parameter ranges, the mean-field system reproduces the main macroscopic cooperation and environmental trajectories, and the trajectory-level root-mean-square error generally decreases with population size and average degree. The analysis further shows that environmental feedback reshapes the learned action-value ordering, while reinforcing feedback can produce pronounced dependence on the initial learning bias and resource level. The environmental timescale also plays an important role: a rapid response can drive the resource state to a boundary before learning adapts, whereas a slower response preserves the interaction between behavioral learning and environmental recovery. These results provide a population-level description of coupled reinforcement learning and environmental dynamics and characterize the performance of the mean-field approximation within the tested network and parameter ranges.
A mean-field RL framework is introduced in which the rewards and transition dynamics depend on the population only through an unknown low-dimensional aggregate statistic, and it is shown that under a fixed neural-network parameter count and optimization budget, learning a low-dimensional population representation impro...
Aditya Makkar, B. Unger, Jeongyeol Kwon et al.· 0 citations
We introduce a general class of interacting reinforced stochastic systems evolving on a network. The dynamics are governed by two distinct interaction structures: a sampling network, which determines how information is acquired, and a reinforcement network, which determines how the sampled information influences future...
Yogesh Dahiya, G. Kaur, Neeraja Sahasrabudhe· 0 citations
The results indicate that evolutionary stability and learning accessibility are distinct properties of a coupled game--learning system in a transparent three-agent governance-motivated game involving a government, a platform firm, and users.
Recent advances in mean-field games and its multi-population variants enable large-scale heterogeneous multi-agent systems to be modeled through representative agents and their associated mean-field distributions. However, existing approaches do not explicitly account for uncertainty in the behavior of other population...
It is shown that temporal trajectories can act like a coordinated population, reducing control energy by 19–24% in demand response while still reaching the desired final distribution.
This work considers a minimal model of active agents which collectively learn to reach a target macroscopic state via a decentralized adaptation process via a decentralized adaptation process, and describes the evolution of the policy distribution across the agent population with a kinetic theory.