A Bayesian offline MARL framework that infers latent global states from decentralized observation-action histories and iteratively refines a posterior world model using only offline data is proposed, supporting a principled connection between world model reliability, offline data coverage, and policy performance in partially observable offline MARL.
Abstract
Model-based offline multi-agent reinforcement learning (MARL) provides a promising approach to multi-agent decision-making under partial observability, where the global state is unavailable from offline data. However, limited dataset coverage can lead to substantial out-of-distribution (OOD) model errors, which may mislead policy optimization and degrade downstream performance. Existing model-based offline MARL methods typically rely on access to global states or mitigate model errors through pessimistic or conservative policy optimization, leaving the reliability and refinement of learned world models less explored. We propose a Bayesian offline MARL framework that infers latent global states from decentralized observation-action histories and iteratively refines a posterior world model using only offline data. Building on posterior information loss, we extend this criterion to the multi-agent setting as multi-agent posterior information loss (MPIL), which provides a reliability measure for world model refinement and offline evaluation. Theoretically, we establish an upper bound on policy regret that is monotonic in MPIL and further characterize its dependence on the size and state-joint-action coverage of the offline dataset. Experiments on partially observable multi-agent variants of D4RL MuJoCo tasks demonstrate competitive policy performance across diverse offline data conditions, while MPIL-guided refinement improves world model prediction and downstream policy performance. These results support a principled connection between world model reliability, offline data coverage, and policy performance in partially observable offline MARL.
ICMAPE converts the fixed-confidence identification objective into a reward derived from inference confidence, so that standard reinforcement learning machinery can be applied to decentralized pure exploration.
This work proposes Multi-Agent Flow-Pretrained Policy Optimization (MA-FPPO), which uses online fine-tuning to improve the cooperative behavior of models pretrained with flow matching through new interactions with the environment.
Guo-Wei Zou, Hao-Nan Chen, Hai-Tao Wang et al.· arXiv.org· 0 citations
This paper proposes a model-based bootstrap framework for uncertainty quantification of OPE in finite-horizon, time-inhomogeneous Markov decision processes (MDPs), and establishes bootstrap distributional consistency, asymptotically valid confidence intervals, and consistent variance estimation for the target policy va...
World models improve sample efficiency by training policies on imagined trajectories, but their usefulness depends on learning representations that capture the information needed for future control. We study whether self-supervised joint-embedding prediction (JEPA) can provide this learning signal for multi-agent reinf...
Brandon Kaplowitz, Osaze James Obahor, C. S. D. Witt· arXiv.org· 0 citations
Learning new tasks from limited data inherently introduces uncertainty in meta-learning. While Bayesian meta-learning aims to enhance traditional algorithms by quantifying uncertainty in model parameters, existing methods are centralized and cannot handle decentralized data. In this paper, we present the Distributed Va...
Xian-Yang Zhang, Wei-Jie Kang, Ying Li et al.· Neural Networks· 0 citations
G2MAF applies one globally normalized, projected critic gradient to guide and coordinate all agents'corrections while keeping the action both feasible and close to the frozen policy proposal.
Guo-Wei Zou, Hai-Tao Wang, Guo-Xin Wang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.