Skip to content

Interactive Distributionally Robust Multi-Agent Learning with General Function Approximation

Sep 2026 · 0 citations
Computer Science

TL;DR

This work introduces the robust Multi-Agent Decoupling Coefficient (robust MADC) to characterize the intrinsic exploration complexity arising from strategic interactions and adversarial transition uncertainty, and establishes sublinear robust regret guarantees governed by the robust MADC rather than explicitly by the state and joint action space sizes.

Abstract

Model misspecification poses a fundamental challenge in multi-agent reinforcement learning, where transition uncertainty can be amplified by strategic interactions among agents. Distributionally robust Markov games (DRMGs) provide a principled framework for addressing such uncertainty, yet existing methods often rely on restrictive assumptions or scale poorly to large state and joint action spaces. We study online learning in general-sum DRMGs with general function approximation and $\phi$-divergence uncertainty sets. We propose RoMEX-$\phi$, a model-free framework that integrates equilibrium-based exploration with dual fitted learning. Through a functional dual representation of the robust multi-agent Bellman operator, RoMEX-$\phi$ enables tractable worst-case value estimation from nominal interaction data using a centered empirical robust discrepancy. We introduce the robust Multi-Agent Decoupling Coefficient (robust MADC) to characterize the intrinsic exploration complexity arising from strategic interactions and adversarial transition uncertainty. We establish sublinear robust regret guarantees governed by the robust MADC rather than explicitly by the state and joint action space sizes, replacing tabular dependence with intrinsic function-class complexity. Numerical experiments on a scalable general-sum DRMG under total variation uncertainty show that RoMEX-$\phi$ is substantially more resilient to transition shifts than its non-robust counterpart while remaining competitive with an exact tabular robust baseline. Our results provide a scalable framework for distributionally robust multi-agent reinforcement learning with general function approximation.

View source

Similar papers

Preprint Oct 2026

Toward Online Robust Zero-Sum Markov Games with Function Approximation

Online robust zero-sum Markov games pose a coupled operator-learning and game-solving problem: from nominal interaction data, the learner must recover a worst-case minimax Bellman operator while maintaining strategic coverage and controlling stage-game approximation error. Existing robust reinforcement-learning theory...

Chen-Hao Zhou, Ji-Yu Wei · 1 citation
Preprint Oct 2026

Learning in Inverse Games: Tractable Training with Probabilistic Guarantees

Inverse game theory seeks to learn agents'unknown objectives from observed equilibrium behavior. Existing residual-based approaches can lead to non-convex problems and need not ensure strong monotonicity of the learned game, limiting reliable equilibrium prediction. We develop a tractable convex framework for learning...

Arghya Mallick, R. R. Baghbadorani, Peyman Mohajerin Esfahani et al. · 0 citations
Preprint Oct 2026

Bayesian Offline Multi-Agent Reinforcement Learning with Latent World Models

A Bayesian offline MARL framework that infers latent global states from decentralized observation-action histories and iteratively refines a posterior world model using only offline data is proposed, supporting a principled connection between world model reliability, offline data coverage, and policy performance in par...

Xi Xiong, Lu Liu, Yi-Ming Li et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Variance-Averse $n$-Step Offline Reinforcement Learning for Sparse Long-Horizon Environments

Generative actors are transforming offline reinforcement learning (RL) by enabling expressive policy classes that model complex action distributions. However, this expressiveness also exposes a key challenge in heterogeneous datasets: generative policies can reproduce unreliable action modes whose return distributions...

Gu-Heon Kang, Minhae Kwon · 0 citations
#machine learning Preprint Oct 2026

Test-time Multi-agent Coordination by Decomposed Value Gradient Flow

This work proposes scalable coordination via optimal unified transport (SCOUT), the first offline MARL framework to combine a generative foundation model with a learned value function through test-time action refinement, and proves a single-term KL bound on the joint soft-value gap that vanishes as transport converges.

Dong-Su Lee, Hao-Ran Xu, Amy Zhang · 0 citations
#artificial intelligence Preprint Oct 2026

Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents

Risk-sensitive reinforcement learning (RL) addresses decision-making under uncertainty and adverse outcomes. Learning risk-sensitive policies from fixed datasets is important when additional interaction with the environment is costly. Existing work on risk-sensitive RL focuses on online settings or specific risk measur...

Han-Zhen Wang, Boxiang Lyu, Zhong-Xiang Dai et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.