Counterfactual Multi-Agent Credit Assignment Optimization Fused with Spatial Causal Adjacency Matrix

Abstract

Counterfactual Multi-Agent Policy Gradients (COMA) are often hampered by dense, irrelevant noise in the centralized critic, which stems from the full-conditioning assumption in high-dimensional action spaces and results in unstable gradient estimation. To address this fundamental limitation, this paper introduces SCAM-COMA, a novel framework that reformulates credit assignment through structural causal constraints. We propose a Spatial Causal Adjacency Matrix (SCAM) that dynamically integrates spatial position priors and interaction trajectories to explicitly model directed inter-agent dependencies. Unlike traditional methods that treat agent interactions as a monolithic global problem, SCAM-COMA utilizes this matrix as a structural mask to filter out irrelevant action noise across both spatial and logical dimensions. Furthermore, we re-derive the conditional counterfactual baseline within localized causal subgraphs, effectively confining action marginalization to causally relevant agents and thereby reducing variance in the advantage function. Empirical evaluations on the StarCraft II Multi-Agent Challenge (SMAC) demonstrate that SCAM-COMA not only achieves faster convergence and superior win rates than the baseline but also exhibits significant improvements in spatial scalability and robustness for non-monotonic, time-sensitive cooperative tasks.