A striking finding is reported: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection.
Abstract
As autonomous agents powered by foundation models are increasingly integrated into social and economic systems, understanding the principles governing their collective behavior is essential for ensuring safety and cooperation. Classical game theory, the dominant framework for modeling rational interaction, is built upon the assumption of `decoupled agency,'where agents treat their own decision-making as independent of the environment and other actors. Modern AI agents, however, jointly predict their own future actions alongside external observations. Here, we report a striking finding: when interacting in stylized social dilemmas, foundation model agents engaging in optimal planning consistently converge to stable cooperation, directly contradicting classical game-theoretic predictions of mutual defection. To understand this phenomenon, we introduce the `embedded Bayesian agent,'a theoretical model for foundation model agents. By shifting from decoupled to embedded agency, these agents model themselves as part of the universe they inhabit, maintaining epistemic uncertainty about their own decision-making algorithms. We show that by inferring whether others are behaviorally similar, an embedded agent treats its own deliberation during planning as evidence: a decision to cooperate predicts a similar decision by a similar partner. We formalize this mechanism of similarity inference through the `embedded equilibrium,'a novel solution concept replacing the Nash equilibrium to provide a foundational game theory for the social behavior of modern AI agents.
The main findings demonstrate that individuals update strategies primarily through self-adjustment based on historical payoffs, with imitation playing merely an auxiliary role and the optimal self-adjustment proportion is approximately 0.9, and low sensitivity coefficients and mutation rates favor the emergence of coop...
Hao-Chen Wu, Meng-Cheng Sun, Lu-He Yang et al.· Games· 0 citations
SAGE is a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate, and distills strategically related interactions into counterfactual hypotheses about previously missing considerations, allowing past experience to recalibrate th...
The Hybrid Coevolutionary Opinion Game (H-COG) is introduced, in which analytical and LLM-driven agents share one network, choose their neighbors by opinion similarity in every round, and hold stances drawn from real Reddit comments on gun control and abortion.
Ming Jiang, An-Tzu Teng, Jun-En Liu et al.· arXiv.org· 1 citation
This work develops a framework for reverse-engineering why a given set of agents engage in a particular set of cooperation-related behaviors and argues that many different mechanisms can often produce similar observed behaviors and introduces a set of experimental designs along with conditions under which they can succ...
This work extends the mechanistic study of ICL to strategic multi-agent settings, introduces REE as a diagnostic tool for distinguishing reasoning from extrapolation, and provides a reusable framework for probing the boundaries of LLM reasoning in recursive belief tasks.
Y. Liu, Wen-Wen Li, Yi-Fan Dou et al.· Conference on Empirical Meth...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.