This article introduces a framework for designing and running simulated experiments with LLM‐powered agents and applies the framework to the exploration–exploitation dilemma and shows that LLM‐based experiments reproduce patterns observed among human participants.
Abstract
We explore how Large Language Models (LLMs) can serve as synthetic subjects to inform strategy research. We introduce a framework for designing and running simulated experiments with LLM‐powered agents. We argue that this approach is useful for rapid, low‐cost prototyping of human experiments and for generating novel hypotheses. We apply the framework to the exploration–exploitation dilemma and show that LLM‐based experiments reproduce patterns observed among human participants. We then vary parameters and boundary conditions to illustrate how the same setup can support design iteration and surface hypotheses about when and why established results change. In the conclusion, we discuss the promise and limitations of artificial intelligence agents as “model organisms” for strategy.
Artificial intelligence (AI) agents are beginning to enter firms as tools that can execute work, from writing code to coordinating complex tasks across systems. This article argues that their value for strategy extends beyond task automation: AI agents can also be used to simulate strategic interactions and assess how strategies perform under alternative assumptions. In our exploration–exploitation application, these simulations reproduce core patterns from prior human experiments and reveal where those patterns weaken or reverse. Used this way, AI agents can help firms prototype strategic choices, stress‐test assumptions, and direct managerial attention toward promising leads before larger commitments of time and effort.
Exploration is essential for reliable autonomy in multi-agent systems, yet it remains unclear whether large language model (LLM) agents can explore effectively when interacting with one another. We show that modern LLM agents fail to do so, often exhibiting myopic and polarized interaction patterns that lead to suboptimal coordination and increased regret. We formalize this challenge as the Multi-Agent Exploration problem, modeling it as a partially observable stochastic game (POSG) problem in which agents must probe peers to infer their capabilities and identify effective interaction strategies. To address this, we introduce Multi- Agent Contextual Exploration (MACE), a lightweight framework that explicitly promotes exploration through structured peer selection. Across both contextual and parametric diversity settings, MACE substantially improves exploration behavior and downstream task performance. We further show theoretically that the value of exploration increases with agent diversity. Overall, our results highlight a fundamental limitation of current LLM agents and underscore the importance of explicitly guided exploration for reliable multi-agent autonomy. Code will be released in https://github.com/deeplearning-wisc/mace
Hyeong Kyu Choi, Jiatong Li, Wendi Li et al.· 1 citation
This work argues that studying AI Scientists as human-agent systems (HAS) is both underexplored and undervalued, and calls for new research that adopts the HAS lens to develop mathematical frameworks for understanding and fostering human-AI synergy in scientific discovery.
P. Emami, Sameera Horawalavithana, T. Nguyễn et al.· 0 citations
Intelligent machines have the potential to uncover problem-solving strategies beyond human discovery. Emerging evidence from competitive gameplay, such as Go and chess, demonstrates that AI systems are evolving from mere tools to sources of cultural innovation adopted by humans. However, the conditions under which intelligent machines transition from tools to drivers of persistent cultural change remain unclear. We identify three key dimensions that modulate machine influence on human problem-solving: the discovered strategies must be non-trivial, learnable, and offer a clear advantage. Using a cultural transmission experiment, we demonstrate that when these conditions are met, machine-discovered strategies can be transmitted, understood, and preserved by human populations, leading to enduring cultural shifts. Conversely, using agent-based simulations, we show how machine influence is constrained in the absence of these conditions. These findings provide a framework for understanding how machines can persistently expand human cognitive skills and underscore the need to consider their broader implications for human cognition and cultural evolution. AI can uncover problem-solving strategies beyond human discovery. Here, the authors show that AI-discovered strategies propagate and persist in human populations, producing cultural shifts when non-trivial, learnable, and advantageous.
L. Brinkmann, Thomas F. Eisenmann, Anne-Marie Nussberger et al.· Nature Communications· 0 citations
Large Language Models (LLMs) are increasingly used in simulations, either in academic settings for research studies or in industry for prototyping. Previous research has investigated the extent to which agents can mimic human behavior in socioeconomic settings; however, there is limited research on greedy decision-making by agents in simulated resource allocation environments. Furthermore, there is limited work on cross-model evaluation. Our research investigates the decision-making of ten agents across two different experimental conditions: one in which agents are able to communicate with other agents, and one in which emotional contexts are directly injected into the prompt. Based on a conceptual framework of greed, we find that agents predominantly exhibited greed-like behavior across all conditions. Interaction and self-reported social connection did not meaningfully influence the agents’ decision-making. We evaluated four different models: gpt-5-mini, gemini-3-flash-preview, claude-haiku-4-5-20251001, and grok-3-mini-fast-beta, and observed that while models differed in their self-reported connection scores, they did not differ significantly in greediness scores. The code and experimental artifacts are available at https://github.com/tiaL-ops/simCo.
Landy Rakotoarison, Fanamby T. Randriamahenintsoa· International Conference on...· 0 citations
An analysis that derives task-specific algorithmic design spaces from human-designed methods, maps both human- and agent-designed methods into these spaces, and quantifies their algorithmic differences at the module level suggests that although current agents can occasionally match or surpass human SOTA performance, their algorithmic designs remain within human-derived algorithmic design spaces.
Yikang Yang, Zhengxin Yang, Luzhou Peng et al.· 0 citations
An organizing framework for understanding LLM‐based agents is established, systematically deconstructing both single‐agent and multi‐agent systems into their core components, and the architectural principles and key mechanisms that underpin their intelligence are analyzed.
Yuheng Cheng, Ceyao Zhang, Zheng-Wen Zhang et al.· WIREs Data Mining and Knowle...· 1 citation