This work instantiates MAPL using MARL from human feedback with a learned reward model and multi-agent direct preference optimization (MADPO), and shows that MAPL can improve collaboration quality and efficiency while approaching the performance of MARL with fixed, well-defined rewards.
Abstract
Several works have explored multi-agent reinforcement learning (MARL) in LLM collaboration. However, constructing reliable rewards is difficult in practice, as complete and accurate metrics are often unavailable and hard to aggregate. Preference learning provides an alternative by learning from comparative human or AI feedback. Yet, its extension to multi-agent systems remains underexplored. To address this gap, we formulate preference-based multi-agent systems (MAS) from decentralized and centralized collaboration perspectives. We also introduce a general multi-agent preference learning framework (MAPL) to solve these problems. MAPL allows iterative updates by comparing the current solution with decentralized or centralized solutions generated by various agents. We instantiate MAPL using MARL from human feedback (MARLHF) with a learned reward model and multi-agent direct preference optimization (MADPO). Experiments on collaborative writing, coding, tool use, and travel planning show that MAPL can improve collaboration quality and efficiency while approaching the performance of MARL with fixed, well-defined rewards. Within MAPL, MARLHF generally outperforms MADPO on most tasks but remains sensitive to data coverage, agent and comparator models, and the underlying MARL algorithms.
Focusing on multi-agent path finding as an exemplary problem, this paper proposes to simplify two popular approaches to MAPF, namely multi-agent reinforcement learning and adaptive search, to enable seamless combination and transferability of methods without substantial engineering effort.
Thomy Phan· Proceedings of the Thirty-Fi...· 0 citations
This work proposes Multi-Agent Flow-Pretrained Policy Optimization (MA-FPPO), which uses online fine-tuning to improve the cooperative behavior of models pretrained with flow matching through new interactions with the environment.
Guo-Wei Zou, Hao-Nan Chen, Hai-Tao Wang et al.· arXiv.org· 0 citations
The integration of reinforcement learning (RL) into the optimization of multi-agent collaboration for Large Language Models (LLMs) is an important combination of two advanced areas, Multi-Agent Systems (MAS) and LLMs. This paper thoroughly examines the main approaches, evaluation standards, recent progress and existing...
Qian-Ling Zhang· Applied and Computational En...· 0 citations
ICMAPE converts the fixed-confidence identification objective into a reward derived from inference confidence, so that standard reinforcement learning machinery can be applied to decentralized pure exploration.
Training robust policies in Multi-Agent System (MAS) remains challenging due to high sample complexity, unstable policy updates, and insufficient coordination among agents. Multi-Agent Proximal Policy Optimization (MAPPO) has recently demonstrated strong performance in cooperative multi-agent tasks; however, it does no...
In complex, partially observable environments such as real-time strategy games, relying on decentralized multi-agent systems to learn efficient cooperative strategies is a challenging task. This paper proposes a Role Cognition Decomposition algorithm based on Multi-Agent Reinforcement Learning (RCD-MARL), which combine...