Skip to content

ICMAPE: In-Context Multiagent Pure Exploration

Sep 2026 · arXiv.org · Vol abs/2609.33986 · 0 citations · 47 references
Computer Science

TL;DR

ICMAPE converts the fixed-confidence identification objective into a reward derived from inference confidence, so that standard reinforcement learning machinery can be applied to decentralized pure exploration.

Abstract

In some multi-agent systems, the quantity to be optimized is not an externally specified reward but the information acquired about unknown properties of the environment as done in active sequential hypothesis testing (ASHT) problems. However, the ASHT literature tends to focus on finite single-agent problems with well-specified models, while there is currently a gap for practical multi-agent methods that can perform active sequential testing. We fill this gap with ICMAPE, a Bayesian learning-based framework for decentralized multi-agent pure-exploration driven by inference objectives. ICMAPE converts the fixed-confidence identification objective into a reward derived from inference confidence, so that standard reinforcement learning machinery can be applied to decentralized pure exploration. It jointly learns a centralized neural inference network that estimates a posterior distribution over hypotheses from global trajectory data, and decentralized policies that select actions from local observation histories and learn when to stop collecting data once the target confidence is reached. On two synthetic benchmarks and a Maryland nitrate concentration monitoring task based on real-world data, ICMAPE-TD3 achieves target accuracy with fewer exploration steps.

View source

Similar papers

MA-FPPO: Multi-Agent Flow-Pretrained Policy Optimization

This work proposes Multi-Agent Flow-Pretrained Policy Optimization (MA-FPPO), which uses online fine-tuning to improve the cooperative behavior of models pretrained with flow matching through new interactions with the environment.

Guo-Wei Zou, Hao-Nan Chen, Hai-Tao Wang et al. · 0 citations
Open access Aug 2026

Multi-Agent Reinforcement Learning via Agent-Specific Preference

This paper introduces Multi-AGent Preference-Integrated lEarning (MAGPIE), a framework that leverages agent-specific preference signals in the multi-agent learning process and can derive Nash equilibrium solutions.

Ni Mu, Yao Luan, Yiqin Yang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

SRPO: Setwise Relative Policy Optimization for Multi-Agent LLMs

SRPO (Setwise Relative Policy Optimization), which treats the active set the minimal set of outputs consumed by one transition, as one multi-agent action, and unifies division of labor and joint co-evolution as actions with different set sizes.

Sheng-Tian Yang, Zi-Yun Xiong, Yu Li et al. · 4 citations
Preprint Sep 2026

MATES: Learning Multi-Agent Interactions by Transforming Observations for Frozen Single-Agent Policies

Multi-agent reinforcement learning (MARL) commonly trains decentralized policies from scratch, requiring agents to acquire individual task competence and coordination simultaneously. Yet many multi-agent problems admit a compatible single-agent counterpart in which the underlying task can be learned in isolation. We in...

E. Abboud, Oren Gal · 0 citations

Improving LLM Collaboration via Multi-Agent Preference Learning

This work instantiates MAPL using MARL from human feedback with a learned reward model and multi-agent direct preference optimization (MADPO), and shows that MAPL can improve collaboration quality and efficiency while approaching the performance of MARL with fixed, well-defined rewards.

Shuo Liu, Xinzichen Li, Tian-Le Chen et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.