Multi-agent systems powered by large language models (LLMs) have demonstrated potential for collaborative problem-solving, yet increasing the number of agents often introduces redundant reasoning and communication overhead, sometimes degrading performance. We propose AgentDropout, a dynamic strategy inspired by dropout regularization in neural networks, which selectively deactivates low-contribution agents during multi-agent collaboration. At each round of discussion, AgentDropout computes a semantic novelty score for every agent by measuring the divergence of its output relative to the current group consensus. Agents whose novelty score falls below an adaptive threshold are temporarily deactivated, reducing token consumption without sacrificing viewpoint diversity. We evaluate AgentDropout on mathematical reasoning (GSM8K), commonsense reasoning (StrategyQA), and collaborative code generation (HumanEval) tasks. Across three independent runs, AgentDropout achieves accuracy comparable to or modestly above fixed 5-agent debate while reducing total token consumption by 38.0–43.5% and debate rounds by 20.5% on average. Pareto analysis reveals a promising efficiency–quality trade-off, suggesting that dynamic agent deactivation may be useful for deploying multi-agent LLM systems under computational budget constraints.
Zhengxi Xiao, Qi Guo, Yuyue Wang et al.· 2026 8th International Confe...· 1 citation
Words recur constantly in natural language use, yet it remains unclear whether language models reactivate prior representations or re-evaluate repeated words afresh, and whether post-training changes this default behavior. We apply repetition priming (Shiffrin and Schneider, 1977) to 15 models across five model families (1.5B-14B parameters) in two tasks, semantic categorization and cloze completion, with matched human experiments using identical stimuli. We find that base models exhibit automatic processing: they show immediate facilitation that remains stable across lags, partially survives context removal, and correlates with attention to prior occurrences. Instruct models exhibit controlled processing: their facilitation decays with lag, collapses without expected context, and reverses to interference at larger scales. Within the Qwen 2.5 family, this dissociation increases monotonically with model scale, suggesting that post-training progressively alters repetition processing. Humans show a hybrid profile, with lag-sensitive facilitation resembling instruct models but without interference, suggesting that neither model type fully captures human cognition. Our findings reveal a qualitative shift in how language models process repeated information after post-training and provide mechanistic evidence for the divergence between model behaviors.
