Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?

The effectiveness of multi-agent LLM deliberation depends not only on the agents’ individual predictions, but also on how they communicate and collaborate. We study this mechanism through the lens of Friedkin-Johnsen (FJ) opinion dynamics, a tractable model for analyzing stubbornness, influence, and opinion change in multi-agent systems that captures empirically observed deliberation patterns. We show that the FJ parameters are input-dependent, turning multi-agent deliberation into a mixture of experts. This perspective implies that multi-agent systems can outperform single agents and static ensembles when routing reflects agent competence. Since competence is latent in practice, we analyze how influence is established through observable proxies: agents’ self-assessed confidence, their perceived confidence, and initial alignment with other agents’ views.

Introduction. Large language models (LLMs) have led to significant advancements in natural language processing across various tasks. Recently, multi-agent systems (MASs) composed of interacting LLMs have attracted attention for their potential to improve performance, particularly in tasks involving strategic reasoning, negotiation, and generative design [7, 38, 23, 3, 11]. In MASs, multiple agents communicate iteratively to deliberate and refine predictions, with the promise that diverse agents can contribute complementary expertise and improve decision-making. However, the empirical benefits of MASs over single-agent models or static ensembles are mixed [27, 39, 31]. A key factor in the success of MASs is understanding how influence is distributed during deliberation. Not all agents are equally persuasive, and the central question is: What makes some agents more influential than others in a multi-agent deliberation?

Discussion / Conclusion. We have provided a framework to study the mechanisms of collaboration in multi-agent LLM systems (MASs) and gained new insights into the factors governing the emergence of influence, including the confidence of agents, their communication behavior and opinion alignment. Building on the observation that the Friedkin-Johnsen model captures MAS deliberation dynamics, we have cast MASs as mixtures of experts (MoE) with adaptive routing. This has allowed us to derive conditions when MASs can outperform simpler ensembles and single agent models. Accordingly, their strengths arise from adaptive routing and local specialization of diverse and well calibrated agents, while its limitations arise from miscalibrated agent confidence, misleading consensus, and routing errors.