1 Introduction↩︎

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, from mathematical reasoning and code generation to complex problem-solving tasks [1], [2]. Recent advances have extended these capabilities through LLM-based agents that can autonomously interact with external environments and utilize tools [3], [4]. These agent systems have shown that equipping LLMs with the ability to perceive, reason, and act substantially enhances their practical utility. However, individual agents often face inherent limitations in handling complex tasks that require diverse expertise, multiple perspectives, or iterative refinement.

To overcome these limitations, Multi-Agent Systems (MAS) have emerged as a promising paradigm where multiple LLM-based agents collaborate to solve challenging problems [5]. Frameworks such as MetaGPT [6], Magnetic-One [7], and AgentOrchestra [8] employ orchestrator agents to decompose tasks and coordinate specialized agents, while GPTSwarm [9] formalizes MAS as temporal-spatial directed acyclic graphs with distributed agent nodes. Specialized agent frameworks such as ChatDev [10] demonstrate that role-playing agents with structured communication protocols can tackle complex software development tasks. These systems leverage inter-agent communication to make agents share responses, challenge each other’s reasoning [11], and iteratively refine their own solutions [12] to converge to high-quality answers.

Despite these successes, current MAS architectures face critical efficiency challenges stemming from communication overhead and redundancy. One prominent research direction focuses on workflow optimization through pruning strategies. Methods like AgentPrune [13] and AgentDropout [14] employ task-specific training to identify and eliminate redundant communication edges or underperforming agents from the collaboration graph. Another line of work introduces capable orchestrators that train specialized LLMs to dynamically determine optimal topologies and agent configurations [15], [16]. While these approaches achieve performance improvements, they inherently rely on task-specific training data and incur high computational costs. Moreover, recent studies on cross-context communication optimization [17] and parallel agent execution [18] reveal that, for both distributed and orchestrator-based MAS, much redundant computation exists in agent interactions such as repetitive key-value prefilling and non-optimal workflow execution, which contribute minimal value while consuming considerable resources.

The fundamental challenge lies in determining which agent interactions genuinely contribute to improved outcomes without requiring expensive task-aware training. Individual LLM agents also face inherent limitations from lacking external feedback and self-reflection mechanisms [19], making principled communication topology design essential. Graph-based optimization methods demand downstream data for training to learn domain-aware pruning policies, while orchestrator-based approaches necessitate training meta-agents with sufficient capacity to reason about complex collaboration dynamics. Both paradigms struggle with controllability and generalization: trained policies may overfit to specific task distributions, and the training process itself introduces substantial latency and cost. Furthermore, recent work has shown that LLMs exhibit conformity biases in multi-agent settings [20], [21], where agents tend to change their initially correct answers when exposed to multiple incorrect peer responses. Such conformity among LLMs partly reflects the communication redundancy issue in multi-agent systems and indicates that ineffective interactions are probably foreseeable. Considering all the above, we raise a critical question: Can we identify and eliminate redundant or ineffective communications in multi-agent systems without task-specific training?

To address this challenge, we propose a training-free framework, CONsensus- and Confidence-driven Ad hoc Teaming (CONCAT), which leverages intrinsic agent information to achieve efficient collaboration. Our approach is motivated by two observations from empirical analysis: (i) More than 60% agent communications yield neutral or negative impacts, where agents frequently maintain their answers regardless of incoming information, or correct answers degrade to incorrect ones after collaboration. (ii) Collaboration effectiveness can be predicted using simple heuristics based on answer similarity and confidence scores of agents. Building on these insights, CONCAT realizes ad hoc networking via two modules: (1) Consensus clustering and leader selection: answer-based clustering groups agents by their responses; within each cluster, the highest-confidence agent is designated as a leader, and only leaders participate in subsequent exchanges; (2) Benefit-driven edge pruning: a Theory-of-Mind-inspired predictor estimates the utility of communication for each pair of leaders, and the lowest-scoring links are pruned. By constructing sparse topologies through leader selection and benefit-driven pruning, CONCAT maintains competitive accuracy while reducing latency and token usage relative to existing methods.

Our contributions are listed as follows:

• Our empirical analysis on multi-agent collaboration dynamics reveals the prevalence of ineffective communications in existing MAS frameworks and the predictability of collaboration benefits using intrinsic agent signals.

• We propose CONCAT, a training-free MAS framework that combines answer-based agent clustering with Theory-of-Mind-based collaboration benefit prediction to construct efficient communication topologies dynamically.

• Experiments across mathematics, general reasoning, and code generation benchmarks demonstrate that CONCAT achieves higher efficiency than both training-aware and -free baselines, without task-specific training.

2 Background↩︎

2.1 Task Definition↩︎

Following [9], [13], and [14], we formalize an LLM-based Multi-Agent System (MAS) as a directed graph \(\mathcal{G} = (\mathcal{V}, \mathcal{E})\), where \(\mathcal{V} = \{v_1, v_2, \ldots, v_N\}\) represents the set of \(N\) agents, and \(\mathcal{E} \subseteq \mathcal{V} \times \mathcal{V}\) denotes the communication edges between agents. Each directed edge \((v_i, v_j) \in \mathcal{E}\) indicates that agent \(v_j\) can reference and incorporate the reasoning output of agent \(v_i\) during collaboration.

For each agent \(v_i \in \mathcal{V}\), we define its state at round \(t\) as: \[s_i^{(t)} = \{a_i^{(t)}, c_i^{(t)}\},\] where \(a_i^{(t)}\) represents the agent’s answer to the given task, and \(c_i^{(t)} \in [0, 1]\) denotes its confidence. During the collaboration process, agent \(v_i\) updates its state by aggregating information from its incoming neighbors \(\mathcal{N}_{\text{in}}(v_i) = \{v_j : (v_j, v_i) \in \mathcal{E}\}\): \[s_i^{(t+1)} = f_{\theta}\left(s_i^{(t)}, \{s_j^{(t)} : v_j \in \mathcal{N}_{\text{in}}(v_i)\}\right)\] where \(f_{\theta}\) is the LLM-based reasoning function parameterized by \(\theta\).

Communication Redundancy. The key challenge in designing efficient MAS lies in determining an optimal sparse graph structure \(\mathcal{G}^*\) that maintains task performance while minimizing computational overhead. A fully-connected graph (\(|\mathcal{E}| = N(N-1)\)) enables maximal information exchange but incurs quadratic communication complexity. Our goal is to identify a sparse topology \(\mathcal{G}^*\) where \(|\mathcal{E}^*| \ll N(N-1)\), such that: \[\mathcal{G}^* = \arg\max_{\mathcal{G}} \frac{\text{Performance}(\mathcal{G})}{\text{Latency}(\mathcal{G})}\]

In the following subsections, we present empirical observations that motivate our approach to constructing such sparse structures.

2.2 Observation 1: Ineffective Collaboration↩︎

To understand collaboration dynamics in MAS, we conduct statistical analysis using both AgentDropout [14] and LLM-Debate [11]. Our analysis reveals a critical insight: referencing other agents frequently degrades performance rather than improving it.

2.2.0.1 Performance Drop with Collaborations

As shown in Figure 1, agents with references are more frequently misled rather than correctly guided. Notably, when agents receive no references in Round 0 and get two references in Round 1, they change their answers from correct to wrong over \(15\%\) more than they correct their answers from wrong ones.

2.2.0.2 Fine-Grained Analysis via LLM-Debate

To further investigate this phenomenon, we employ LLM-Debate [11] to track answer correctness transitions across agent configurations. As shown in Figure 2, non-beneficial outcomes dominate across all agent configurations: Correct\(\rightarrow\)Correct, Wrong\(\rightarrow\)Wrong, and Correct\(\rightarrow\)Wrong together account for over 83% of all agent pairs, while Wrong\(\rightarrow\)Correct never exceeds 17%. Therefore, we conclude this insight as:

2.3 Observation 2: Predictability of Collaboration Effectiveness↩︎

Building on the insight that many collaborations are ineffective, we investigate whether collaboration outcomes can be predicted a priori from observable agent signals, enabling proactive topology optimization. Specifically, given a focal agent \(v_i\) and a source agent \(v_j\), we ask: does the collaboration benefit \(b_{j\rightarrow i}\) correlate with observable quantities such as answer similarity \(s_{ij}\) and confidence scores \(c_i, c_j\)? Here \(b_{j\rightarrow i}\) is defined as the expected change in \(v_i\)’s answer correctness after referencing \(v_j\).

Predictability via Intrinsic Signals. Each agent pair is labeled as helpful (Wrong\(\rightarrow\)Correct) or not helpful (all other transitions) based solely on the observed answer correctness before and after collaboration (Table 4 in Appendix 9). We then define a simple, training-free dissent strength score: \(d_{j\rightarrow k} = \bar{c}_j \cdot (1 - \text{agree}_{jk})\), where \(\bar{c}_j\) is the mean confidence of source agent \(j\) across all focal agents in the same configuration and \(\text{agree}_{jk}\) indicates whether \(j\) and \(k\) share the same answer. Intuitively, a confident source who disagrees with the focal agent is more likely to provide a corrective signal. As shown in Figure 3, this single feature achieves ROC-AUC of 0.74–0.86 across all agent configurations and both benchmarks, substantially above the random baseline of 0.50. This demonstrates that collaboration benefit is predictable from observable signals alone, providing principled motivation for the benefit-based edge pruning in CONCAT.

3 Method↩︎

3.1 Overview↩︎

Building upon the two empirical observations, we propose CONCAT, Consensus- and Confidence-Driven Ad Hoc Teaming, to address the computational inefficiency and collaboration quality issues in multi-agent systems. As depicted in Figure 4, CONCAT operates in three phases: (1) Initialization: each agent independently generates an answer; (2) Ad Hoc Teaming: agents are grouped via consensus clustering and leader selection, reducing active participants from \(N\) to \(K\) cluster leaders, and benefit-driven edge pruning reconstructs the sparse communication graph at each round; (3) Final Answer Aggregation: an LLM synthesizer aggregates the answers of all agents to produce the final answer. The pseudo code of CONCAT is provided in Algorithm [alg:concat] in Appendix 7.

3.2 Phase 1: Initialization↩︎

The framework initializes with each agent \(v_i \in \mathcal{V}\) independently generating an initial response without observing peer outputs:

\[s_i^{(0)} = \{a_i^{(0)}, c_i^{(0)}\} = f_{\theta}(q, \emptyset),\] where the confidence score \(c_i^{(0)} \in [0,1]\) is computed as the average token probability across all generated output tokens. This independence ensures diverse initial perspectives by preventing premature consensus and the groupthink phenomenon.

3.3 Phase 2: Ad Hoc Teaming↩︎

The core of CONCAT resides in its adaptive debate mechanism, which iteratively refines answers through selective leader interactions. Each round \(t \in \{1, \ldots, m-1\}\) executes six coordinated steps to construct and utilize a sparse topology.

3.3.0.1 Clustering and Leader Selection

At each round \(t\), we first partition the agent population into consensus clusters based on answer similarity: \[\mathcal{C}^{(t)} = \text{ClusterBySimilarity}(\{a_i^{(t-1)}\}_{i=1}^N)\] The similarity metric is task-dependent: for problems with deterministic answers (e.g., multiple-choice, mathematical reasoning), we employ exact matching; for code generation tasks, we use Jaccard similarity over node types of abstract syntax trees. Specifically, we parse code snippets into abstract syntax trees, extract all node types from each tree, and compute the Jaccard coefficient as the ratio of intersection to union of node type sets.

From each cluster \(C_j \in \mathcal{C}^{(t)}\), we select a single leader via confidence maximization in Equation 1 , which results in the leader set in Equation 2 . \[\label{maxconf} l_j = \arg\max_{v_i \in C_j} c_i^{(t-1)}\tag{1}\]

\[\label{allleaders} \mathcal{L}^{(t)} = \{l_1, l_2, \ldots, l_K\} \subseteq \mathcal{V}\tag{2}\]

3.3.0.2 Collaboration Benefit Prediction and Pruning

For each ordered pair of leaders \((l_j, l_k) \in \mathcal{L}^{(t-1)} \times \mathcal{L}^{(t)}\) with \(j \neq k\), we predict the collaboration benefit \(b_{j \rightarrow k}\), defined as the expected utility of \(l_k\) referencing \(l_j\), using a Theory-of-Mind (ToM) based heuristic [22]. We first classify \(l_j\) as a supporter (answer similarity \(s_{jk} \geq \theta_{\text{sim}}\)) or challenger, and compute its effective signal strength \(\hat{c}_j = c_j \cdot s_{jk}\) (supporter) or \(\hat{c}_j = c_j(1-s_{jk})\) (challenger). The benefit for the Challenger case is derived from a Bayesian Expected Utility of Communication (EUC) framework linearized via Taylor expansion: \[\label{eq:benefit95main} \begin{align} b_{j\to k} = \; & \underbrace{4c_k(1{-}c_k)\cdot\hat{c}_j}_{\text{correction gain}} - \underbrace{\alpha\cdot\frac{1+2c_k}{2+2c_k+2\hat{c}_j}}_{\text{inertia discount}} \\ & + \underbrace{\alpha\cdot(1{-}c_k)}_{\text{epistemic openness}}, \end{align}\tag{3}\] where the correction gain \(4c_k(1{-}c_k)\hat{c}_j\) is the first-order Taylor approximation of the EUC, the inertia discount models LLM anchoring via a Beta-Binomial posterior [23], and the epistemic openness term follows Value of Information theory [24]. For the Supporter case: \(b_{j\to k} = \alpha(\hat{c}_j - c_k)\). The full theoretical derivation is in Appendix 11.

With predicted benefits computed for all leader pairs, we construct a sparse communication topology through adaptive thresholding. The pruning threshold is computed in Equation 4 , where \(p \in [0, 1]\) controls edge retention rate and \(\tau_{\min}\) enforces an absolute quality floor. Therefore, the edge set is defined accordingly in Equation 5 .

\[\label{pruning95thres} \tau^{(t)} = \max\left(\text{Percentile}(\{b_{j \to k}\}, p \times 100), \tau_{\min}\right)\tag{4}\] \[\label{edge95set} \mathcal{E}^{(t)} = \{(l_j, l_k) \mid b_{j \rightarrow k} \geq \tau^{(t)}\}\tag{5}\]

3.3.0.3 Leader Answer Refinement

For each leader \(l_k \in \mathcal{L}^{(t)}\), we construct a personalized debate context by collecting responses from beneficial peers: \[\mathcal{R}_k^{(t)} = \{a_j^{(t-1)} \mid (l_j, l_k) \in \mathcal{E}^{(t)}\}\]

Each leader then refines its answer by invoking the language model with the original query and curated peer responses as depicted in Equation 6 , while non-leader agents maintain their previous state without model invocation in Equation 7 . \[\label{leader95refine} s_j^{(t)} = \{a_j^{(t)}, c_j^{(t)}\} = f_{\theta}(q, \mathcal{R}_j^{(t)})\tag{6}\]

\[\label{non95leader} s_i^{(t)} = s_i^{(t-1)}, \quad \forall v_i \notin \mathcal{L}^{(t)}\tag{7}\]

3.4 Final Answer Aggregation↩︎

After \(m-1\) debate rounds, we aggregate all agent responses \(\{a_i^{(m-1)}\}_{i=1}^N\) to produce the final answer. LLM-based synthesizers are employed to generate the final answer by reasoning over all agent responses: \[\hat{y} = f_{\theta}\left(P_{final} \oplus \{a_i^{(m-1)}\}_{i=1}^N\right),\] where \(P_{final}\) is the aggregating prompt.

4 Experiment↩︎

This section will include comparative experiments and an ablation study to verify the effectiveness of CONCAT and its core modules. §4.2 answers the question: Is the proposed method CONCAT more efficient than other baselines of multi-agent systems? §4.3 investigates the importance of clustering and heuristic edge pruning in CONCAT to improve the efficiency of multi-agent collaboration.

4.1 Experiments Setup↩︎

4.1.0.1 Datasets

We conduct experiments on three benchmarks covering diverse domains, which include GSM8k [25] for mathematical tasks, MMLU [26] for general reasoning, and HumanEval [27] for code generation.

4.1.0.2 Baselines

We compare our method, CONCAT, with the following baselines: (1) Chain-of-Thought (CoT) [28]; (2) CoT with self-consistency (SC-CoT) [29]; (3) LLM-Debate [11]; (4) Vanilla MAS; (5) AgentDropout [14]. Among these baselines, (1)-(4) are training-free, and (5) requires task-specific training. Besides, (4) and (5) are applied with five topologies (i.e., Star, Chain, Random, Layered, and Fully Connected).

4.1.0.3 Backbone LLMs

Three LLMs, Llama-3-8B-Instruct [30], Qwen2.5-14B-Instruct, and Qwen2.5-72B-Instruct [31] are used as backbone models of multi-agent systems, which have different model sizes.

4.1.0.4 Implementation Details

We use vLLM [32] to deploy Llama-3-8B-Instruct on one NVIDIA RTX 3090 24G GPU, and Qwen2.5-14B-Instruct on four NVIDIA RTX 3090 24G GPUs with a tensor parallel size of 4. Meanwhile, we use vLLM-Ascend [33] to serve Qwen2.5-72B-Instruct on four Ascend 910B NPUs with a tensor parallel size of 4. For every LLM, we set temperature as 0.7 and top_p as 0.8. The single empirical parameter \(\alpha\) in the benefit predictor is set to 0.2, selected via hyperparameter search (see §4.4). More implementation details are provided in Appendix 10. To ensure reliability, all reported results are averaged over three repetitive runs.

4.2 Main Result↩︎

We compare CONCAT with five baselines on three LLMs and three datasets to validate our framework across varying domains and model sizes. CONCAT achieves superior efficiency while maintaining competitive performance. As shown in Table 1 and Figure 5, CONCAT consistently outperforms baseline methods in terms of efficiency across different base models. Specifically, on Llama3-8B-Instruct, CONCAT achieves an efficiency score of 1.56, substantially higher than LLM-Debate (0.70), all Vanilla MAS variants (0.59-0.76), and all AgentDropout variants (0.67-0.92). The efficiency gain is even more pronounced on Qwen2.5-14B-Instruct, where CONCAT reaches an efficiency of 2.85, which is 2.02\(\times\) higher than LLM-Debate (1.41), 1.94\(\times\) higher than the best Vanilla MAS variant (1.47), and 1.58\(\times\) higher than the best AgentDropout variant (1.80). Importantly, CONCAT maintains competitive average accuracy of 64.97% on Llama3-8B and 86.02% on Qwen2.5-14B, demonstrating its ability to balance accuracy and computational cost.

CONCAT substantially reduces latency and token consumption. On Llama3-8B-Instruct, CONCAT reduces average latency by 55.5% vs.LLM-Debate and 40.4% vs.the best AgentDropout variant. On Qwen2.5-14B-Instruct, the latency reduction reaches 50.1% vs.LLM-Debate and 35.6% vs.the best AgentDropout variant. Token consumption follows the same trend: CONCAT averages 1.9M total tokens on Llama3-8B, representing 45.7% and 48.6% reductions compared to LLM-Debate (3.5M) and Vanilla MAS Layered (3.7M), respectively. Notably, CONCAT achieves slightly higher accuracy than Vanilla MAS Layered (64.97% vs.%) while reducing latency by 33.3%. Detailed per-benchmark token statistics are provided in Appendix 12.

4.3 Ablation Study↩︎

To validate the effectiveness of each component in CONCAT, we conduct ablation experiments by removing or randomizing the answer-based clustering and edge pruning modules individually.

4.3.0.1 Impact of Edge Pruning.

As shown in Table 2, removing clustering increases average latency by 54.9% while reducing accuracy by 1.50%, confirming that consensus-based leader selection is the primary driver of computational efficiency. Removing benefit-prediction edge pruning yields a smaller latency increase of 6.2% but causes a 0.76% accuracy drop, most notably on MMLU (\(-\)​1.96%), indicating that edge pruning functions as a quality gate that filters harmful communications. The two modules are thus complementary: clustering reduces interaction scale while edge pruning preserves interaction quality.

4.3.0.2 Impact of Random Baselines.

To further isolate the contribution of each algorithmic choice, we compare CONCAT against two random baseline variants on Qwen2.5-14B-Instruct. w/ Rand. Edge Pruning retains confidence-driven leader selection but replaces benefit-prediction pruning with random edge removal, achieving accuracy of 85.79% with average latency 26.95s (efficiency 3.18). w/ Rand. Selection & Pruning additionally replaces confidence-driven leader selection with random agent selection, yielding accuracy of 84.68% with latency 26.72s (efficiency 3.17). Both random variants achieve slightly lower latency than CONCAT, yet fall 0.23–1.34 points below CONCAT in average accuracy (86.02%), demonstrating that the ToM-based benefit predictor and confidence-driven leader selection provide meaningful accuracy gains over random alternatives. Notably, w/ Rand. Edge Pruning outperforms CONCAT on MMLU (79.08% vs.%), which we attribute to imperfect confidence calibration on heterogeneous knowledge tasks.

4.4 Hyperparameter Sensitivity Analysis↩︎

The benefit predictor in CONCAT contains a single empirical parameter \(\alpha = 0.20\), which jointly controls the inertia-discount weight and the epistemic-openness weight in the Challenger formula (Eq. 3 ). We evaluate five candidate values \(\alpha \in \{0.10, 0.15, 0.20, 0.25, 0.30\}\) on Qwen2.5-14B-Instruct across all three benchmarks, using mean efficiency as the selection criterion, where mean efficiency is defined as average accuracy divided by average latency.

As shown in Table 3, \(\alpha = 0.10\) leads to significantly degraded efficiency (Mean Eff.= 1.88 vs.\(\geq\)​2.82 for other values), driven by anomalously high GSM8K latency (80.01s vs.\(\sim\)​51s). A very small \(\alpha\) renders the inertia discount negligible, causing the benefit predictor to over-retain edges and reverting toward dense communication. For \(\alpha \geq 0.15\), mean efficiency remains within a narrow range of 2.83–2.86, demonstrating low sensitivity to \(\alpha\) in this regime. We select \(\alpha = 0.20\) as it achieves the overall highest average accuracy across all benchmarks (86.02%) while maintaining competitive mean efficiency (2.85).

5 Conclusion↩︎

This paper explores methods to enhance the efficiency of LLM-based multi-agent systems by eliminating redundant communications in a training-free manner. We propose CONCAT, Consensus- and Confidence-Driven Ad Hoc Teaming, a self-organization MAS framework that realizes comparable accuracy while reducing the overall latency.

Our study points to several directions for further improvement of LLM-based multi-agent systems, including developing more robust ad hoc networking mechanisms that precompute the sparse topology for MAS inference, scaling up multi-agent systems through self-organization, and realizing controllable multi-agent collaborations using the Bayesian Theory of Mind model to supervise communication processes. Furthermore, Improving the confidence calibration of individual agents through methods such as temperature scaling or Bayesian calibration may further enhance the reliability of the ToM-inspired benefit predictor on heterogeneous knowledge benchmarks. Moreover, extending training-free and self-organizational multi-agent systems to broader multimodal domains and diverse real-world tasks such as web search is also promising for future research.

Limitations↩︎

Our work focuses on achieving efficient LLM-based multi-agent systems through consensus-based agent teaming and heuristic edge pruning for sparse communication. While our approach demonstrates promising results, several limitations should be acknowledged.

5.0.0.1 Confidence Quantification

In our current implementation, an agent’s confidence for a given response is calculated as the average token probability across output tokens. However, this represents a simplified measure of uncertainty. Alternative confidence quantification methods, such as entropy [34] and Chain-of-Embeddings [35], may provide more accurate uncertainty estimates and warrant further investigation.

5.0.0.2 Extension to Real-World Scenarios

We validate CONCAT on three common domains: mathematics, general reasoning, and code generation. However, the generalizability of CONCAT to other complex benchmarks, such as GAIA [36], requires further verification. Additionally, our current implementation uses predefined, domain-specific agent roles that remain fixed throughout execution. Consequently, applying CONCAT to new domains necessitates careful design of targeted roles through prompt engineering, which may require domain expertise and iterative refinement.

Acknowledgments↩︎

We used Claude Sonnet 4.6 and Deepseek V4 Pro to polish the writing in this paper, including rephrasing sentences for clarity, improving grammatical fluency, and smoothing transitions. All scientific content, arguments, and conclusions were conceived and written by the authors. No AI-generated content was introduced beyond surface-level language editing.

6 Related Work↩︎

6.0.0.1 LLM-Based Multi-Agent System

LLM-based multi-agent systems have emerged as a promising paradigm for solving complex tasks through collaborative intelligence, primarily adopting two architectural patterns: orchestrator-worker and distributed collaboration. In orchestrator-worker architectures, a central planner agent decomposes complex tasks into manageable subtasks and coordinates specialized worker agents to complete their respective responsibilities. Magnetic-One [7] employs a dedicated orchestrator agent that dynamically plans task execution and delegates subtasks to domain-specific agents with heterogeneous capabilities. Hong et al. [6] propose MetaGPT, an innovative meta-programming framework that encodes standardized operating procedures into prompt sequences, enabling agents with human-like domain expertise to verify intermediate results through an assembly line paradigm. AgentOrchestra [8] enhances orchestration flexibility by allowing the planner to adaptively adjust agent compositions and workflow structures based on task requirements and intermediate execution feedback.

In contrast, distributed multi-agent architectures formalize collaboration through decentralized communication patterns without centralized control. GPTSwarm [9] represents multi-agent systems as temporal-spatial directed acyclic graphs, where different agents serve as distributed nodes that communicate and collaborate through graph edges, enabling flexible information flow and parallel task execution. This graph-based formalization allows agents to interact dynamically based on task dependencies and information requirements, supporting more scalable and fault-tolerant collaboration patterns compared to centralized orchestration approaches.

6.0.0.2 Efficient LLM-Based Multi-Agent System

Recent research has explored various strategies to improve the efficiency of LLM-based multi-agent systems. For orchestrator-worker architectures, several approaches focus on optimizing the central coordination mechanism and execution paradigm. [16] propose MAS-GPT, which trains a 32B model to generate complete multi-agent systems adaptively for different queries, achieving efficiency through single-inference generation rather than iterative planning. Puppet [15] enhances coordination flexibility through an evolving orchestration mechanism that dynamically adjusts agent workflows based on intermediate execution states. [18] propose to utilize parallel plan execution in M1-Parallel, which simultaneously runs different workflow plans and selects the fastest completion to reduce overall latency.

In contrast, distributed multi-agent systems have witnessed efficiency improvements through graph-based communication optimization and KV-cache management techniques. AgentPrune [13] reduces communication overhead by employing graph neural networks to learn and prune unnecessary communication edges in the agent interaction topology. [14] propose to utilize dynamic agent elimination strategies in AgentDropout, which removes redundant nodes and their associated communications to minimize token consumption while preserving collaboration effectiveness. Li et al. [17] introduce KVCOMM, an online cross-context KV-cache communication mechanism that enables efficient sharing and reuse of cached representations across agents, avoiding redundant processing of shared contextual information.

Despite these advances, existing graph-based communication optimization methods for distributed multi-agent systems face critical limitations. These approaches are typically designed for specific topology structures and require training graph neural networks on task-specific datasets with carefully annotated communication patterns. This training-based paradigm suffers from prolonged training time, limited generalization to unseen task distributions and agent configurations, and persistent redundant communications that remain even after optimization. Additionally, orchestrator improvement methods necessitate large-scale, high-quality datasets capturing diverse multi-agent interactions and substantial training resources, while parallel inference acceleration incurs high computational costs with weak controllability over efficiency-resource trade-offs, collectively limiting their practical applicability across diverse application scenarios.

6.0.0.3 Theory of Mind in Multi-Agent Systems

Theory of Mind (ToM) [37] refers to the ability to attribute mental states, including beliefs, desires, and intentions, to oneself and others. Recent work has explored leveraging ToM in LLM-based multi-agent systems to improve coordination. [38] evaluate LLM agents in cooperative text games requiring ToM inference, revealing that explicit ToM modeling can improve collaborative task performance. [39] scaffold ToM reasoning for multi-agent tasks, enabling LLMs to hypothesize about other agents’ states and coordinate more effectively. More recent work [40] proposes adaptive ToM for real-time multi-agent coordination, while [41] extends ToM evaluation to multimodal embodied settings via a dedicated benchmark. [42] provide a systematic evaluation of BDI-based ToM architectures in multi-agent LLM systems. Distinct from these approaches that directly model agent intent, CONCAT employs a lightweight ToM-inspired heuristic [22] to predict communication benefit without explicit mental state representation, enabling efficient training-free topology optimization.

6.0.0.4 LLM Conformity

The conformity behavior of large language models, where LLMs adjust their responses based on information from previous interactions or peer outputs, has recently attracted research attention due to its implications for multi-agent collaboration. [21] investigate conformity in large language models and reveal that repeated wrong answers can mislead LLMs from correct responses to incorrect ones, while LLMs with confident initial responses tend to resist conforming to alternative answers. Do as We Do [20] explores how LLMs conform or resist conforming according to the consistency of previous discussions, demonstrating that larger models exhibit less conformative tendencies and maintain their original judgments more robustly. Both studies identify critical factors influencing conformity, including response confidence, answer repetition frequency, and model scale. However, existing investigations primarily employ rigid experimental settings that present LLMs with either unanimous consensus or controlled opposition scenarios, largely ignoring realistic situations where models are provided with multiple diverse answers or mixed signals containing both correct and incorrect information. This limitation restricts our understanding of how LLMs navigate complex, heterogeneous information landscapes in practical multi-agent systems, where agents may produce varied outputs with different confidence levels and correctness.

7 Algorithmic Description of CONCAT↩︎

Algorithm [alg:concat] describes the workflow of our proposed framework, CONCAT.

8 ToM-based Collaboration Benefit Prediction↩︎

The algorithm classifies the inter-agent relationship based on answer similarity, distinguishing between supporters (\(s_{jk} \geq \theta_{\text{sim}}\)) and challengers. For each case, the effective signal strength \(\hat{c}_j\) captures how strongly agent \(v_j\)’s signal challenges or supports agent \(v_k\): \[\hat{c}_j = \begin{cases} c_j \cdot s_{jk} & \text{if SUPPORTER} \\ c_j \cdot (1 - s_{jk}) & \text{if CHALLENGER} \end{cases}\]

8.0.0.1 Challenger case.

The benefit formula is derived from a Bayesian Expected Utility of Communication (EUC) framework [22], linearized via Taylor expansion around \(\hat{c}_j=0.5\):

\[\label{eq:benefit} \begin{align} b_{j\to k} = \; & \underbrace{4c_k(1{-}c_k)\cdot\hat{c}_j}_{\text{correction gain}} - \underbrace{0.2\cdot\frac{1+2c_k}{2+2c_k+2\hat{c}_j}}_{\text{inertia discount}} \\ & + \underbrace{0.2\cdot(1{-}c_k)}_{\text{epistemic openness}}, \end{align}\tag{8}\]

The coefficient \(4c_k(1{-}c_k)\) is the derivative of the exact EUC formula at \(\hat{c}_j{=}0.5\), capturing the intuition that agents with moderate confidence are most receptive to correction. The inertia discount \(p_{\text{stay}} = \frac{1+2c_k}{2+2c_k+2\hat{c}_j}\) is the Beta-Binomial posterior mean [23], modeling LLM anchoring effects [43]. The epistemic openness term \((1{-}c_k)\) follows Value of Information theory [24]: uncertain agents gain more from additional signals. The benefit threshold \(\tau_{\min}=0\) is theoretically justified by the exact EUC formula (Appendix 11): \(b_{j\to k}<0\) when the source signal \(\hat{c}_j\) falls below the focal agent’s correction threshold \(\hat{c}_j^*(c_k)\), meaning communication is predicted to reduce expected correctness. Negative-benefit edges are therefore absolutely filtered regardless of the percentile cutoff \(p\).

8.0.0.2 Supporter case.

When \(v_j\) agrees with \(v_k\) (\(s_{jk} \geq \theta_{\text{sim}}\)), the benefit quantifies confidence reinforcement: \[b_{j\rightarrow k} = \alpha \cdot (\hat{c}_j - c_k), \quad \hat{c}_j = c_j \cdot s_{jk}\] A supporter with higher effective confidence than \(v_k\) provides positive value; an uncertain supporter yields negative benefit.

This lightweight heuristic contains a single empirical parameter \(\alpha=0.2\), selected via hyperparameter search in §4.4. It operates purely on observable signals without training data, and is motivated by the empirical finding that dissent strength predicts collaboration benefit with ROC-AUC of 0.74–0.86, as shown in Figure 3. The full theoretical derivation, covering the exact EUC formula, three strict propositions, Taylor linearization, and Beta-Binomial conjugate update, is provided in Appendix 11.

9 Collaboration Outcome Labels↩︎

Table 4 defines the four collaboration outcome categories used in Figure 2 and for computing the helpful/not-helpful binary label in Figure 3. Each agent pair is classified solely by the focal agent’s answer correctness before (\(S_1\)) and after (\(S_2\)) collaboration, without reference to confidence scores.

10 Supplementary Implementation Details↩︎

For Llama-3-8B-Instruct, Qwen2.5-14B-Instruct, and Qwen2.5-72B-Instruct, we configure the maximum sequence length for model outputs to 32,768 tokens. For CONCAT, we set the edge retention rate as 0.7 and the code similarity threshold for clustering as 0.45. The clustering algorithm on the code generation task is hierarchical clustering. For SC-CoT, five responses for each question are sampled for majority voting. For multi-agent methods, we utilize a unified setting with five agents and two-round collaboration. The answer aggregation prompts on MMLU, GSM8K, and HumanEval are shown in Figure 6, 7, and 8.

11 Theoretical Derivation of the Benefit Predictor↩︎

This appendix provides the complete mathematical derivation underlying the ToM-based benefit predictor in Algorithm [alg:tom95predict].

11.1 Problem Setup and Assumptions↩︎

For a focal agent \(v_k\) with confidence \(c_k\) and a source agent \(v_j\) with effective signal strength \(\hat{c}_j\) (Eq.of the Challenger case), we define the Expected Utility of Communication (EUC): \[b_{j\to k}^{\text{exact}} \triangleq \mathbb{E}[U \mid s_j] - \mathbb{E}[U]\] where \(U\) is a 0-1 utility that equals 1 if \(v_k\) answers correctly. The key assumptions are: (A1) \(c_i \triangleq P_i(a_i = a^*)\): we interpret confidence as a proxy for the subjective probability of correctness; in practice, \(c_i\) is computed as the average token probability of the response (see §3.2), which serves as an empirical approximation of this quantity; (A2) \(v_k\) makes a binary choice between \(\{a_k, a_j\}\); (A3) conditional independence of beliefs given \(a^*\); (A4) \(v_k\) uses \(c_j\) as its estimate of \(v_j\)’s correctness, following first-order ToM [22].

11.2 Exact EUC Formula (Challenger Case)↩︎

Before communication: \(\mathbb{E}[U]_{\text{before}} = c_k\).

After observing \(v_j\)’s signal, \(v_k\) performs Bayesian belief update. Two hypotheses compete:

The normalization factor is \(Z = c_k(1-\hat{c}_j) + (1-c_k)\hat{c}_j\). The Bayesian-optimal decision chooses the larger posterior: \[b_{j\to k}^{\text{exact}} = \frac{\max\bigl(c_k(1-\hat{c}_j),\;(1-c_k)\hat{c}_j\bigr)}{Z} - c_k\]

Proposition 1 (Sign): \(b_{j\to k}^{\text{exact}} > 0 \iff \hat{c}_j > \hat{c}_j^*(c_k)\), where the threshold \[\hat{c}_j^*(c_k) \;=\; \frac{c_k^2}{1 - 2c_k + 2c_k^2}\] is a monotonically increasing function of \(c_k\) that equals \(c_k\) only at \(c_k = 0.5\). When \(c_k > 0.5\), the threshold exceeds \(c_k\): a highly confident focal agent requires a correspondingly stronger challenge signal to benefit. When \(c_k < 0.5\), the threshold falls below \(c_k\): an uncertain focal agent benefits even from a moderately confident dissenter.

Proposition 2 (Monotonicity): \(b_{j\to k}^{\text{exact}}\) is monotonically increasing in \(\hat{c}_j\) and monotonically decreasing in \(c_k\).

Proposition 3 (Zero point): \(b_{j\to k}^{\text{exact}} = 0 \iff \hat{c}_j = \hat{c}_j^*(c_k)\).

These propositions justify setting \(\tau_{\min}=0\): by Proposition 1, edges with \(b_{j\to k} < 0\) correspond to source signals too weak to overcome the focal agent’s prior, and are therefore predicted to be harmful or neutral on average.

11.3 Taylor Linearization↩︎

The exact formula is numerically unstable when \(c_k \to 1\) because the denominator \(Z \to 0\). It also does not model LLM anchoring effects [43]. We expand around \(\hat{c}_j = 0.5\): \[b_{j\to k}^{\text{exact}} \approx (1-2c_k) + 4c_k(1-c_k)\cdot(\hat{c}_j - 0.5)\]

The coefficient \(4c_k(1-c_k)\) has an elegant interpretation: it is maximized at \(c_k=0.5\), where the agent is most receptive to correction, and approaches 0 as \(c_k \to 0\) or \(c_k \to 1\), where extreme confidence makes the agent insensitive to challenges. Absorbing the constant \(-2c_k(1-c_k)\) into the intercept and simplifying gives the correction gain term \(4c_k(1-c_k)\cdot\hat{c}_j\) in Eq. 3 .

11.4 Beta-Binomial Inertia Discount↩︎

LLMs exhibit anchoring effects: even when \(\hat{c}_j > c_k\), the agent does not always switch answers. We model the probability of \(v_k\) maintaining its answer via Beta-Binomial conjugate updating [23]:

• Prior strength: \(\alpha_0 = 1 + 2c_k\), where confidence acts as an equivalent prior sample count.

• Challenge evidence: \(\beta_0 = 1 + 2\hat{c}_j\), where challenge strength acts as equivalent counter-evidence.

• Posterior mean: \(p_{\text{stay}} = \frac{1+2c_k}{2+2c_k+2\hat{c}_j}\).

When \(c_k\) is high, \(p_{\text{stay}}\) is large and the agent is harder to dislodge. When \(\hat{c}_j\) is high, \(p_{\text{stay}}\) is small and a strong challenge overcomes inertia. This is consistent with empirical conformity findings [21].

11.5 Epistemic Openness Term↩︎

By Value of Information (VoI) theory [24], an agent with higher uncertainty benefits more from additional information. The term \((1-c_k)\) directly proxies \(v_k\)’s epistemic uncertainty, providing a floor for communication benefit even when \(\hat{c}_j\) is not particularly high. Combining all components with empirical weight \(\alpha=0.2\), selected via hyperparameter search in §4.4, gives the final Challenger formula in Eq. 3 .

12 Token Consumption↩︎

As shown in Table 5, CONCAT consumes substantially fewer tokens than full-communication baselines across all three benchmarks. On average, CONCAT uses 1.9M total tokens on Llama-3-8B-Instruct, representing a 45.7% reduction compared to LLM-Debate (3.5M) and a 48.6% reduction compared to Vanilla MAS Layered (3.7M). The savings are most pronounced on GSM8K, where CONCAT’s 5.1M total tokens compare favorably against LLM-Debate’s 8.9M and Vanilla MAS Layered’s 9.6M. Among all MAS methods, AgentDropout (Chain) achieves the second-lowest token consumption (1.7M avg.) owing to its sparse topology, yet CONCAT surpasses it in efficiency by maintaining higher average accuracy (64.97% vs.%) with a lower-latency communication schedule.

References↩︎