1 Introduction↩︎

Recent advances in large language models (LLMs) have enabled the development of autonomous intelligent agents that can interact with their environment, plan actions, and make decisions with minimal human intervention [1]. As individual LLM-based agents [2] exhibit domain-specific strengths, such as proficiency in programming [3], biological analysis [4], or mathematical reasoning [5], multi-agent decision-making frameworks have emerged to exploit the complementary capabilities of multiple specialised agents [2], [6], [7]. In principle, these frameworks coordinate agents’ actions and beliefs to integrate heterogeneous expertise and produce more informed and reliable decisions.

In the context of LLMs, multi-agent decision making is commonly realised through voting and debate [8]. Voting, particularly majority voting, aggregates agents’ independent judgments into a collective decision [9]. While computationally efficient and simple, it disregards variation in agent expertise, which may reduce informativeness and lead to suboptimal decisions. In contrast, multi-agent debate (MAD) [10] enables agents to iteratively exchange perspectives and update their beliefs through structured, multi-round discussions, with the goal of reaching a more coherent and higher-quality collective decision. Despite this promise, recent empirical studies [8] indicate that multi-round debate is often less reliable and less effective than simple voting-based approaches. MAD systems are shown to frequently converge toward majority-held misconceptions rather than correct reasoning [11]. In homogeneous settings, straightforward ensemble techniques often outperform multi-round debate [8]. MAD is also reported to fail to surpass even single-agent baselines [12]. More broadly, LLMs exhibit substantial performance degradation in multi-turn settings regardless of protocol [13], suggesting the limitation is not unique to debate. These observations motivate studying when multi-LLM systems fail to extract all available information, regardless of protocol.

In this paper, we analyse multi-agent LLM decision making through Blackwell’s informativeness framework [14]. This perspective allows us to compare voting, debate, and posterior-level aggregation in terms of the information structures. We show that voting and debate are less informative than the agents’ joint private information, which serves as the Blackwell-optimal reference rule. Motivated by this analysis, we develop MA-PoP, a practical method for multiple-choice QA that estimates the pooled posterior directly by combining per-agent posteriors. Our contributions are as follows:

• We provide a Blackwell-based formalisation of multi-agent LLM aggregation, showing how voting and debating can lose information relative to posterior-level access to agents’ joint private evidence.

• We derive and instantiate MA-PoP as a direct estimator of the pooled posterior, implemented as a product of per-agent posteriors. The estimator is exact under conditionally independent private information and serves as an evidence-accumulation approximation when agents are correlated.

• We evaluate MA-PoP across six multiple-choice QA benchmarks, covering both homogeneous and heterogeneous MAD configurations, and compare against state-of-the-art debate [8], [15] and voting-based aggregation methods [16].

Our theoretical claims are made within the Blackwell information-structure abstraction and are scoped to multiple-choice QA settings with fixed candidate answers.

2 Background↩︎

This section provides the background on the Blackwell’s informativeness framework [14], [17]. To increase the readability, we use calligraphic letters to denote sets, lowercase letters to represent variables, and Greek letters to indicate functions.

Given the likelihood \(\sigma(d | s)\) of the observed data \(d \in \mathcal{D}\) and the prior over states \(\rho(s)\), the posterior of the state can be obtained via Bayes’ rule as \(\Pr(s | d) = \frac{\sigma(d | s) \rho(s)}{\Pr(d)}\), where \(\Pr(d)\) is the prior distribution of observed data \(d\).

Intuitively, the expected value of an information structure can be interpreted as follows:

1. for each observation \(d\), select the action to maximise the expectation of the utility w.r.t. \(\Pr(s | d)\),

2. weight that utility by the probability of observing \(d\),

3. sum (or integrate) over all observations.

Intuitively, garbling introduces a stochastic transformation that adds noise to the original signal distribution. Consequently, the expected utility for any decision problem cannot increase under garbling; instead, it can only stay the same or decrease.

Here, we state the “partial” result (excluding the feasibility) of the Blackwell’s informativeness theorem as follows:

Intuitively, a information structure results in a lower expected utility.

3 Problem statement↩︎

We study multi-agent decision making and classification through the standard Bayesian-game setting [18], in which each Bayesian agent starts from a non-informative prior over states, \(\rho(s)\), assigning equal probability to all possible outcomes. After receiving private information \(d_m\), each agent updates its belief via Bayesian inference, yielding the posterior \(\Pr(s \mid d_m)\). This abstraction closely resembles the multi-LLM setting, where agents share an uninformative prior before training but acquire distinct private information through their training data or observations. 1 maps the decision-theoretic terms used here to their classification counterparts.

r0.5

For classification tasks, let \(y \in \mathcal{Y}\) represent label of an instance \(x \in \mathcal{X}\), and \(d\) denote private information (e.g., private training data) observed by an agent. The prediction made by an agent indexed by \(m\), whose private information (or training data) is denoted as \(d_{m}\), for the instance \(x\) can be written as \(\Pr(y | x, d_{m})\). To relate to the decision-making process in 2, the label of an instance in classification corresponds to the world state in the Blackwell’s informativeness setting (i.e., \(\mathcal{S} \equiv \mathcal{Y}\)), the action as the predicted label outcome of the system, and the utility is the negative loss function. Readers are referred to 1 for further details.

The setting considered in this work is the decision-making with multiple agents in classification, where each agent indexed by \(m\) is trained on a private training dataset \(d_{m}\). The objective is to make an informative decision regarding the label \(y\) of a testing instance \(x\). Two prominent approaches are commonly employed in the literature: (i) majority voting in which agents independently produce predictions that are subsequently aggregated into a final decision, and (ii) debate, where agents exchange their opinions, and update their beliefs accordingly over multiple rounds of interaction.

The objective of this work is to determine whether these two mechanisms, voting and debate, are sufficiently informative under the Blackwell’s informativeness framework, or whether an alternative decision-making mechanism may yield a better informative outcome.

4 Methodology↩︎

This section presents our aggregation result and applies it to classification with multiple LLM agents.

4.1 Information aggregation is a garbling↩︎

We leverage 1 to formally prove that any aggregation on the private information, denoted as \(g \in \mathcal{G} = \{g: g = \gamma(d_{1}, \dots, d_{M})\}\), is a garbling (or “noisy” version) of \((d_{1}, \dots, d_{M})\), where \(\gamma: \mathcal{D}^{M} \to \mathcal{Z}\) is an aggregation function and \(\mathcal{Z}\) is an arbitrary set. To simplify the notations, we denote \(d_{1:M}\) as \((d_{1}, \dots, d_{M})\). The following proposition is a direct consequence of the definition of garbling, but it is the key step that lets us instantiate Blackwell’s ordering for multi-agent aggregation.

Intuitively, 1 shows that no aggregation mechanism can outperform the pooled information structure of all private information under the Blackwell ordering framework. The novelty of the above theoretical results is not the garbling theorem, but its instantiation in multi-agent decision making.

4.2 Decision making in multi-agent classification↩︎

In classification with an ensemble of \(M\) agents, aggregation is often performed in the label space, meaning \(\mathcal{Z} \equiv \mathcal{Y}\). For example, in a weighted averaging the output prediction can be calculated as follows: \[g_{\text{weighted}} = \gamma_{\text{weighted}}(d_{1:M}) = \textstyle \sum_{m = 1}^{M} w_{m} \Pr(y | x, d_{m}), \label{eq:weighted95average}\tag{2}\] where: \(w_{m} \in [0, 1]\) and \(\sum_{m = 1}^{M} w_{m} = 1\). When \(w_{m} = \frac{1}{M}\), this resembles the ensemble learning (also known as single-round-debate voting).

We employ 1 to show that the decision obtained via either aggregated information sources or debate is no more informative under the Blackwell’s framework than the decision made from the information source of pooled original private observations.

This result holds independently of protocol details, equilibria, debate structure, or learning dynamics. We, therefore, intentionally omit the details of communication protocols or equilibrium formation within multi-agent systems. For our purposes, it suffices to observe that common collective-decision procedures, such as voting, or debate, are instances of information aggregation mappings applied to agents’ private signals. Within this simplified framework, the following is an informal proof sketch of the corollary.

As a result, in any classification, every Bayesian decision maker always prefers the information source \((\mathcal{D}^{M}, \sigma_{0}( d_{1:M} | y, x))\) to either the information source obtained via label aggregation or debate. The expected utility value of the information source \((\mathcal{D}^{M}, \sigma_{0}( d_{1:M} | y, x))\) can be written as follows: \[\textstyle \omega(\sigma_{0}) = \sum_{d_{1:M} \in \mathcal{D}^{M}} \max_{\hat{y} \in \mathcal{Y}} \varphi(\hat{y}, y) \sigma_{0}(d_{1:M} | y, x) \rho(y | x),\] where \(\hat{y}\) denotes the predicted label. Since \(\varphi(\hat{y}, y)\) is the negative-loss function (see the interpretation in 1), its maximal value is: \(\varphi^{*} = \varphi(\hat{y} = y, y)\). The expected utility can, therefore, be rewritten as follows: \[\textstyle \omega(\sigma_{0}) = \sum_{d_{1:M} \in \mathcal{D}^{M}} \varphi^{*} \times \max_{y \in \mathcal{Y}} \sigma_{0}(d_{1:M} | y, x) \rho(y | x). \label{eq:expected95utility95classification}\tag{3}\]

3 means that in a multi-agent classification, for any set of private datasets \(d_{1:M} \in \mathcal{D}^{M}\), every Bayesian decision maker prefers having the predicted label as follows: \[\begin{align}[b] \hat{y}^{*} & = \textstyle \operatorname*{argmax}_{y \in \mathcal{Y}} \sigma_{0}(d_{1:M} | y, x) \rho(y | x) \textstyle = \operatorname*{argmax}_{y \in \mathcal{Y}} \Pr(y, d_{1:M} | x) \textstyle = \operatorname*{argmax}_{y \in \mathcal{Y}} \Pr(y | x, d_{1:M}). \end{align} \label{eq:predicted95label}\tag{4}\]

The result in 4 is also known as maximum Bayesian pooled posterior. Within this Blackwell information-structure abstraction, voting and debate are less informative than access to the full pooled private information. Next, we present a practical method to approximate the pooled posterior for decision making in multi-agent classification.

4.2.0.1 Estimate the pooled posterior

If all of the private information observed by each agent, \(d_{m}, m \in \{1, \dots, M\}\) is available, it is straight-forward to obtain \(\Pr(y | x, d_{1:M})\). This is, however, not true in practice because the private information (e.g., private training data) \(d_{m}\) of each agent is often unavailable to other agents \(m^{\prime} \in \{1,\dots,M\} \setminus \{m\}\). Instead, it is only possible to sample the prediction made by each agent’s posterior \(\Pr(y | x, d_{m})\). If this sampling process is reasonably cheap, one can repeatedly sample multiple predictions of one agent to approximate its posterior, denoted as \(\tilde{\Pr}(y | x, d_{m})\). In the following, we assume the availability of \(\tilde{\Pr}(y | x, d_{m}), m \in \{1, \dots, M\}\), and use these to approximate the pooled posterior \(\Pr(y | x, d_{1:M})\).

Given the availability of each agent’s approximated posterior \(\tilde{\Pr}(y | x, d_{m})\), the pooled posterior of interest can then be estimated by applying Bayes’ rule as follows: \[\begin{align}[b] \Pr(y | x, d_{1:M}) &\propto \Pr(y|x) \textstyle\prod_{m=1}^{M} \Pr(d_m|y,x) \quad \text{assume } (d_m \perp d_{m'} | y,x,\;m \neq m') \\ &\propto \Pr(y|x) \textstyle\prod_{m=1}^{M} \frac{\Pr(y|x,d_m)\Pr(d_m)}{\Pr(y|x)} \\ &\propto [\underbrace{\Pr(y|x)}_{\text{uniform}}]^{1-M} \textstyle\prod_{m=1}^{M} \Pr(y|x,d_m) \\ &\propto \textstyle\prod_{m=1}^{M} \Pr(y|x,d_m) \approx \prod_{m=1}^{M} \tilde{\Pr}(y|x,d_m). \end{align} \label{eq:PoP}\tag{5}\] The result in 5 shows that the pooled posterior is approximately proportional to the product of each agent’s posterior (up to a normalisation constant). We adopt two standard assumptions in the information and LLM aggregation literature [16], [19]–[22]: (i) the conditional independence of agents’ private knowledge, and (ii) uniform prior \(\Pr(y | x)\). The conditional independence assumption yields a closed-form approximation of the Bayesian posterior pooling and is particularly appropriate in heterogeneous settings where agents acquire information from distinct sources or specialise in different domains (e.g., software engineering versus biology or law). 5 should therefore not be read as an unconditional optimality guarantee. The pooled posterior in 4 is Blackwell-optimal when the joint private information \(d_{1:M}\) is available, whereas the product estimator in 5 is exact only under conditional independence of agents’ private information. We treat this assumption not as a factual claim about modern LLMs, but as an idealised reference point. As agent dependence increases (e.g., overlapping of pre-training and fine-tuning data), the pooled posterior estimated in 5 is overconfident. Empirically, this overconfidence does not lead to a catastrophic degradation, but a smooth performance changes as observed in 5 6 7.

The assumption of uniform prior \(\Pr(y | x)\) in 5 should be read as the absence of external information about class label distribution. If a prior \(\Pr(y\mid x)\) is available, it can be incorporated directly through the more general form \(\Pr(y\mid x,d_{1:M}) \propto \Pr(y\mid x)^{1-M}\prod_m \Pr(y\mid x,d_m)\).

4.2.0.2 Approximate \(\tilde{\Pr}(y | x, d_{m})\) of LLM agents in multiple-choice questions

We employ the similarity of embeddings between the agent’s response \(y^{(m)} \sim \Pr(y | x, d_{m})\) and each of \(J\) provided answers (or options) \(y_{o_{j}}, j \in \{1, \dots, J\}\) in a multiple-choice question (e.g., \(y_{o_{j}}\) is a natural language expression as: A-benign, B-malignant or C-no finding as in 11) to approximate \(\tilde{\Pr}(y | x, d_{m})\). Formally, for each pair of agent’s response \(y^{(m)}_{n}\) and a provided list of options, \(y_{o_{j}}\), we calculate the similarity of their embeddings, denoted as \(\tau(y^{(m)}_{n}, y_{o_{j}})\). For example, \(\tau\) can be the dot product or cosine similarity between the two embedding vectors of \(y^{(m)}_{n}\) and \(y_{o_{j}}\). The predictive probability distribution over all provided options can be approximated as follows: \[\textstyle \tilde{\Pr}(y = y_{{o}_{j}} | x, d_{m}) = \frac{1}{N} \sum_{n = 1}^{N} \frac{\phi \left( \tau(y^{(m)}_{n}, y_{o_{j}}) \right)}{\sum_{j' = 1}^{J} \phi \left( \tau(y^{(m)}_{n}, y_{o_{j'}}) \right)}, \label{eq:approximate95posterior}\tag{6}\] where \(N\) is the number of responses for the same question, and the additional model \(\phi(\cdot)\) is introduced to map the similarity score to a predictive logit (e.g., plays a role as Platt scaling [23]). Note that we estimate each agent’s posterior from full generated responses by measuring their semantic compatibility with each candidate option using an NLI cross-encoder, followed by calibration. This avoids relying on raw LLM logits, whose token-level probabilities over labels or answer strings can be noisy, length-biased, and misaligned with full-answer semantics; see Appendix 9 for details.

Given the approximation of each agent’s posterior in 6 , we can approximate the pooled posterior \(\Pr(y | x, d_{1:M})\) by multiplying these probabilities and normalising to obtain the predicted label (or the selected answer) as shown in 4 .

Note that in the conventional multi-class classification, the order of appearance of class labels is fixed. However, in the setting of multiple choice questions, the or provided options can be freely permutated. Hence, the model \(\phi(\cdot)\) used for calibrating the prediction probabilities in 6 must be permutation-equivariant. In our implementation, we formulate \(\phi\) as a fully-connected neural network following the Deep Sets architecture [24] to make the prediction permutation-equivariant.

5 Experiments↩︎

5.0.0.1 Baselines

We follow the setting in [8] to evaluate MA-PoP compared to majority voting and MAD. The MAD approaches include: Centralised MAD [2], where a central agent aggregates peer responses and generates the updated response at each round, Decentralised MAD [6], where each agent observes all other agents’ responses from the previous round, Sparse MAD [25]–a variant of Decentralised MAD with a sparse communication topology to enhance efficiency, and Free MAD [15], where decisions are made by evaluating the entire trajectory rather than the final consensus. The voting-based approach includes: majority voting, Self-Consistency (SC) [9], which samples \(N\) responses from each agent and selects the most frequent answer across all agents, Log-linear opinion pooling (LLP) [22], which relaxes the conditional independence assumption by assigning learnable per-agent weights, and Inverse Surprising Popularity (ISP) [16], which leverages first-order information (e.g., agent accuracies) and second-order information (e.g., answer correlations) to aggregating LLMs’ responses. Each agent is sampled with the same number of responses, \(N\), to fairly compare.

We consider a pool of \(M = 5\) agents for multi-agent methods in our main comparison and will ablate on the effect of \(M\). For single-agent baselines, we average across five independent runs. Qwen2.5-7B-Instruct [26], Llama-3.1-8B-Instruct [27], Mistral-7B-Instruct-v0.3 [28], Falcon-H1-7B-Instruct [29], and Gemma-2-9B-Instruct [30] are used for heterogeneous pool. In the homogeneous setting (i.e., all agents are instances of the same LLM), MA-PoP is simplified to maximising the posterior of a single agent, while in the heterogeneous setting, MA-PoP is carried out following 1 in 12. We also keep the same number of Monte Carlo samples \(N = 5\) throughout all the experiments. For all MAD baselines, we report results with \(T \in \{2, 3, 5\}\) debate rounds. We treat the heterogeneous-agent experiments as the primary test of multi-agent information pooling, and the homogeneous-agent experiments as a test of the per-agent posterior estimation pipeline (6 ).

5.0.0.2 Benchmarks

consist of six natural language question answering tasks, each associated with the following datasets: (i) Factual Question Answering (MMLU Professional Medicine and Formal Logics [11], [31]), (ii) Natural Language Inference (HellaSwag [32]), (iii) Commonsense Reasoning (CSQA [33]), (iv) Alignment Labeling (HH-RLHF [34]), where we adopt the “AI labeller alignment” practice [35], similarly to [8], and (v) Medical Entrance Exam (MedMCQA) [36]. For a fair comparison, all baselines are evaluated on the same data subsets. Readers are referred to 10 for further details of the datasets used.

5.0.0.3 Results on standard MAD benchmarks

are shown in 2 with five different LLMs across five benchmarks, while 2 in 13 shows the gap among the best single model/ SOTA MAD, majority voting, Self-Consistency, LLP, ISP and MA-PoP. Empirical results demonstrate that multi-agent methods consistently improve performance over single-agent baselines across most datasets. However, the degree of improvement varies significantly by task, highlighting the sensitivity of these methods to the underlying task complexity.

MA-PoP achieves the highest performance across all evaluated tasks. Although majority voting has strong performance in homogeneous agent settings [8], it is not on par with MAD methods in heterogeneous settings and sometimes even underperforms individual agents (e.g., see HellaSwag in 2). Furthermore, other debate methods show inconsistent performance across different numbers of rounds (\(T=2, 3, 5\)). On some datasets like HellaSwag and HH-RLHF, additional debate rounds provide marginal gains or even degrades performance compared to naive voting. These findings empirically validates recent critiques [8], [37] regarding the diminishing returns of iterative debate compared to voting (or single-turn ensembles). Notably, on HellaSwag, where traditional debate methods often underperform the best single agent, MA-PoP successfully leverages complementary knowledge of multiple agents, exceeding single-agent performance. This demonstrates that the limitations of conversational debate stem not from multi-agent collaboration itself, but from suboptimal aggregation strategies.

Across different debate rounds, traditional MAD methods show diminishing or even negative returns as the debate progresses. This is consistent with the broader finding of [13] that LLMs exhibit significantly lower performance in multi-turn conversations than single-turn. Multi-turn debate inherits the same conversational-degradation pathologies. In contrast, MA-PoP achieves its superior results in a single turn by directly aggregating the initial probabilistic beliefs, providing a more computationally efficient and stable alternative to iterative communication.

The LLP baseline provides a dependence-aware reference point: under conditional independence, MA-PoP corresponds to a log-linear pool with all per-agent weights fixed to one, whereas LLP learns these weights from a small validation set (about 300 pooled examples, see 10). MA-PoP’s gains over LLP should not be read as evidence that conditional independence holds; rather, in the low-supervision regime considered here, fixed unit weights provide a useful inductive bias relative to learning global per-agent weights from limited data.

5.0.0.4 Heterogeneous and Homogeneous Pool of LLMs

To assess robustness to dependence, we evaluate MA-PoP across a spectrum of agent-diversity regimes: heterogeneous agents, reduced heterogeneous pools, homogeneous agents, and larger models with likely overlapping training data. 3 demonstrates that MA-PoP maintains strong performance even with reduced agent diversity, demonstrating its robustness to ensemble size (see 14). However, we also observe that increasing the number of agents beyond a certain threshold does not yield proportional performance gains across all datasets. This limitation stems from the conditional independence assumption on agents’ private training data imposed in 5 . While introducing additional agents can initially enrich the collective knowledge, thereby improving performance, this effect diminishes once the pool of agents begins to contribute overlapping information rather than genuinely novel evidence. Beyond this point, adding more agents no longer enhances informativeness and instead leads to a performance plateau.

4 in 14 presents results at the extreme setting consisting of only 2 agents (Qwen-7B and Llama-8B). This is noteworthy because majority voting fundamentally fails when two agents disagree. In that case, majority voting randomly selects one out of the two answers, resulting in an unreliable decision. In contrast, MA-PoP can, however, leverage the strength of each agent’s posterior distribution to make informed decisions without the need of consensus. Empirically, MA-PoP performs competitively and achieves the best results on most benchmarks in this setting, suggesting that posterior-level aggregation can remain useful even when majority voting becomes unreliable.

Following [8], 6 5 in 15 evaluate MA-PoP in the homogeneous setting (e.g., all 5 agents are instances of the same LLM). In this setting, the pooled posterior in MA-PoP is simplified into a single posterior because all these agents are the same. Nevertheless, MA-PoP still outperforms both single-agent baselines and traditional MAD methods. Notably, Centralised MAD exhibits severe degradation with additional rounds of debate, highlighting the risks of iterative debate when agents share the same model and may amplify shared biases [12]. Across these heterogeneous and homogeneous regimes, performance gains decrease as redundancy increases, but do not collapse.

5.0.0.5 Larger and more capable models

We extend the evaluation by employing LLMs with larger capacity. Specifically, for the homogeneous setting, Qwen2.5-32B-Instruct [38] is used as a base model. As shown in 7 (see 16), the performance of majority voting remains comparable to that of multi-agent methods. For the heterogeneous setting, Qwen2.5-32B-Instruct [26], Falcon-H1-34B-Instruct [29], and Gemma-2-27B-Instruct [30] are used for 3-agent debate. The results in 8 (see 16) for Centralised MAD agree with our previous evaluation in 2 3, in which Centralised MAD cannot improve performance and is even worse than single models and majority voting. Other MAD methods perform better than single agent but are inferior compared to our proposed method MA-PoP.

5.0.0.6 Calibration of MA-PoP

We evaluate calibration quality by measuring Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) in 9, with corresponding reliability diagrams shown in 3. We test four models (Qwen-7B, Falcon-7B, Gemma-9B, and Falcon-34B) on the MedMCQA dataset (see 17), comparing performance with and without our calibration module. Results demonstrate that our method achieves substantial calibration improvements across different model scales, producing well-calibrated probability estimates that enable effective multi-agent consensus mechanisms.

5.0.0.7 Ablation of MA-PoP

We isolate each component of MA-PoP through three ablations against the full method (10 in 18). (1) w/o PoP replaces cross-agent pooling with a single best agent’s calibrated posterior, averaged over \(N=5\) Monte Carlo samples (6 ). The results show that calibration alone yields part of the gain over the single-agent baseline, while cross-agent pooling provides additional improvements across most benchmarks. (2) w/o calibration replaces the Deep Sets with softmax-normalised NLI logits. Multiplying these uncalibrated scores yields only marginal gains, which confirms that well-formed probabilities are a prerequisite for multiplicative aggregation. (3) Temperature scaling replaces the Deep Sets with a single learned scalar temperature [23], [39]. This improves over no calibration on some benchmarks (e.g., MMLU Pro.Med., CSQA) but underperforms on others (HellaSwag, MMLU Formal Logic) and consistently falls short of the MA-PoP, suggesting that a scalar temperature is insufficient to capture the permutation-equivariant structure of multiple-choice options. Overall, these results indicate that pooling and calibration are complementary, with pooling providing the larger contribution and calibration being necessary to realise it.

5.0.0.8 Number of Monte Carlo samples

We analyse the impact of the number of Monte Carlo samples, denoted by \(N\), on the quality of posterior estimation in MA-PoP. The results in 4 (see 19) for the homogeneous setting show substantial improvement when increasing from \(N = 1\) to \(N = 5\) samples, with performance stabilising beyond \(N \ge 5\). This demonstrates that posterior estimates converge with additional samples, though with diminishing marginal returns, suggesting \(N = 5\) provides a good balance between estimation quality and computational cost.

5.0.0.9 Efficiency

We evaluate computational cost by measuring the input, output, and total number of tokens consumed per sample across all methods on the MedMCQA dataset in the 5-agent heterogeneous setting. 11 in 20 reports token usage accumulated across all debate rounds. MA-PoP achieves token efficiency comparable to single-round voting baselines and substantially lower than MAD methods, particularly at higher debate rounds. The additional NLI cross-encoder scoring step required by MA-PoP runs in under one second per sample on a single GPU and is negligible relative to LLM generation cost, which is about 25 seconds.

6 Related work↩︎

6.0.0.1 Multi-agent debate (MAD)

enables agents to iteratively exchange arguments and critiques over multiple rounds, with the aim of improving reasoning through deliberation and mutual correction [6], [40], [41]. While debate has shown potential for exposing diverse perspectives and correcting individual errors [42], [43], recent empirical studies highlight significant limitations: multi-round debate often fails to outperform strong single-agent baselines and can be less reliable than simple majority voting [8], [12]. Free-MAD [15] proposes a consensus-free approach where decisions are made by evaluating the entire debate trajectory rather than the final consensus. Within the Blackwell information-structure abstraction, existing voting-based and debate-oriented approaches are less informative than Bayesian pooled posterior, as formalised in [corollary:optimal95informativeness]. In contrast, we ground multi-agent decision making in Blackwell’s informativeness framework to generate decisions with informative-theoretic guarantees.

6.0.0.2 Opinion Pooling and Voting

Opinion pooling concerns the aggregation of probabilistic beliefs from multiple experts into a single opinion [44]. Early work includes the DeGroot [45], in which agents iteratively update their beliefs via weighted linear combinations, corresponding to linear opinion pooling (e.g., 2 ). Beyond linear pooling, log-linear pooling (LLP) has been proposed [46] to account for dependencies among agents’ beliefs. LLP assigns each agent \(m\) a non-negative weight \(w_{m} \ge 0\), yielding a pooled belief of the form \(\prod_{m = 1}^{M} [\Pr(y | x, d_{m})]^{w_{m}}\). Log-linear pooling has also been studied in the context of incoherent opinion pooling through renormalisation procedures [22].

Voting in an ensemble of LLMs instantiates linear opinion pooling, with majority voting as the most common aggregation rule. To reduce uncertainty, self-consistency selects the most frequent answer across multiple sampled prompts or reasoning paths from a single model [9], [47]. Beyond simple voting, more advanced schemes include mixture-of-experts [48], pairwise ranking-based aggregation [49], confidence-weighted voting [50], and hybrids that combine output agreement with internal representation consistency [51]. Recent work such as Inverse Surprising Popularity [16] further improves aggregation by leveraging agent accuracies (i.e., first-order) and inter-answer (i.e, second-order) correlations.

21 provides further related work, including multi-agent decision-making, LLM uncertainty estimation and product of experts [52].

7 Discussion and conclusion↩︎

In this paper, we study the information generated by voting and debate in multi-agent decision-making. We show that, under Blackwell’s informativeness framework, decisions based on these sources are provably no more informative than those based on pooled information. This insight motivates Bayesian pooled-posterior maximisation as an information-theoretic reference rule. MA-PoP provides a practical approximation to this rule by multiplying estimated individual posteriors, a form that is exact under conditional independence and appropriate when agents contribute diverse, weakly correlated evidence.

Building on this, we propose MA-PoP, a practical method for approximating agent posteriors in LLM-based settings. MA-PoP consistently outperforms prior state-of-the-art methods on standard debate benchmarks. A key limitation of this work is that MA-PoP’s product estimator relies on conditional independence of agents’ private information; in realistic LLM deployments, overlapping pre-training, fine-tuning, and toolchains can violate this assumption and lead to overconfident pooled posteriors, as discussed in Appendix 22. In fact, the empirical gains of MA-PoP decrease as agent redundancy increases, but the degradation is gradual, suggesting that the method remains useful as an approximation when independence is only partially satisfied.

8 Proof of [corollary:optimal95informativeness]↩︎

9 Logits vs. Semantic Embedding for posterior probability estimation↩︎

While using LLM logits directly is an intuitive alternative, there are two limitations that make it not robust enough for reliable posterior estimation in multi-agent setting:

Multi-Token Label Ambiguity. LLM logits correspond to the next-token probability conditioned on the prompt, not the probability of an entire candidate answer. When candidate labels span multiple tokens, the first-token logit is only a partial signal. For example, in HellaSwag, given the context "A man is sitting on a roof. he", the candidate answers are ["is using wrap to wrap a pair of skis.", "is ripping level tiles off.", "is holding a rubik’s cube.", "starts pulling up roofing on a roof."]. Computing the full joint probability would require autoregressive decoding over every candidate, which is computationally expensive. It will also require to normalise probabilities by answer length, but this does not provide true probabilities (it effectively measures average per-token probability) and can bias results toward longer answers that contain more common (i.e., higher-probability) words.

Signal vs. Noise. Even for single-token labels (e.g., A/B/C/D in multiple-choice), the LLM’s probability mass is distributed across the entire vocabulary, not just the valid candidates. Consider a typical scenario where the first-token distribution assigns overwhelming mass to a task-irrelevant token: the probability of “The”, “answer”, “is”, “A”, “B”, “C”, “D”] is [0.9, 0.03, 0.03, 0.01, 0.01, 0.01, 0.01]. After masking to valid candidates and renormalising, we obtain a uniform distribution [0.25, 0.25, 0.25, 0.25] that has lost all discriminative signal. LLMs frequently allocate dominant probability mass to high-frequency tokens that are irrelevant to the task, leaving the valid-label logits in a low-signal regime where noise dominates.

Instead, our method captures the semantics of the complete answer rather than relying on a single token and operates in a continuous representation space, which is more robust than using logits directly.

10 Experimental Details↩︎

10.1 Hyperparameters↩︎

Following [8], we set the sampling temperature to 1.0, and use nucleus sampling probability of 0.9 to enable stochastic sampling from homogeneous agents. The maximum tokens for all models is 512.

10.2 Dataset↩︎

MMLU [31] consists of 13, 869 multiple-choice questions ([11] use the 3,406 high-school-level questions). MMLU (Formal Logic) is designed to evaluate a model’s proficiency in formal reasoning, symbolic manipulation, and logical analysis. Following [8], we use the entire test split comprised of 126 question for evaluation. MMLU (Professional Medicine) is a benchmark designed to evaluate professional-level reasoning in medical domains. It requires knowledge of medical concepts, clinical reasoning, and biomedical science to answer its questions. Following [8], [53], we use the full test split, which contains 272 items.

HellaSwag [32] is a natural language inference (NLI) benchmark dataset focused on sentence completion. It evaluates whether a model can select the most plausible continuation of a given context from multiple candidates, a task requiring both linguistic competence and commonsense reasoning. From the original test split, We follows the setting in [53] which randomly sample 300 questions for our evaluations.

CommonsenseQA [33] is a multiple-choice question answering dataset designed to evaluate a model’s ability to apply commonsense knowledge in natural language understanding. Following [8], we randomly sample 300 questions from the original test split.

HH-RLHF [34] is a collection of human-annotated data designed to train and evaluate language models for alignment with human preferences, focusing on helpfulness and harmlessness. The dataset is annotated with relative preferences, comprising ‘chosen’ and ‘rejected’ sample pairs. Similar to the “AI labeler alignment” practice [35], [8] ask the LLM agent to select the example that is more helpful and less harmful. To avoid selection bias, randomly shuffle the order of “chosen" and”rejected" in the input prompt. We follows [8] and use a random subset of 300 pairs from the original test split.

MedMCQA [36] is a large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. We randomly sample 300 questions from the original test split.

10.2.0.1 Implementation details

All methods are implemented in PyTorch [54] and run on NVIDIA GPU RTX A6000 and A100. For permutation-equivariant model \(\Phi\) in 6 , we select a Cross-Encoder Natural Language Inference model nli-deberta-v3-large [55] and use the entailment logit as the similarity score.

Because the amount of available training data varies across datasets, we pool samples from all sources to train a single permutation-equivariant model. Specifically, we collect 14 samples from the MMLU Formal Logic validation set, 31 samples from the MMLU Professional Medicine validation set, and 50 samples each from the training or validation splits of the remaining datasets, without any overlapping to the testing sets. All inputs are padded to length 5, the maximum number of options across datasets. The calibration model \(\Phi\) in 6 is implemented as a Deep Sets architecture [24], a fully-connected network with two hidden layers of 32 and 64 units respectively, each followed by a ReLU activation. It is trained for 100 epochs with cross-entropy loss using SGD (momentum of 0.9, and weight decay of \(5\times10^{-4}\)), a batch size of 256, and an initial learning rate of 0.01. The log-linear opinion pooling (LLP) weights and temperature scaling in 10 are trained with a two-layer MLP using Adam (learning rate 0.01) and cross-entropy loss on the same training set as the permutation-equivariant model. For a fair comparison, LLP shares the calibration head with MA-PoP.

11 Prompt Templates↩︎

11.1 MAD Templates↩︎

Following setting in [8], we provide the prompt template for multi-agent debate.

11.2 Task Templates↩︎

Following setting in [8], we provide the exact input format used for each dataset. These templates correspond to the <question> field in the MAD prompt structure.

One example of HellaSwag dataset is as below:

11.3 Embedding Option Templates↩︎

We provide the exact option format used for CommonsenseQA dataset for similarity estimation. The score is calculated by estimating the similarity between the agent output and those option formats.

12 Algorithm of MA-PoP↩︎

13 The gap among the best single model/ SOTA MAD, aggregation methods and MA-PoP↩︎

2 shows the gap among the best single model/ SOTA MAD, majority voting and MA-PoP. Empirical results demonstrate multi-agent methods consistently improve performance over single-agent baselines across most datasets.

14 Number of LLM agents in heterogeneous setting↩︎

3 shows results with 4 agents (Qwen-7B, Falcon-7B, Gemma-9B and Llama-8B). 4 presents results at the extreme setting consisting of only 2 agents (Qwen-7B and Llama-8B), which is a setting where majority voting fundamentally fails when two agents disagree.

15 Homogeneous pool of LLMs↩︎

Following [8], 6 5 evaluate MA-PoP in the homogeneous setting (e.g., all 5 agents are instances of the same LLM). In this setting, the pooled posterior in MA-PoP is simplified into a single posterior because all these agents are the same. For MAD and Majority Voting, the reported results are from [8].

16 Larger and more capable models↩︎

For the homogeneous setting, Qwen-32B is used as a base model. As shown in 7, the performance of majority voting remains comparable to that of multi-agent methods. For MAD and Majority Voting, the reported results are from [8]. For the heterogeneous setting, Qwen2.5-32B-Instruct [26], Falcon-H1-34B-Instruct [29], and Gemma-2-27B-Instruct [30] are used for 3-agent debate.

17 Calibration of MA-PoP↩︎

We measure the Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) in 9 and plot the reliability diagram (calibration curve) in 3 across 4 models (Qwen-7B, Falcon-7B, Gemma-9B, and Falcon-34B) on MedMCQA dataset.

18 Ablation results of MA-PoP↩︎

We conduct three ablations to isolate the contribution of each component of MA-PoP in 10: (1) w/o PoP: a single best agent whose posterior is estimated by averaging calibrated NLI scores across \(N=5\) Monte Carlo samples (6 ), without the cross-agent product in 5 . (2) w/o calibration: MA-PoP using softmax-normalised NLI logits in place of the calibrated posterior estimate. (3) Temperature scaling: MA-PoP in which the Deep Sets calibration head is replaced by temperature scaling [23], [39], which learns a single scalar temperature to rescale the logits via softmax. Overall, these results indicate that pooling and calibration are complementary, with pooling providing substantial additional gains and calibration being necessary to realise them. The w/o calibration condition shows that multiplying uncalibrated scores yields only marginal gains, confirming that well-formed probabilities are a prerequisite for multiplicative aggregation. Temperature scaling improves performance on some benchmarks (e.g., MMLU Pro. Med., CSQA) but underperforms on others (HellaSwag, MMLU Form. Log.), and consistently falls short of the full MA-PoP. This suggests that a single scalar parameter is insufficient to capture the permutation-equivariant structure of multiple-choice options. These results indicate that calibration and posterior pooling are complementary, with pooling providing the larger contribution and calibration being necessary to realise it.

19 Number of Monte Carlo samples↩︎

We analyse the impact of the number of Monte Carlo samples, denoted by \(N\), on the quality of posterior estimation in MA-PoP. The results in 4 for the homogeneous setting show substantial improvement when increasing from \(N = 1\) to \(N = 5\) samples, with performance stabilising beyond \(N \ge 5\).

20 Computational Efficiency Analysis↩︎

We evaluate the computational cost by measuring input, output, and the total number of tokens consumed per sample across all methods on the MedMCQA dataset in the heterogeneous setting of 5 agents. 11 presents token usage statistics accumulated across all debate rounds. MA-PoP demonstrates competitive efficiency compared to MAD approaches. The running time of NLI cross-encoder is less than 1 second per sample, which is negligible compared to the time needed for the whole inference, which is about 25 seconds.

21 Extend related work↩︎

21.0.0.1 Multi-agent decision making

dates back to the 1970s, originating in decision support systems, and has since evolved into complex multi-agent systems [56]. The literature spans a diverse range of techniques, including rule-based systems [57], [58], game-theoretic methods [59], [60], reinforcement learning [61]–[64], and, more recently, LLM-based agents, each with distinct trade-offs. Rule-based methods remain interpretable but brittle and hard to scale or optimise. Game‑theoretic approaches offer theoretical guarantees at equilibrium, but their integration into real‑world pipelines is hindered by impractical modelling assumptions (e.g., requiring an infinite number of rounds to reach equilibrium). Reinforcement learning has delivered remarkable results, but training is data- and compute-intensive, and performance largely depends on reward specification. LLM‑based agents demonstrate strong performance due to their reasoning capabilities; however, current decision‑making aggregation remains ad‑hoc, typically relying on majority voting or debate and lacking formal guarantees on decision quality. In contrast, we ground multi-agent decision making in Blackwell’s informativeness framework to generate decisions with information-theoretic guarantees.

21.0.0.2 LLM uncertainty estimation

seeks to quantify an LLM agent’s probabilistic belief in a given context. A straightforward approach is to elicit verbalised confidence through natural-language prompts that explicitly ask the model to report its confidence level [50], [65]–[67]. However, such self-reported confidence often fails to correspond to the model’s true underlying belief state [68]. An alternative line of work employs sample-based estimation, where uncertainty is inferred from variability across multiple stochastic model outputs [69]. The practical instantiation of our method, presented in 4, follows this sample-based direction.

21.0.0.3 Product of Experts

(PoE) [52] can be seen as an inspiration to our work, but it is fundamentally different from MA-PoP. In particular, PoE is primarily a modelling and training paradigm that combines multiple expert likelihoods without offering decision-theoretic optimality guarantees. In contrast, MA-PoP is derived from a decision-theoretic perspective: it approximates Bayesian posterior pooling over agents’ private information and is Blackwell-optimal in the independent-private-information limit. In realistic LLM settings, where agents may be correlated, MA-PoP should instead be viewed as an approximation to posterior-level evidence aggregation.

22 Limitations and Future Works↩︎

Despite promising empirical results, our methodology has an intrinsic limitation: it relies on the assumption of conditional independence among agents’ private observations or private training datasets (see 5 ). In practice, this assumption implicitly requires sufficient diversity of agents’ private information sources. When diversity is present, conditional independence serves as a tractable approximation that facilitates posterior factorisation and enables a practical estimator of the pooled posterior.

The conditional independence assumption is, however, unlikely to hold in practical LLM deployments. Agents’ private knowledge, spanning pre-training data, fine-tuning procedures, and toolchains, typically overlaps. This leads to overconfidence when using PoP in 5 and, in turn, misleads downstream decision-makers. As future work to mitigate this limitation, we propose leveraging open-source and open-weight models to systematically characterise inter-agent dependencies and quantify their impact on pooled posterior.

References↩︎

---

1. Corresponding authors↩︎