1 Introduction↩︎

Multi-agent systems built on top of large language models (LLMs) have rapidly become a workhorse for complex reasoning, planning, code generation, scientific question answering, and tool orchestration [1]–[4]. In the canonical architecture, several specialised LLM agents — each typically instantiated as a separate model call with its own role prompt — collaborate by exchanging natural language messages. A planner proposes a strategy in text; a critic reads the proposal and replies in text; a coder edits the plan in text; and so on. The result is a visible, inspectable, human-readable communication trace that doubles as an audit log and a debugging surface. The way such a system partitions a complex task across agents — which subtask to assign to which agent — is itself a non-trivial design choice, and recent work has begun to study adaptive task-decomposition strategies empirically [5].

Despite its success, the text-only communication protocol is being increasingly questioned. Three structural limitations stand out:

1. Inference cost. Every message forces the sender to decode its internal reasoning into a token sequence, and forces the receiver to re-encode that sequence back into a representation. For an \(L\)-layer model with vocabulary size \(V\) and a message of \(T\) tokens, the per-message overhead is \(\mathcal{O}(L \cdot T \cdot d)\) extra FLOPs on top of the agent’s own reasoning.

2. Information loss during discretization. The sender’s hidden state — a high-dimensional vector that summarises its entire context — must be compressed into a single token drawn from a vocabulary of size \(V\). The mutual information between the hidden state and the chosen token is bounded by \(\log_2 V\) bits, typically \(\le 15\) bits in modern tokenisers, whereas the hidden state itself carries tens of thousands of bits. Alternative reasoning paths, calibrated confidences over alternatives, and fine-grained semantic distinctions are simply discarded.

3. Redundancy and ambiguity of natural language. Generated text is optimised for linguistic fluency rather than task-relevant information density. Idioms, hedging, and vague referents add overhead; disagreements about role assignment or background knowledge can render entire messages irrecoverable.

In response, a new line of work — collectively called latent communication — has emerged. The core idea is to let agents exchange their continuous internal representations directly: embeddings at the input layer, hidden states from intermediate layers, or key–value (KV) caches from the attention mechanism. By skipping the language bottleneck, latent communication can preserve more information, save inference time, and avoid the failure modes of natural language. The downside is interpretability: the channel is opaque to humans and harder to inspect, debug, or align.

The field has grown explosively. The accompanying repository Awesome-Latent-Communication already tracks more than fifteen distinct methods, and the diversity of design choices is striking: some methods transmit embeddings, others transmit hidden states, still others transmit KV-caches. Some methods align the last layer of the sender to the first layer of the receiver; others align all layers. Some fuse information by concatenation; others by prepending, addition, or learned cross-attention. Some are training-free; others require distillation. A new researcher entering the area is therefore confronted by a fragmented landscape with no shared vocabulary.

1.0.0.1 Contributions.

This paper introduces a unified framework that organises the literature along three orthogonal axes and uses it to systematically categorise eighteen representative works. Specifically:

• We propose a 3-axis decomposition — WHAT (types of communicated information), WHICH (sender–receiver alignment), and HOW (information fusion strategy) — that uniquely determines the design space of any latent communication protocol.

• Under this framework, we analyse eighteen methods published between 2024 and 2026, summarise their key innovations, strengths, and limitations, and slot each into a unified comparison table.

• We extract five generalisable takeaways about the design trade-offs (e.g., “KV-cache carries more information than hidden states but is more architecture-dependent”) that we believe will inform future method design.

• We identify six open problems — including cross-architecture alignment, security of latent channels, and the unification of latent communication with latent chain-of-thought — that we expect to shape the next generation of research.

1.0.0.2 Organisation.

The remainder of the paper is organised as follows. 2 introduces preliminary concepts. 3 makes the case for latent communication by quantifying the limitations of natural language. 4 presents the unified framework along the WHAT / WHICH / HOW axes. 5 walks through the eighteen representative methods under the framework. 6 discusses the dominant training-free implementation paradigm. 7 surveys empirical results. 8 lays out open problems. 9 relates latent communication to adjacent research areas. 10 concludes.

2 Background and Preliminaries↩︎

This section fixes the notation and terminology used throughout the paper.

2.1 Multi-Agent LLM Systems↩︎

A multi-agent LLM system (MAS) consists of \(N\) LLM agents \(\mathcal{A} = \{A_1, A_2, \ldots, A_N\}\), each equipped with a role-specific system prompt, optional tool access, and a communication channel. At each step, an agent \(A_i\) (the sender) produces a message that is delivered to one or more peer agents (the receivers). A controller — explicit or implicit — decides the order of speakers. The communication channel is the focus of this paper: classical systems use a natural language channel (2.2); the methods surveyed in this paper use a latent channel ([sec:prelim:latent]).

2.2 Natural Language vs.Latent Communication↩︎

• Natural Language Communication (NL-Comm). The sender generates a discrete token sequence \(y = (y_1, y_2, \ldots, y_T)\) by sampling from a vocabulary \(\mathcal{V}\). The receiver re-encodes the token sequence into its own embedding space. The two-step pipeline — sender decode \(\rightarrow\) token transport \(\rightarrow\) receiver encode — is what we refer to as the language bottleneck.

• Latent Communication (Latent-Comm). The sender exposes one of its internal continuous representations — the input embedding, the hidden state of a particular layer/token, or the KV-cache — and the receiver injects this representation into its own computation without round-tripping through the vocabulary.

A high-level comparison of the two pipelines is shown in 1.

2.3 Prefill and Decode↩︎

LLM inference is split into two phases that we will repeatedly refer to:

• Prefill phase. Given a prompt \(x = (x_1, \ldots, x_T)\), the model processes the entire sequence in parallel and produces the first output token. All key–value pairs computed during prefill are stored in the KV-cache.

• Decode phase. The model generates one token at a time. At each step \(t > T\), it takes the previously generated token \(y_{t-1}\) and the cached KV from earlier steps, and produces a new token \(y_t\) (and a new KV entry).

The distinction matters for latent communication because the kind of internal state available differs between the two phases. During prefill, the sender has access to per-token hidden states and KV-caches for every input token. During decode, the sender has only the hidden state of the most recently generated token plus an ever-growing KV-cache.

2.4 Embedding, Hidden State, KV-Cache, Activation↩︎

We adopt the following precise definitions, which the rest of the paper relies on:

A schematic of these quantities in a Transformer block is included in the left panel of 1.

2.5 Why Now?↩︎

Latent communication has become practical only recently. Three enabling trends converged around 2023–2024:

1. Open-weight LLMs at scale. Methods that pipe a sender’s hidden state into a receiver’s forward pass require white-box access to both models. The release of Llama, Qwen, Mistral, and similar families has made such access routine.

2. KV-cache engineering. The KV-cache has gone from an implementation detail to a first-class optimisation target, with rich infrastructure for compression, sharing, and off-loading. Methods that transmit KV-caches piggy-back on this infrastructure.

3. Multi-agent frameworks. Frameworks like LangGraph, AutoGen, CrewAI, and MetaGPT have lowered the cost of orchestrating multiple LLM agents, making the latent channel itself a meaningful object of study rather than a curiosity.

3 The Case for Latent Communication↩︎

Before diving into the framework, we articulate the case for and against latent communication. We argue that the trade-off is context-dependent: latent communication is preferable when (a) the agents are tightly coupled, (b) the cost of natural language overhead dominates, and (c) the channel can be made interpretable enough for downstream debugging.

3.1 Limitations of Natural Language Communication↩︎

3.1.1 High Inference Cost↩︎

Consider a two-agent system where agent \(A_1\) produces a \(T\)-token message to agent \(A_2\). The total cost is:

• \(A_1\)’s decode of \(T\) tokens: \(\mathcal{O}(L \cdot T \cdot d)\) FLOPs, where \(L\) is the number of layers and \(d\) is the hidden dimension. The KV-cache read/write is the dominant term.

• \(A_2\)’s re-encoding of \(T\) tokens: the same \(\mathcal{O}(L \cdot T \cdot d)\) FLOPs in prefill.

• The token-by-token transport itself: negligible.

So the overhead of natural language communication is roughly \(2 \times\) the cost of generating the message, even before accounting for \(A_2\)’s own reasoning. Latent communication can reduce this to a single embedding/hidden-state/KV-cache transport that the receiver injects without re-encoding.

3.1.2 Information Loss During Discretization↩︎

The pipeline is \[\mathbf{h}_{\text{context}} \xrightarrow{\text{linear}} \mathbf{z} \in \mathbb{R}^{V} \xrightarrow{\text{sample}} y \in \mathcal{V}, \label{eq:discretisation}\tag{1}\] where \(\mathbf{h}_{\text{context}}\) is the sender’s high-dimensional hidden state, \(\mathbf{z}\) is the logit vector over the vocabulary, and \(y\) is the sampled token. The mutual information \(I(\mathbf{h}_{\text{context}}; y)\) is upper-bounded by \(H(y) \le \log_2 |\mathcal{V}| \approx 15\text{--}17\) bits. Meanwhile, \(\mathbf{h}_{\text{context}}\) itself typically lives in \(\mathbb{R}^d\) with \(d \ge 4096\) and is parameterised by 32-bit floats, so its raw representational capacity exceeds \(40{,}000\) bits. The compression factor is therefore on the order of \(10^{3}\)–\(10^{4}\).

Concretely: a hidden state encodes not just which token to say next, but also the alternatives considered, their relative probabilities, the salience of different parts of the context, and uncertainty. All of this is lost the moment we sample a single token. A visual comparison of these information densities (2 (a)) and the resulting communication pipelines (2 (b)) is given in 2.

3.1.3 Redundancy and Ambiguity of Natural Language↩︎

Generated text is optimised for linguistic coherence (a stylistic objective from pre-training) rather than for task-relevant information density. Sentences are padded with politeness markers, hedging, and reformulation. References to prior context are often under-specified (“the previous step”, “that approach”), forcing the receiver to reconstruct the referent.

When sender and receiver disagree on background knowledge, role assignment, or terminology, the natural language channel can become lossy in a semantic sense that goes beyond the numerical bits/token argument. In contrast, latent channels operate on the agents’ own representational manifolds and avoid this kind of semantic mismatch — at the cost of interpretability.

3.2 Advantages of Natural Language Communication↩︎

Latent communication is not a universal replacement. Natural language retains one decisive advantage:

• High interpretability. A natural language message is immediately readable by humans. This is essential for debugging, alignment auditing, safety review, and human–AI interaction. Latent messages, in contrast, are opaque vectors that require auxiliary tooling to interpret.

In practice, the field has converged on a hybrid view: natural language for tasks where human oversight is needed (e.g., final answers, justifications) and latent communication for intermediate, agent-to-agent signalling. A schematic of this hybrid view is shown in 2 (b).

3.3 When to Prefer Latent Communication↩︎

Synthesising the above, latent communication tends to win when all of the following hold:

1. The two agents are tightly coupled (e.g., a planner feeding directly into an executor).

2. The communication is intermediate — the user does not need to see the message.

3. The sender and receiver share (or can be aligned to) a common latent space (e.g., same backbone, or compatible architectures).

4. Latency is a binding constraint (e.g., real-time pipelines, edge deployment, or large agent counts).

Conversely, natural language wins when interpretability, cross-organisation interoperability, or human oversight is required.

4 A Unified Framework for Latent Communication↩︎

We now present the central contribution of this paper: a unified framework that organises all existing latent communication methods along three orthogonal axes. We claim that every latent communication method can be uniquely described by a triple: \[\text{Method} = (\underbrace{\text{WHAT}}_{\text{type of information}},\;\underbrace{\text{WHICH}}_{\text{alignment}},\;\underbrace{\text{HOW}}_{\text{fusion}}). \label{eq:triple}\tag{2}\]

The framework is summarised schematically in 3.

4.1 Axis 1 — WHAT: Types of Communicated Information↩︎

The first axis asks: what continuous quantity does the sender expose to the receiver? The dominant choices in the literature are Embeddings, Hidden States, and KV-Caches, with several methods exploring other quantities (state deltas, persistent memory, attention-only signals).

4.1.1 Embeddings↩︎

The sender transmits its input embedding \(\mathbf{e}_i \in \mathbb{R}^d\) for one or more tokens. Embeddings are the lowest-level continuous representation; they are model-agnostic in the sense that any model with a compatible embedding dimension can in principle consume them. CIPHER [6] is the canonical example: it computes a weighted embedding where the weights are derived from the sender’s output logits, so that the embedding encodes the sender’s full vocabulary distribution rather than a single sampled token.

4.1.2 Hidden States↩︎

The sender transmits the hidden state \(\mathbf{h}_i^{(\ell)}\) of token \(i\) at layer \(\ell\). Hidden states are richer than embeddings: they encode the agent’s intermediate reasoning, including the effect of attention over its context. AC [7], Interlat [8], SDE [9], ThoughtComm [10], and Mixture of Thoughts [11] all use hidden states as the communicated quantity.

4.1.3 KV-Caches↩︎

The sender transmits its per-token, per-layer KV-cache. The receiver can then resume generation as if it had pre-filled the sender’s context. KVComm [12], Cache-to-Cache [13], LatentMAS [14], Q-KVComm [15], LRAgent [16], RelayCaching [17], Agent Memory [18], Agent Primitives [19], and Edge LLM Handover [20] all use KV-caches.

4.1.4 Other Communicated Quantities↩︎

A small but growing set of methods transmits non-standard quantities:

• State delta trajectory (SDE [9]): the change in hidden state at each layer, rather than the state itself. This compresses the information into a direction in latent space and has been shown to be more robust when sender and receiver architectures differ slightly.

• Persistent KV-cache memory (Agent Memory [18]): a disk-persistent 4-bit-quantised KV-cache, used to offload the cache to edge devices.

• Visual-latent wormhole (Vision Wormhole [21]): a sender’s hidden state is rendered into a VLM’s visual input space, exploiting the VLM’s visual pathway as a universal channel.

• Centralised workspace state (BIGMAS [22]): a shared workspace in which agents deposit and read structured latent messages, mediated by an orchestrator.

4.1.5 Comparative Summary↩︎

1 summarises which information type each method uses.

4.2 Axis 2 — WHICH: Sender–Receiver Alignment↩︎

The second axis asks: which parts of the sender correspond to which parts of the receiver? Alignment has two sub-dimensions: latent information alignment (does the sender’s semantic space match the receiver’s?) and layer alignment (which layer of the sender feeds into which layer of the receiver?).

4.2.1 Latent Information Alignment↩︎

If the sender and receiver are the same model (e.g., two instances of Llama-3-8B), their latent spaces are identical by construction — no alignment is needed. If they are different but architecturally compatible (e.g., two Llama-3 fine-tunes), the spaces are close but not identical; methods such as Interlat [8] and Cache-to-Cache [13] apply learned projection heads to bridge the gap. If they are architecturally heterogeneous (e.g., Llama-3 and Qwen-2), a Universal Visual Codec [21] or a learned interaction layer [11] is needed.

2 indicates which methods perform explicit alignment.

4pt

4.2.2 Layer Alignment↩︎

The second sub-axis specifies the layer-to-layer correspondence between sender and receiver. Two natural extremes appear repeatedly:

• Last \(\rightarrow\) First. The sender exposes the hidden state of its last layer, and the receiver injects it at its first layer. Used by CIPHER, AC, Interlat. This is the simplest mapping and works well when the sender’s last layer is the most semantically rich.

• All \(\rightarrow\) Corresponding. The sender exposes the hidden state of every layer, and the receiver injects each one into the corresponding layer (i.e., layer \(\ell\) of the sender feeds layer \(\ell\) of the receiver). Used by Cache-to-Cache, LatentMAS, SDE. This preserves the layer-wise structure of the sender’s computation and is the natural choice for KV-cache methods.

Intermediate variants include:

• Selected \(\rightarrow\) Selected. The sender selects \(n \ge 1\) layers via a heuristic or learned gate, and the receiver injects them at the same indices. Used by AC, KVComm, Q-KVComm.

• Sparse top-\(k\) attention. A sub-variant of selected \(\rightarrow\) selected in which the receiver attends over only the top-\(k\) most relevant layers (used by KVComm).

4.3 Axis 3 — HOW: Information Fusion Strategy↩︎

The third axis asks: how is the communicated information incorporated into the receiver’s computation? The major options are:

4.3.1 Concatenation↩︎

The sender’s latent is concatenated with the receiver’s prompt embedding (or hidden state) along the token axis. This is the simplest fusion and is used by CIPHER, Interlat, and several early hidden-state methods.

4.3.2 Prepending (Token-axis Prepend)↩︎

The sender’s latent is prepended to the receiver’s KV-cache. This is the natural fusion for KV-cache methods: the receiver can attend over the sender’s context as if it were the first few tokens of its own prompt. Used by KVComm, LatentMAS, and others.

4.3.3 Mathematical Operation↩︎

The sender’s latent is combined with the receiver’s hidden state (or KV-cache) by an element-wise operation: addition, subtraction, or a small learned linear projection. Used by AC (addition of last-token hidden states), SDE (addition of state deltas), and others.

4.3.4 Cross-Attention↩︎

The receiver attends over a set of sender latents using a learned cross-attention layer. Used by Mixture of Thoughts [11], where a primary expert attends over a top-\(K\) set of peer experts’ projected hidden states.

4.3.5 Cache Restoration / Direct Injection↩︎

The receiver replaces part of its own KV-cache with the sender’s KV-cache. Used by RelayCaching [17] and Agent Memory [18], where the goal is to avoid recomputation rather than to mix information.

4.3.6 Comparative Table↩︎

3 lists the fusion strategy for each method.

4.4 Combining the Axes↩︎

The three axes are orthogonal: a method’s WHAT, WHICH, and HOW can be chosen largely independently. This means the design space has a multiplicative rather than additive structure. With three options for WHAT, three for WHICH, and five for HOW, there are 45 conceptually distinct positions; the 18 methods surveyed in this paper occupy about 17 of them, suggesting the design space is not yet saturated.

A bird’s-eye view of how all 18 methods fit into the framework is given in 4.

5 Method Analysis under the Framework↩︎

This section provides a one-paragraph analysis for each of the eighteen methods, structured as: (a) core idea, (b) framework placement (WHAT / WHICH / HOW), (c) strengths, (d) limitations, and (e) reported results and code. Methods are grouped by the WHAT axis for narrative flow.

5.1 Embedding-Based Methods↩︎

5.2 Hidden-State-Based Methods↩︎

5.3 KV-Cache-Based Methods↩︎

5.4 Hybrid / Heterogeneous Methods↩︎

5.5 Survey and Aggregation Works↩︎

6 Implementation: The Training-Free Paradigm↩︎

A striking observation across the 18 surveyed methods is that most are training-free. The training-free property is a major advantage in practice: it means the methods can be deployed on top of any pre-trained LLM with no additional data, no GPU hours, and no risk of catastrophic forgetting.

4 lists the training regime for each method.

4pt

6.0.0.1 Why training-free dominates.

Training-free methods have three structural advantages:

1. Composability. They can be applied on top of any pre-trained LLM, including new releases, without re-training.

2. No data requirement. They do not need parallel latent–text corpora, which are expensive to construct.

3. Robustness. They cannot suffer from distribution shift between training and deployment.

6.0.0.2 When training helps.

Training becomes necessary when:

• The sender and receiver are architecturally heterogeneous and no hand-designed alignment works (e.g., Vision Wormhole, MoT).

• The fusion function is non-trivial and cannot be expressed as concatenation or addition (e.g., Cache-to-Cache’s learned fuser).

• The system needs to learn a routing policy over a large set of agents (e.g., MoT’s router). In general, choosing which subtask to assign to which agent is an adaptive selection problem in its own right, and a growing body of work studies the empirical design space of such selection strategies for LLM-based systems [5].

The field appears to be converging on a hybrid: training-free WHAT/WHICH axes combined with lightweight training on a small adapter for HOW (fusion). We expect this pattern to continue.

7 Benchmark Analysis and Empirical Insights↩︎

This section synthesises reported results from the 18 methods. Direct cross-method comparison is challenging because the methods use different backbones, benchmarks, and reporting conventions; we therefore focus on trends and order-of-magnitude effects rather than head-to-head numbers.

7.1 Benchmark Suites↩︎

Methods are typically evaluated on a mix of:

• Math reasoning: GSM8K, MATH, AIME.

• General knowledge: MMLU, ARC.

• Code generation: HumanEval, MBPP, LiveCodeBench.

• Multi-modal reasoning: MathVista, MMMU, ChartQA.

• Agentic QA: HotpotQA, 2WikiMultiHopQA, MuSiQue.

• Game-like reasoning: Game24, Six Fives, Tower of London.

• Competitive tabletop games: Mahjong, Uno, Honor of Kings — increasingly used as testbeds for evaluating inter-agent coordination under partial observability, for which dedicated toolkits such as RainbowArena [4] provide standardised APIs, opponent pools, and replay infrastructure.

7.2 Reported Quantitative Trends↩︎

• Latency reduction. Latent communication methods consistently reduce latency by 2–24\(\times\) relative to NL-Comm. The largest gains (24\(\times\)) are reported by Interlat on long-context multi-agent tasks.

• TTFT speedups. KV-cache-based methods (RelayCaching, Agent Memory) report TTFT speedups of 4.7\(\times\)–136\(\times\) relative to full re-prefill.

• Token savings. Latent methods typically reduce tokens generated by 3–4\(\times\) (Agent Primitives reports 3–4\(\times\) lower token usage vs.text-based MAS).

• Accuracy. Most methods report accuracy competitive with or better than NL-Comm baselines; SOTA gains are reported by SDE on complex reasoning, MoT on ID/OOD benchmarks, and LatentMAS on collaborative reasoning.

A schematic comparison of representative methods on the trade-off dimensions (accuracy, latency, generality, engineering complexity) is given in 5.

7.3 Insights↩︎

Three empirical patterns stand out:

1. Long context benefits most. The latency advantage of latent communication grows with context length, because the receiver avoids re-encoding a long prompt.

2. Same-model agents dominate. Most methods assume sender and receiver share the same backbone; cross-architecture methods (Vision Wormhole, MoT) are still rare and require training.

3. KV-cache is the emerging default. Of the 18 methods, 9 use KV-caches as the communicated quantity, and the share is growing.

8 Open Problems and Future Directions↩︎

Latent communication is a young field. We identify six open problems that we expect to shape the next generation of research. A mind-map view is given in 5.

8.1 Cross-Architecture Alignment↩︎

Most existing methods assume homogeneous agents (same backbone). The few methods that support heterogeneous agents (Vision Wormhole, MoT) require training a learned alignment module per pair of architectures. A general, training-free, \(\mathcal{O}(N)\) cross-architecture alignment method remains elusive.

8.2 Security and Robustness↩︎

A latent channel is opaque: there is no natural language to inspect for adversarial content. An attacker who controls the sender could embed adversarial perturbations in the hidden state that, while not affecting the receiver’s output text, cause the receiver to behave maliciously. Conversely, a compromised receiver could exfiltrate the sender’s hidden state. We see almost no work on security of latent channels and consider this a critical gap.

8.3 Compression and Quantisation↩︎

KV-cache methods are bottlenecked by the size of the cache they transmit. Q-KVComm and Agent Memory are the first methods to attack this problem with adaptive quantisation, but the design space is large. We expect learned compression, token-level compression, and layer-level compression to be active research areas.

8.4 Theoretical Understanding↩︎

The field is largely empirical. We lack a theoretical account of when latent communication should outperform natural language, how much information is actually transmitted, and what the upper bound on speedup is. Information-theoretic and learning-theoretic analyses are an open opportunity.

8.5 Latent Communication vs.Latent CoT↩︎

Latent CoT is the practice of performing chain-of-thought reasoning in latent space within a single agent (e.g., Coconut, LatentSeek). Latent Communication is the practice of exchanging latent messages between agents. The two share machinery (KV-cache reasoning, hidden-state deltas) but differ in where the latents flow: within one model or between two. A unified framework that handles both directions of latent flow is a natural next step. We discuss this in more detail in 9.

8.6 Real-World Deployment↩︎

Edge devices, mobile phones, and embedded systems have very different constraints from data centres. The KV-cache sharing techniques surveyed here (LRAgent, Agent Memory, Edge LLM Handover) are early steps in this direction, but we expect the battery, memory, and bandwidth constraints of edge deployment to drive significant new research.

9 Related Work↩︎

Latent communication sits at the intersection of four research areas: latent chain-of-thought, multi-agent reinforcement learning, emergent language, and KV-cache compression. We briefly situate our framework within each.

9.1 Latent Chain-of-Thought↩︎

Latent CoT is the practice of reasoning in continuous latent space within a single LLM. Representative works include Coconut [24], LatentSeek [25], and the awesome-lists [26], [27]. The relationship to latent communication is summarised in 6.

The two share machinery: both manipulate hidden states, KV-caches, and state deltas. They differ in where the latents flow: within one model (Latent CoT) or between two (Latent Communication). A unified framework that handles both directions of flow is a promising direction.

9.2 Multi-Agent Reinforcement Learning (MARL)↩︎

MARL has a long tradition of learned communication between agents [28]. Methods such as CommNet, TarMAC, and IC3Net learn continuous message vectors that are exchanged between agents. The modern LLM-based latent communication methods surveyed in this paper can be seen as a white-box, inference-time counterpart to MARL’s learned communication. The Five Ws survey [23] is a recent effort to bridge the two. On the empirical side, toolkits such as RainbowArena [4] provide standardised tabletop-game environments where both RL and LLM-based agents can be evaluated under controlled multi-agent conditions, offering a natural experimental bridge between the MARL and LLM-MAS communities.

9.3 Emergent Language↩︎

Emergent language studies the symbolic protocols that arise when agents are trained to communicate [29]. The protocols are typically discrete (unlike our continuous latents), but the underlying question — what should agents communicate? — is the same.

9.4 KV-Cache Compression and Sharing↩︎

A large body of work optimises the KV-cache for single-model inference: H2O [30], Scissorhands, KIVI, KVQuant, and others. The methods surveyed in this paper (KVComm, Cache-to-Cache, LatentMAS, Q-KVComm, etc.) extend this line of work to the multi-agent setting, where the cache is shared between agents.

10 Conclusion↩︎

We have presented a unified framework for latent communication in LLM-based multi-agent systems. The framework organises 18 representative methods along three orthogonal axes: WHAT (types of communicated information — Embeddings, Hidden States, KV-Caches, and others), WHICH (sender–receiver alignment — latent information alignment and layer alignment), and HOW (information fusion strategy — concatenation, prepending, mathematical operations, cross-attention, and cache restoration). The framework exposes five generalisable takeaways, surfaces six open problems, and bridges latent communication to adjacent research areas including latent CoT, MARL, emergent language, and KV-cache compression.

We hope this framework provides a shared vocabulary for the rapidly growing latent communication community and lowers the barrier to entry for new researchers. The field is moving fast — we expect the next 12–18 months to bring new methods, new benchmarks, and (hopefully) a theoretical understanding of when and why latent communication outperforms natural language.

10.0.0.1 Reproducibility.

All figures in this paper are derived from publicly available method illustrations (referenced inline). The companion repository at https://github.com/enochliu98/Awesome-Latent-Communication is continuously updated with new papers, code links, and reproducible figure prompts.

10.0.0.2 Acknowledgements.

We thank the authors of the 18 surveyed methods for making their code and data publicly available, and the open-source community for curating the companion awesome-list. We also thank the anonymous reviewers for their constructive feedback.

11 Method Quick-Reference Table↩︎

3pt

References↩︎