Scalable Peptide Design via
Memory-Efficient Equivariant Transformer
June 23, 2026
1 Introduction↩︎
Designing peptides that bind a specified protein pocket is a central problem in structure based drug discovery [1]. The task is a form of sequence and structure co-design, where peptide sequence, conformation, and pocket binding geometry are coupled through physical interactions. Accurate modeling therefore requires full atom geometric reasoning over side chain packing, hydrogen bonding, shape complementarity, and local steric compatibility. This makes E(3) equivariant architectures [2]–[5] a natural choice, since they respect the symmetries of three dimensional space while operating directly on molecular geometry.
Latent generative frameworks [6], [7] offer a practical way to balance full atom fidelity with generative efficiency. In this formulation, a VAE compresses fine grained atomic structures into block level latent points, so that local atomic details are preserved through reconstruction rather than represented explicitly throughout the diffusion process. Generation is then performed over a compact latent graph conditioned on the target pocket, with the decoder mapping the generated latent states back to full atom geometry. As the geometric backbone is instantiated throughout the VAE and LDM components, its expressiveness and memory efficiency become central to the scalability of the overall generative system.
This places a strong demand on the backbone used for full atom peptide and pocket complexes. Such complexes often contain hundreds to thousands of atoms. Existing geometric backbones commonly introduce coordinate dependence either through a dense distance matrix [8], or through explicit local molecular graphs [3], [5], [9]–[11]. These mechanisms are effective, but they require storing or recomputing pairwise biases, neighbor lists, or edge features, which increases memory traffic and restricts the feasible model size and batch size. The issue becomes more pronounced when scaling generative models, where improvements in sample quality often require both larger backbones and larger training sets.
To address this backbone level bottleneck, we introduce Meet (Memory Efficient Equivariant Transformer), an E(3) equivariant Transformer backbone for scalable atomistic peptide modeling. Meet keeps coupled scalar and vector feature streams, where the scalar stream is rotation invariant and the vector stream is rotation equivariant. Its geometric operations are written in forms that are compatible with memory efficient attention kernels such as FlashAttention [12]. In particular, Meet initializes vector features through global coordinate aggregation, folds pairwise distance information into augmented query and key dot products, and injects covalent bond information through a sparse bond adapter. Together these design choices avoid materializing quadratic activation tensors when memory efficient attention is used, while preserving E(3) equivariance of the full backbone.
We evaluate this backbone inside a two stage latent generative framework inspired by UniMoMo [7]. Within this framework, Meet serves as the geometric backbone for the encoder, block type decoder, structure decoder, and LDM denoiser. This setting lets us test whether a more memory efficient equivariant backbone improves the complete generation system. Figure 1 summarizes the architecture and the main geometric modules. Our scaling experiments focus on the LDM denoiser, whose capacity directly affects the quality of generated peptides in our framework.
Together, the architecture and generative evaluation lead to three concrete contributions.
Efficient architecture. Meet replaces dense distance biases with query and key augmentation, replaces local-graph vector initialization with global attention aggregation, and uses sparse bond adaptation for chemical adjacency. These choices give linear peak activation memory in the number of atoms for a fixed model size when paired with memory efficient attention kernels.
Large scale datasets. Starting from \(8.64\) million AFDB [13] domains, we construct approximately \(100\) million candidate segments and sample \(100\mathrm{K}\) and \(1.2\mathrm{M}\) training structures using sliding window enumeration, structure quality filtering, interface screening, and sequence overlap clustering.
Systematic scaling. On the \(100\mathrm{K}\) benchmark, Meet improves over PepGLAD [14], PepFlow [15], UniMoMo [7], and DiffPepBuilder [16] on binding free energy and physical validity. Scaling the latent denoiser across four DiT-style [17] model sizes on \(1.2\mathrm{M}\) datasets further improves generation quality and restores sample diversity.
2 Methods↩︎
Our peptide generator follows a two-stage latent generative framework inspired by UniMoMo [7]. A VAE first encodes each full-atom peptide–pocket complex into block-level latent variables, a conditional latent diffusion model generates peptide latents from the target-pocket context, and the VAE decoder maps the generated latents back to peptide sequence and full-atom geometry. Meet serves as the geometric backbone in the VAE encoder, sequence decoder, structure decoder, and latent denoiser. The complete training objectives and sampling procedure are provided in Appendix 5.
In the following section, we first present an overview of the Meet architecture in §2.1. We then describe the distance-aware attention mechanism (§2.2) that is shared across the backbone, followed by the four main modules shown in Figure 1, including feature initialization (§2.3), equivariant self-attention (§2.4), equivariant feed-forward layer (§2.5), and bond adapter (§2.6). Finally, we analyze the space complexity in §2.7.
2.1 Architecture Overview↩︎
Meet is a memory-efficient equivariant Transformer that maintains two coupled feature streams, scalar and vector. We use the same notation as Figure 1, where \({\boldsymbol{H}}\) denotes the scalar stream, \({\boldsymbol{V}}\) denotes the vector stream, and \({\boldsymbol{X}}\) denotes atom coordinates. For a molecule with \(N\) atoms, let \({\boldsymbol{X}}\in \mathbb{R}^{N \times 3}\) have rows \({\boldsymbol{x}}_i \in \mathbb{R}^3\). We represent per-atom features as a pair \(({\boldsymbol{H}}, {\boldsymbol{V}})\), where \({\boldsymbol{H}}\in \mathbb{R}^{N \times d}\) are scalar (rotation-invariant) features and \({\boldsymbol{V}}\in \mathbb{R}^{N \times 3 \times d}\) are vector (rotation-equivariant) features, following the common scalar-vector decomposition used in equivariant neural networks [4]. Under a rigid transformation \(({\boldsymbol{R}}, {\boldsymbol{t}})\) with \({\boldsymbol{R}}\in \mathrm{SO}(3)\) and \({\boldsymbol{t}}\in \mathbb{R}^3\), features transform as \({\boldsymbol{H}}\mapsto {\boldsymbol{H}}\) and \({\boldsymbol{V}}\mapsto {\boldsymbol{R}}\,{\boldsymbol{V}}\). All modules of Meet are therefore E(3)-equivariant under the full Euclidean group.
Given input atom attributes \({\boldsymbol{A}}\) and coordinates \({\boldsymbol{X}}\), the backbone first constructs an initial equivariant state via an initialization operator \(\mathcal{I}\), \[({\boldsymbol{H}}^{(0)}, {\boldsymbol{V}}^{(0)}) \;=\; \mathcal{I}\bigl({\boldsymbol{A}}, {\boldsymbol{X}}\bigr),\] and then applies \(L\) Transformer blocks to produce \(({\boldsymbol{H}}^{(L)}, {\boldsymbol{V}}^{(L)})\). Each block consists of a self-attention layer \(\mathcal{A}\) and a vector–scalar mixing feed-forward layer \(\mathcal{F}\), optionally preceded by a bond adapter \(\mathcal{B}\) when a bond edge set \(\mathcal{E}\) is provided. \[\begin{align} (\bar{{\boldsymbol{H}}}^{(\ell)}, \bar{{\boldsymbol{V}}}^{(\ell)}) &\;=\; ({\boldsymbol{H}}^{(\ell)}, {\boldsymbol{V}}^{(\ell)}) \;+\; \mathcal{B}^{(\ell)}\!\bigl(\mathrm{Norm}({\boldsymbol{H}}^{(\ell)}, {\boldsymbol{V}}^{(\ell)}),\, \mathcal{E}\bigr), \\ (\tilde{{\boldsymbol{H}}}^{(\ell)}, \tilde{{\boldsymbol{V}}}^{(\ell)}) &\;=\; (\bar{{\boldsymbol{H}}}^{(\ell)}, \bar{{\boldsymbol{V}}}^{(\ell)}) \;+\; \mathcal{A}^{(\ell)}\!\bigl(\mathrm{Norm}(\bar{{\boldsymbol{H}}}^{(\ell)}, \bar{{\boldsymbol{V}}}^{(\ell)}),\, {\boldsymbol{X}}\bigr), \\ ({\boldsymbol{H}}^{(\ell+1)}, {\boldsymbol{V}}^{(\ell+1)}) &\;=\; (\tilde{{\boldsymbol{H}}}^{(\ell)}, \tilde{{\boldsymbol{V}}}^{(\ell)}) \;+\; \mathcal{F}^{(\ell)}\!\bigl(\mathrm{Norm}(\tilde{{\boldsymbol{H}}}^{(\ell)}, \tilde{{\boldsymbol{V}}}^{(\ell)})\bigr). \end{align}\] Each layer is wrapped with pre-normalization and a residual connection, following standard Transformer practice. In module-level descriptions below, \(({\boldsymbol{H}}_{\mathrm{in}}, {\boldsymbol{V}}_{\mathrm{in}})\) and \(({\boldsymbol{H}}_{\mathrm{out}}, {\boldsymbol{V}}_{\mathrm{out}})\) refer to the streams entering and leaving the corresponding panel in Figure 1. The bond adapter can be applied in every block, only in the first block, or disabled entirely depending on bond availability.
2.1.0.1 Normalization.
For the scalar stream we use RMSNorm [18]. For the vector stream we use a rotation-invariant analogue that normalizes by the combined root-mean-square over the spatial and channel axes of each atom’s vector features, \[\mathrm{RMSNorm}_V({\boldsymbol{V}}_i) \;=\; \frac{{\boldsymbol{V}}_i}{\sqrt{\dfrac{1}{3d}\sum_{k,c} {\boldsymbol{V}}_{i,k,c}^{2} + \epsilon}}.\] The scale factor in the denominator is a rotation invariant, so the output transforms as \({\boldsymbol{R}}\,{\boldsymbol{V}}_i\) under a rotation and remains equivariant.
2.2 Distance-aware Attention Mechanism↩︎
A component shared by the initialization layer \(\mathcal{I}\) and the self-attention layers \(\mathcal{A}\) is a distance-aware query–key augmentation. For head \(h\), the attention logit includes an invariant distance penalty, \[A_{ij}^{(h)} \;=\; \rho_h\Bigl(\langle {\boldsymbol{q}}_i^{(h)}, {\boldsymbol{k}}_j^{(h)} \rangle \;-\; s_h^{2}\,\lVert {\boldsymbol{x}}_i - {\boldsymbol{x}}_j\rVert_2^{2}\Bigr), \label{eq:dist-logit}\tag{1}\] where \({\boldsymbol{q}}_i^{(h)}\) and \({\boldsymbol{k}}_j^{(h)}\) are the module-specific query and key features, \(\rho_h\) is the attention scale, and \(s_h\) is a learnable distance scale.
According to FlashBias [19], rather than storing the distance term as an \(N\times N\) bias tensor, we absorb it into the query–key dot product. For coordinates \({\boldsymbol{x}}_i\) with components \((x_{1,i}, x_{2,i}, x_{3,i})\), define \[\phi_{s_h}({\boldsymbol{x}}_i) \;=\; s_h \bigoplus_{k=1}^{3}\bigl[x_{k,i}^{2},\; 1,\; -2 x_{k,i}\bigr], \qquad \psi_{s_h}({\boldsymbol{x}}_j) \;=\; s_h \bigoplus_{k=1}^{3}\bigl[1,\; x_{k,j}^{2},\; x_{k,j}\bigr]. \label{eq:dist-enc}\tag{2}\] Since \(\langle \phi_{s_h}({\boldsymbol{x}}_i), \psi_{s_h}({\boldsymbol{x}}_j)\rangle = s_h^{2}\lVert {\boldsymbol{x}}_i - {\boldsymbol{x}}_j\rVert_2^{2}\), concatenating \(-\phi_s\) to the query and \(\psi_s\) to the key reproduces Equation equation 1 , \[{\boldsymbol{Q}}_i^{(h)} \;=\; \mathrm{concat}\bigl({\boldsymbol{q}}_i^{(h)},\; -\phi_{s_h}({\boldsymbol{x}}_i)\bigr), \qquad {\boldsymbol{K}}_j^{(h)} \;=\; \mathrm{concat}\bigl({\boldsymbol{k}}_j^{(h)},\; \psi_{s_h}({\boldsymbol{x}}_j)\bigr), \qquad A_{ij}^{(h)} \;=\; \rho_h\langle {\boldsymbol{Q}}_i^{(h)},\, {\boldsymbol{K}}_j^{(h)}\rangle. \label{eq:dist-qkaug}\tag{3}\] The augmentation adds only \(9\) entries per head and keeps the attention computation compatible with fused kernels without an external distance-bias tensor. The resulting softmax weights \[\alpha_{ij}^{(h)} \;=\; \mathrm{softmax}_j\bigl(A_{ij}^{(h)}\bigr)\] depend on coordinates only through pairwise distances and are therefore E(3)-invariant.
2.3 Feature Initialization↩︎
The initialization layer \(\mathcal{I}\) prepares the initial scalar and vector streams from atom attributes and coordinates. The scalar stream is obtained by embedding the input atom attributes, denoted as \({\boldsymbol{H}}^{(0)}=\mathrm{Embed}({\boldsymbol{A}})\). The main role of \(\mathcal{I}\) is therefore to construct a non-trivial vector stream \({\boldsymbol{V}}^{(0)}\) from geometry. This step is necessary because every subsequent operation on the vector stream is either a bias-free linear projection along the channel axis or a linear combination of equivariant vectors. If \({\boldsymbol{V}}\) were initialized as zero, the vector branch would remain uninformative and could not carry directional information.
Earlier equivariant backbones typically obtain directional states by aggregating relative positions over explicit local molecular graphs [3], [5], [9]–[11]. Such designs require constructing neighbor edges and storing edge features, which increases memory traffic for large full-atom complexes. We instead initialize vector features through one round of distance-aware multi-head attention over the atoms in a complex, keeping the initialization compatible with memory-efficient attention kernels without constructing an explicit local molecular graph.
Concretely, queries and keys are computed from \({\boldsymbol{H}}^{(0)}\) and augmented with the distance encodings in Section 2.2, yielding invariant per-head attention weights \(\alpha_{ij}^{(h)}\). We collect these weights into a per-head attention matrix \({\boldsymbol{\alpha}}^{(h)} \in \mathbb{R}^{N \times N}\). Unlike a standard attention layer, the value in \(\mathcal{I}\) is the coordinate matrix \({\boldsymbol{X}}\in \mathbb{R}^{N \times 3}\) itself, shared across heads. For each head \(h\), the attended coordinate is centered at the query atom, \[{\boldsymbol{V}}^{(h)} \;=\; {\boldsymbol{\alpha}}^{(h)} {\boldsymbol{X}}- {\boldsymbol{X}}\;=\; \bigl({\boldsymbol{\alpha}}^{(h)} - {\boldsymbol{I}}\bigr){\boldsymbol{X}}\;\in\; \mathbb{R}^{N \times 3}.\] Equivalently, the vector at atom \(i\) is \[{\boldsymbol{v}}_i^{(h)} \;=\; \sum_j \alpha_{ij}^{(h)}({\boldsymbol{x}}_j-{\boldsymbol{x}}_i).\] The centering by \({\boldsymbol{x}}_i\) removes dependence on the absolute coordinate frame and ensures that only relative displacements enter the vector stream.
The \(H\) per-head displacement fields are then stacked along a new channel axis and projected to the model dimension with a bias-free linear map, \[{\boldsymbol{V}}^{(0)} \;=\; \mathrm{stack}_h\bigl({\boldsymbol{V}}^{(h)}\bigr)\,{\boldsymbol{W}}_{\text{out}}^{\mathcal{I}} \;\in\; \mathbb{R}^{N \times 3 \times d},\] where \(\mathrm{stack}_h\) forms a tensor in \(\mathbb{R}^{N \times 3 \times H}\) before the projection mixes the head axis into \(d\) vector channels while leaving the spatial axis untouched. Because \(\alpha_{ij}^{(h)}\) depends only on pairwise distances, it is invariant to rigid transformations. The centered displacement \(\sum_j \alpha_{ij}^{(h)}({\boldsymbol{x}}_j-{\boldsymbol{x}}_i)\) is invariant to translation and transforms as a 3D vector under rotation. The bias-free output projection only mixes channels, so \({\boldsymbol{V}}^{(0)}\) is equivariant.
2.4 Equivariant Self-Attention↩︎
Each Transformer block applies an equivariant self-attention layer \(\mathcal{A}\) that updates both streams in a single multi-head attention pass, mapping \(({\boldsymbol{H}}_{\mathrm{in}}, {\boldsymbol{V}}_{\mathrm{in}})\) to \(({\boldsymbol{H}}_{\mathrm{out}}, {\boldsymbol{V}}_{\mathrm{out}})\) as in Figure 1.
2.4.0.1 Q, K, and U preparation.
The scalar and vector streams are projected independently. An unconstrained linear layer maps \({\boldsymbol{H}}_{\mathrm{in}}\) to \(({\boldsymbol{H}}_Q, {\boldsymbol{H}}_K, {\boldsymbol{H}}_U)\), while a bias-free linear layer followed by flattening on the channel axis of \({\boldsymbol{V}}_{\mathrm{in}}\) yields \(({\boldsymbol{V}}_Q, {\boldsymbol{V}}_K, {\boldsymbol{V}}_U)\). The vector queries and keys are normalized via \(\mathrm{RMSNorm}_V\) and their spatial axis is flattened into the channel axis. The per-head query and key features entering the distance-aware mechanism of Section 2.2 are then \[{\boldsymbol{q}}_i^{(h)} = \mathrm{concat}\bigl({\boldsymbol{H}}_{Q,i}^{(h)},\; \mathrm{flat}({\boldsymbol{V}}_{Q,i}^{(h)})\bigr), \qquad {\boldsymbol{k}}_j^{(h)} = \mathrm{concat}\bigl({\boldsymbol{H}}_{K,j}^{(h)},\; \mathrm{flat}({\boldsymbol{V}}_{K,j}^{(h)})\bigr), \label{eq:attn-qk}\tag{4}\] which are augmented with the distance encoding vectors \(-\phi_s\) and \(\psi_s\) (Equation equation 3 ) to form \({\boldsymbol{Q}}_i^{(h)}\) and \({\boldsymbol{K}}_j^{(h)}\) as before. The value \({\boldsymbol{U}}_j^{(h)}\) concatenates only the scalar and flattened-vector parts, without any distance encoding. \[{\boldsymbol{U}}_j^{(h)} = \mathrm{concat}\bigl({\boldsymbol{H}}_{U,j}^{(h)},\; \mathrm{flat}({\boldsymbol{V}}_{U,j}^{(h)})\bigr). \label{eq:attn-v}\tag{5}\] The dot product \(\langle {\boldsymbol{Q}}_i^{(h)}, {\boldsymbol{K}}_j^{(h)}\rangle\) is E(3)-invariant because the scalar parts are trivially invariant, the flattened vector parts contribute a sum of 3D inner products \(\sum_c \langle {\boldsymbol{V}}_{Q,i,:,c}, {\boldsymbol{V}}_{K,j,:,c}\rangle\) that are invariant by orthogonality of \({\boldsymbol{R}}\), and the distance encoding contributes \(-s_h^{2}\lVert {\boldsymbol{x}}_i - {\boldsymbol{x}}_j\rVert_2^{2}\).
2.4.0.2 Aggregation and output projection.
The invariant attention weights \(\alpha_{ij}^{(h)}\) are applied to \({\boldsymbol{U}}^{(h)}\), and the result is split into scalar and vector halves. \[{\boldsymbol{O}}_i^{(h)} = \sum_j \alpha_{ij}^{(h)}\, {\boldsymbol{U}}_j^{(h)}, \qquad {\boldsymbol{O}}_i^{(h)} \to \bigl({\boldsymbol{O}}_{H,i}^{(h)},\; {\boldsymbol{O}}_{V,i}^{(h)}\bigr).\] The per-head outputs are concatenated across heads and projected back to the model dimension by two independent output projections. \[{\boldsymbol{H}}_{\mathrm{out},i} = \mathrm{concat}_h\!\bigl({\boldsymbol{O}}_{H,i}^{(h)}\bigr)\,{\boldsymbol{W}}_{\text{out}}^{H}, \qquad {\boldsymbol{V}}_{\mathrm{out},i} = \mathrm{concat}_h\!\bigl({\boldsymbol{O}}_{V,i}^{(h)}\bigr)\,{\boldsymbol{W}}_{\text{out}}^{V},\] where \({\boldsymbol{W}}_{\text{out}}^{H}\) is unconstrained and \({\boldsymbol{W}}_{\text{out}}^{V}\) is bias-free and acts on the channel axis only. The scalar branch manipulates invariant quantities throughout, and the vector branch applies only bias-free channel mixing to equivariant tensors, so E(3) equivariance is preserved.
2.5 Equivariant Feed-Forward Layer↩︎
Each attention layer is followed by a feed-forward layer \(\mathcal{F}\) that couples the scalar and vector streams while preserving E(3) equivariance. The layer maps \(({\boldsymbol{H}}_{\mathrm{in}}, {\boldsymbol{V}}_{\mathrm{in}})\) to \(({\boldsymbol{H}}_{\mathrm{out}}, {\boldsymbol{V}}_{\mathrm{out}})\) as in Figure 1. The vector features are first projected through a bias-free linear layer and split into a scalar-summary part and a hidden part. \[({\boldsymbol{V}}_{\mathrm{in}}^{(1)},\; {\boldsymbol{V}}_{\mathrm{in}}^{(2)}) \;=\; \mathrm{split}\bigl({\boldsymbol{V}}_{\mathrm{in}}\,{\boldsymbol{W}}_V^{\text{in}}\bigr), \qquad {\boldsymbol{V}}_{\mathrm{in}}^{(1)} \in \mathbb{R}^{N \times 3 \times d},\; {\boldsymbol{V}}_{\mathrm{in}}^{(2)} \in \mathbb{R}^{N \times 3 \times d_{\text{ff}}}.\] An invariant summary \({\boldsymbol{Y}}_i = [\lVert {\boldsymbol{V}}_{\mathrm{in},i,:,1}^{(1)}\rVert_2, \ldots, \lVert {\boldsymbol{V}}_{\mathrm{in},i,:,d}^{(1)}\rVert_2] \in \mathbb{R}^{d}\) is computed by taking channel-wise norms. The scalar branch uses the SwiGLU nonlinearity [20], and the scalar and vector streams are then updated as \[\begin{align} ({\boldsymbol{h}}^{(1)},\; {\boldsymbol{h}}^{(2)}) &\;=\; \mathrm{split}\bigl([{\boldsymbol{H}}_{\mathrm{in}}, {\boldsymbol{Y}}]\,{\boldsymbol{W}}_H^{\text{in}} + {\boldsymbol{b}}_H^{\text{in}}\bigr), \\ {\boldsymbol{h}}^{(2)} &\;\leftarrow\; \mathrm{SwiGLU}({\boldsymbol{h}}^{(2)}), \qquad {\boldsymbol{V}}_{\mathrm{in}}^{(2)} \;\leftarrow\; \mathrm{SiLU}({\boldsymbol{h}}^{(1)}) \odot {\boldsymbol{V}}_{\mathrm{in}}^{(2)}, \\ {\boldsymbol{H}}_{\mathrm{out}} &\;=\; {\boldsymbol{h}}^{(2)}\,{\boldsymbol{W}}_H^{\text{out}} + {\boldsymbol{b}}_H^{\text{out}}, \qquad {\boldsymbol{V}}_{\mathrm{out}} \;=\; {\boldsymbol{V}}_{\mathrm{in}}^{(2)}\,{\boldsymbol{W}}_V^{\text{out}}. \end{align}\]
2.6 Bond Adapter↩︎
Geometric proximity alone does not fully determine local interactions. Chemical adjacency, covalent bonds, and other structured edge signals encode strong constraints that a dense attention mechanism captures only indirectly. Unlike the dense pairwise distance bias, bond adjacency is sparse and low-degree, so treating it as a dense attention bias would be wasteful. We instead inject bond information through a sparse message-passing adapter \(\mathcal{B}\) whose cost scales linearly with the number of edges.
Let \(\mathcal{E}\) denote the bond edge set and let \(e_{ij} \in \mathbb{R}^{d_e}\) be an edge attribute for \((i,j) \in \mathcal{E}\). For each edge, we use the notation in Figure 1, where \({\boldsymbol{h}}_i\) and \({\boldsymbol{h}}_j\) are the scalar features of the target and source atoms, and \({\boldsymbol{v}}_j\) is the source atom’s vector feature. We concatenate the scalar endpoint features with the edge attribute and pass the result through a small MLP \(f_\theta\). \[{\boldsymbol{m}}_{ij} \;=\; f_\theta\bigl([{\boldsymbol{h}}_i,\, {\boldsymbol{h}}_j,\, e_{ij}]\bigr) \;\in\; \mathbb{R}^{2d}.\] The output \({\boldsymbol{m}}_{ij}\) is split into a scalar message \(\Delta {\boldsymbol{h}}_{i\leftarrow j} \in \mathbb{R}^{d}\) and a gating coefficient \({\boldsymbol{g}}_{i\leftarrow j} \in \mathbb{R}^{d}\). The scalar messages are aggregated by mean-pooling over incoming edges, while the vector message \(\Delta {\boldsymbol{v}}_{i\leftarrow j}\) is constructed by gating the source vector feature \({\boldsymbol{v}}_j\) with the scalar coefficient \({\boldsymbol{g}}_{i\leftarrow j}\). \[\Delta {\boldsymbol{v}}_{i\leftarrow j} \;=\; {\boldsymbol{g}}_{i\leftarrow j} \odot {\boldsymbol{v}}_j.\] The bond-adapter updates are then \[\Delta {\boldsymbol{H}}_i \;=\; \frac{1}{|\mathcal{N}(i)|}\sum_{j \in \mathcal{N}(i)} \Delta {\boldsymbol{h}}_{i\leftarrow j}, \qquad \Delta {\boldsymbol{V}}_i \;=\; \frac{1}{|\mathcal{N}(i)|}\sum_{j \in \mathcal{N}(i)} \Delta {\boldsymbol{v}}_{i\leftarrow j},\] where \(\mathcal{N}(i) = \{j : (i,j) \in \mathcal{E}\}\) and the scalar gate \({\boldsymbol{g}}_{i\leftarrow j}\) is broadcast over the spatial axis of \({\boldsymbol{v}}_j\). Because \(\Delta {\boldsymbol{h}}_{i\leftarrow j}\) and \({\boldsymbol{g}}_{i\leftarrow j}\) are computed only from invariant quantities, they are E(3)-invariant. Because \({\boldsymbol{v}}_j\) is equivariant and the gate acts only on the channel axis, \(\Delta {\boldsymbol{v}}_{i\leftarrow j}\) and \(\Delta {\boldsymbol{V}}_i\) are equivariant as well. Each atom typically has at most a few incident edges (for example, \(\le 4\) covalent bonds), so the adapter has complexity \(\mathcal{O}(|\mathcal{E}|) = \mathcal{O}(N)\) with a small constant factor.
In practice, the adapter can be inserted at the beginning of every block or only in the first block, or disabled altogether when no bond edge set is provided. This flexibility lets the same backbone be reused across tasks that differ in whether explicit chemical adjacency is available.
2.7 Space Complexity Analysis↩︎
We analyze the space (memory) complexity of Meet for a single structure with \(N\) atoms, model dimension \(d\), number of attention heads \(H\) (head dimension \(d_h = d/H\)), \(L\) Transformer blocks, and \(|\mathcal{E}|\) edges. Following standard practice we set the feed-forward hidden dimension \(d_{\text{ff}} = \Theta(d)\), so that all width-dependent terms can be expressed in \(d\) alone. We separately account for parameter memory (model weights) and activation memory (intermediate tensors retained during inference or, more critically, during training). For batched inputs with per-sample lengths \(\{L_b\}_{b=1}^{B}\), \(N\) should be replaced by \(\sum_b L_b\) in the activation terms.
2.7.0.1 Parameter memory.
Model weights are independent of \(N\) and determined only by \(L\) and \(d\).
Feature initialization \(\mathcal{I}\). Scalar query/key projections contribute \(\mathcal{O}(d^2)\), and the output projection from head space to \(d\) channels contributes \(\mathcal{O}(Hd) = \mathcal{O}(d^2)\). One-time cost is \(\mathcal{O}(d^2)\).
Self-attention layers. Each block has independent scalar \({\boldsymbol{Q}}/{\boldsymbol{K}}/{\boldsymbol{U}}\) projections of size \(\mathcal{O}(d^2)\), independent bias-free vector \({\boldsymbol{Q}}/{\boldsymbol{K}}/{\boldsymbol{U}}\) projections of size \(\mathcal{O}(d^2)\), two output projections of size \(\mathcal{O}(d^2)\), and \(\mathcal{O}(H)\) learnable distance scales. Per-layer cost is \(\mathcal{O}(d^2)\).
Feed-forward layers. The scalar and vector branches each maintain input and output projections of size \(\mathcal{O}(d \cdot d_{\text{ff}}) = \mathcal{O}(d^2)\). Per-layer cost is \(\mathcal{O}(d^2)\).
Bond adapter. A small MLP maps \(2d + d_e\) inputs to \(2d\) outputs, with cost \(\mathcal{O}(d^2)\) in any block where the adapter is enabled.
Summing over components, the total parameter memory is \[\underbrace{\mathcal{O}(d^2)}_{\text{init}} \;+\; \underbrace{\mathcal{O}(L d^2)}_{\text{blocks}} \;=\; \mathcal{O}(L d^2).\]
2.7.0.2 Activation memory.
Feature initialization. One round of distance-aware attention has the same activation profile as a single attention layer, giving \(\mathcal{O}(N^2 H)\) under a naive kernel, or \(\mathcal{O}(N H) = \mathcal{O}(Nd)\) with a memory-efficient attention kernel. This is a one-time cost dominated by the \(L\)-layer backbone below.
Self-attention. A naive implementation materializes the \(N \times N\) logit matrix per head, requiring \(\mathcal{O}(N^2 H)\) memory per layer. With a memory-efficient attention kernel the full matrix is never stored, and only running softmax statistics and tile-sized buffers are maintained, reducing the per-layer cost to \(\mathcal{O}(Nd)\). The distance-aware formulation appends only \(9\) extra entries to queries and keys through the encoding in Equation equation 2 , so the effective head dimension increases from \(d_h\) to \(d_h + 9\) but the asymptotic class is unchanged. Crucially, no separate \(N \times N\) distance-bias tensor is required.
Feature streams and feed-forward layer. Each block maintains \({\boldsymbol{H}}\in \mathbb{R}^{N \times d}\) and \({\boldsymbol{V}}\in \mathbb{R}^{N \times 3 \times d}\), together occupying \(\mathcal{O}(Nd)\), plus \(\mathcal{O}(Nd)\) for the feed-forward intermediates. The peak memory across \(L\) layers depends on the checkpointing strategy. Without checkpointing it is \(\mathcal{O}(LNd)\), with \(\sqrt{L}\)-interval checkpointing it reduces to \(\mathcal{O}(\sqrt{L}\,Nd)\) at the cost of one extra forward pass, and full recomputation stores only \(\mathcal{O}(Nd)\) activations at inference time.
Bond adapter. Message passing over a sparse bond edge set stores \(\mathcal{O}(d)\) intermediate features per edge, giving \(\mathcal{O}(|\mathcal{E}|d)\). Under the chemical valence assumption \(|\mathcal{E}| \le cN\), this simplifies to \(\mathcal{O}(Nd)\) and is dominated by the feature-stream cost.
1 summarizes these results. By (i) using memory-efficient attention kernels and (ii) encoding distance information through query–key augmentation rather than through an explicit bias matrix, Meet avoids any \(\mathcal{O}(N^2)\) activation term. The overall peak activation memory therefore scales linearly in \(N\) for a fixed model size, making the architecture applicable to large molecular structures.
| Component | Parameters | Peak Activations |
|---|---|---|
| Initialization | \(\mathcal{O}(d^2)\) | \(\mathcal{O}(Nd)\) |
| Attention (naive) | \(\mathcal{O}(Ld^2)\) | \(\mathcal{O}(LN^2 d/d_h)\) |
| Attention (efficient) | \(\mathcal{O}(Ld^2)\) | \(\mathcal{O}(LNd)\) |
| Feed-forward | \(\mathcal{O}(Ld^2)\) | \(\mathcal{O}(LNd)\) |
| Bond adapter | \(\mathcal{O}(Ld^2)\) | \(\mathcal{O}(LNd)\) |
| Total (efficient) | \(\mathcal{O}(Ld^2)\) | \(\mathcal{O}(LNd)\) |
3 Experiments↩︎
3.1 Datasets↩︎
We constructed our training set using a pipeline similar to that of the previously reported CPSea dataset, while extending its scope to linear peptides and adjusting several filtering thresholds.
Briefly, we use AFDB [13] domains as source structures. For each protein structure, we first compute secondary structure assignments, pLDDT scores, GRAVY scores, hydrophobicity annotations, and the residue-level C-distance matrix. We then apply sliding windows of 3–13 residues to enumerate candidate segments and retain those with average GRAVY \(< 0.5\), hydrophobic residue ratio \(< 0.45\), helix ratio \(< 0.67\), sheet ratio \(< 0.34\), minimum pLDDT \(> 70\), and terminal amide C-distance within \(3.5\)–\(15.5~\text{\AA}\) for retained segments.
The resulting candidates are further filtered using interface-based criteria derived from buried surface area (BSA), requiring total BSA \(> 400~\text{\AA}^2\), relative BSA between \(0.35\) and \(0.85\), relative apolar BSA \(< 0.75\), and limited burial of the two terminal capping side chains (\(< 0.30\) each). In addition, the receptor neighborhood is required to form a connected structural graph under a residue-level connectivity cutoff of \(9.0~\text{\AA}\). Candidates that pass all filters are finally clustered using a sequence-overlap threshold of \(0.2\).
Starting from \(8.64\) million AFDB domains, this pipeline produces approximately \(100\) million candidate segments and \(31\) million clusters of peptide–protein complex structures. From these, we randomly sample \(100\mathrm{K}\) and \(1.2\mathrm{M}\) structures for the training runs reported below.
3.2 Memory Efficiency↩︎
We first evaluate the memory footprint of Meet empirically and examine whether the implementation follows the linear scaling predicted by the analysis in Table 1.
3.2.0.1 Setup.
To avoid confounding the backbone comparison with variability in real complexes, we construct synthetic peptide chains of varying length at full-atom resolution. Each chain is composed of alanine residues. For a given residue count \(n_{\mathrm{aa}}\), we enumerate all heavy atoms in the alanine template, place residues along a linear backbone with \(3.8~\text{\AA}\) spacing, add small Gaussian coordinate noise, and construct both intra-residue and inter-residue covalent bonds. We sweep \(n_{\mathrm{aa}}\) over powers of two from \(2\) to \(1{,}024\), yielding atom counts from roughly \(10\) to \(5{,}000\). For each chain length, we record the peak GPU memory allocated during a single inference forward pass without gradient computation. Meet and the original EPT [8] backbone are evaluated under the same model dimension and number of attention heads.
3.2.0.2 Overall comparison with EPT.
Figure 2 (left) reports peak allocated memory as a function of peptide length on a log–log scale. Across the entire range, Meet consistently uses less memory than EPT, and the gap widens as the chain becomes longer. At \(n_{\mathrm{aa}}=1{,}024\), Meet uses roughly \(300~\mathrm{MB}\) whereas EPT exceeds \(1{,}500~\mathrm{MB}\), a reduction of approximately \(5\times\). The Meet curve remains close to linear, matching the \(\mathcal{O}(LNd)\) activation scaling derived in Table 1. By contrast, EPT follows a visibly steeper trend, consistent with the \(\mathcal{O}(N^{2})\) memory cost of explicitly materializing pairwise distance-bias tensors.
3.2.0.3 Per-module breakdown.
Figure 2 (right) further decomposes the memory use of Meet into its four main components, namely FFN, self-attention, bond adapter, and initialization. All four curves grow approximately linearly as \(n_{\mathrm{aa}}\) increases from \(128\) to \(1{,}024\), indicating that none of the modules introduces a hidden quadratic term. The FFN and self-attention layers account for most of the footprint, as expected from their \(\mathcal{O}(Nd)\) activation tensors. The initialization and bond-adapter modules contribute smaller but still linear overheads.
3.3 Benchmark Results↩︎
We next compare Meet with existing peptide design methods on the \(100\mathrm{K}\) dataset described in Section 3.1.
3.3.0.1 Baselines and model variants.
We consider four recent generative baselines, PepGLAD [14], PepFlow [15], UniMoMo [7], and DiffPepBuilder [16]. For our approach, we evaluate two variants, MEET-XS (extra-small) and MEET-B (base), so that the effect of backbone capacity can be assessed at a fixed data scale.
3.3.0.2 Evaluation metrics.
We emphasize two metrics that capture complementary aspects of design quality. The first is \(\Delta G\), the predicted binding free energy, where lower values indicate stronger predicted binding. The second is PoseBuster [21] pass rate (PB), which measures the physical validity of generated poses and is better when higher. We also report shape complementarity (Shape), solvation-normalized binding energy (\(\Delta G / \Delta\mathrm{SASA}\)), and sequence diversity (Seq.Div.).
3.3.0.3 Results.
Table 2 presents the benchmark results. Both MEET variants outperform all baselines on the two primary metrics. MEET-B achieves a mean \(\Delta G\) of \(-27.40\) and a PoseBuster pass rate of \(0.799\), compared with \(-21.80\) and \(0.561\) for the strongest baseline, UniMoMo. PepGLAD and PepFlow do not pass the PoseBuster checks in this evaluation, and DiffPepBuilder reaches only \(0.001\). Even the smaller MEET-XS already surpasses every baseline, with a mean \(\Delta G\) of \(-25.67\) and a PB of \(0.660\). The same pattern is reflected in Shape and \(\Delta G/\Delta\mathrm{SASA}\), where MEET-B obtains the best values among all methods.
The main trade-off at this scale appears in sequence diversity. MEET-B has lower Seq.Div. (\(0.715\)) than PepGLAD (\(0.939\)) and UniMoMo (\(0.922\)), suggesting that the higher-capacity model samples a more concentrated distribution when trained on \(100\mathrm{K}\) examples. Rather than treating this as an architectural limitation, we examine in the next subsection whether the effect changes with substantially more training data.
| Method | \(\Delta G\) mean\(\,\downarrow\) | \(\Delta G\) med.\(\,\downarrow\) | PB\(\,\uparrow\) | \(\frac{\Delta G}{\Delta\text{SASA}}\) mean | \(\frac{\Delta G}{\Delta\text{SASA}}\) med. | Shape | Seq.Div. |
|---|---|---|---|---|---|---|---|
| PepGLAD | \(-18.02\) | \(-17.47\) | \(0.000\) | \(-0.67\) | \(-1.58\) | \(0.566\) | \(\mathbf{0.939}\) |
| PepFlow | \(-20.81\) | \(-19.55\) | \(0.000\) | \(-2.36\) | \(-2.40\) | \(0.566\) | \(0.798\) |
| UniMoMo | \(-21.80\) | \(-22.30\) | \(0.561\) | \(-2.01\) | \(-2.24\) | \(0.633\) | \(0.922\) |
| DiffPepBuilder | \(\phantom{-}1.21\) | \(-17.96\) | \(0.001\) | \(\phantom{-}0.27\) | \(-1.95\) | \(0.607\) | \(0.843\) |
| MEET-XS | \(-25.67\) | \(-24.17\) | \(0.660\) | \(-2.39\) | \(-2.38\) | \(0.635\) | \(0.838\) |
| MEET-B | \(\mathbf{-27.40}\) | \(\mathbf{-25.73}\) | \(\mathbf{0.799}\) | \(\mathbf{-2.53}\) | \(\mathbf{-2.51}\) | \(\mathbf{0.651}\) | \(0.715\) |
3.5pt
3.4 Scaling to a Larger Training Set↩︎
We then investigate how the system behaves when both the data scale and the capacity of the latent denoiser are increased. In this experiment, the VAE is kept fixed, and the Meet denoising network inside the LDM is scaled on the \(1.2\mathrm{M}\) dataset described in Section 3.1.
3.4.0.1 Model configurations.
We follow the depth and hidden-dimension conventions of DiT [17], using S, B, and L variants for the LDM denoising backbone. We also include an XS variant with half the S-model depth.
3.4.0.2 Training dynamics.
Figure 3 shows training and validation loss curves over \(100\mathrm{K}\) optimization steps. Larger backbones consistently reach lower loss, and the ordering MEET-L \(<\) MEET-B \(<\) MEET-S \(<\) MEET-XS is maintained on both splits throughout training. This trend indicates that the memory-efficient design of Meet does not prevent the denoiser from exploiting additional capacity. Instead, the backbone remains effective as the LDM is scaled to larger models.
3.4.0.3 Generation quality.
Table 3 reports downstream generation metrics for all four model sizes. As capacity increases from XS to L, mean \(\Delta G\) improves from \(-26.26\) to \(-28.22\) and PB increases from \(0.703\) to \(0.732\). Shape complementarity and solvation efficiency follow the same trend. The effect of data scale is also clear, as MEET-XS trained on \(1.2\mathrm{M}\) already matches or exceeds the \(100\mathrm{K}\) MEET-B setting across most metrics.
Importantly, sequence diversity improves substantially at the larger data scale. MEET-L retains a Seq.Div. of \(0.899\), compared with \(0.715\) for MEET-B on \(100\mathrm{K}\). This suggests that the reduced diversity observed in Table 2 is primarily a data-scale effect rather than an inherent consequence of the Meet backbone.
| Model | \(\Delta G\) mean\(\,\downarrow\) | \(\Delta G\) med.\(\,\downarrow\) | PB\(\,\uparrow\) | \(\frac{\Delta G}{\Delta\text{SASA}}\) mean | \(\frac{\Delta G}{\Delta\text{SASA}}\) med. | Shape | Seq.Div. |
|---|---|---|---|---|---|---|---|
| MEET-XS | \(-26.26\) | \(-25.06\) | \(0.703\) | \(-2.39\) | \(-2.38\) | \(0.640\) | \(\mathbf{0.927}\) |
| MEET-S | \(-27.63\) | \(-26.04\) | \(0.727\) | \(-2.52\) | \(-2.51\) | \(0.651\) | \(0.918\) |
| MEET-B | \(-27.61\) | \(-26.44\) | \(0.729\) | \(-2.55\) | \(-2.54\) | \(0.657\) | \(0.915\) |
| MEET-L | \(\mathbf{-28.22}\) | \(\mathbf{-27.04}\) | \(\mathbf{0.732}\) | \(\mathbf{-2.61}\) | \(\mathbf{-2.59}\) | \(\mathbf{0.659}\) | \(0.899\) |
3.5pt
Taken together, these results show that the memory-efficient architecture of Meet enables practical scaling along both the model-capacity and data axes, and that this scaling translates into consistent improvements in peptide design quality.
4 Conclusion↩︎
We introduced Meet, a memory-efficient E(3)-equivariant Transformer backbone for full-atom peptide design. Its main design principle is to express geometric computation in forms compatible with memory-efficient attention, including distance-aware query–key augmentation, global vector initialization, and sparse bond adaptation. This removes the quadratic activation bottleneck of dense geometric attention while preserving coupled invariant and equivariant feature streams.
Integrated into a VAE and latent diffusion framework, Meet improves both efficiency and generation quality. The backbone shows linear memory scaling in peptide length, outperforms prior peptide design methods on binding affinity and physical validity, and supports systematic scaling of the latent denoiser on larger training data. These results indicate that backbone efficiency is not only an implementation concern, but a key enabler for scaling full-atom generative models.
5 Introduction of Latent Generative Framework↩︎
We instantiate Meet in a two-stage latent generative framework for target-specific full-atom peptide design. The overall design follows the motivation of latent generative modeling in UniMoMo [7], where detailed atomistic structures are first compressed into a compact latent representation and conditional generation is then performed in this lower-dimensional space. This separation is useful for peptide design because the atomic system contains many local constraints, while the global design decision is naturally organized at the residue or block level.
5.0.0.1 Atom-level VAE.
For a peptide–protein complex, let \({\boldsymbol{X}}\in \mathbb{R}^{N \times 3}\) denote atom coordinates, let \(A_i\) denote atom types, and let \(S_b\) denote the block type of residue or fragment \(b\). The VAE maps the full-atom complex into block-level latent variables \[{\boldsymbol{Z}}= ({\boldsymbol{Z}}_H,{\boldsymbol{Z}}_X), \qquad {\boldsymbol{Z}}_H \in \mathbb{R}^{M \times d_z}, \qquad {\boldsymbol{Z}}_X \in \mathbb{R}^{M \times 3},\] where \(M\) is the number of blocks. The invariant latent \({\boldsymbol{Z}}_H\) captures block identity and local chemical context, whereas the equivariant latent \({\boldsymbol{Z}}_X\) records block-level geometry. The encoder first computes atom-level scalar and vector features with Meet, then aggregates atom features within each block to parameterize an approximate posterior \[q_{\phi}({\boldsymbol{Z}}\mid {\boldsymbol{X}},A,S) = \prod_{b=1}^{M} q_{\phi}({\boldsymbol{Z}}_{H,b}\mid {\boldsymbol{X}},A,S)\, q_{\phi}({\boldsymbol{Z}}_{X,b}\mid {\boldsymbol{X}},A,S).\] We regularize the invariant latent toward a standard Gaussian prior and the coordinate latent toward a Gaussian centered at the corresponding block center \({\boldsymbol{r}}_b\), \[p({\boldsymbol{Z}}) = \prod_{b=1}^{M} \mathcal{N}({\boldsymbol{Z}}_{H,b};{\boldsymbol{0}},{\boldsymbol{I}})\, \mathcal{N}({\boldsymbol{Z}}_{X,b};{\boldsymbol{r}}_b,{\boldsymbol{I}}).\] This prior encourages each block latent point to remain close to the full-atom geometry it represents, while still allowing the latent diffusion model to later model global peptide placement. In our framework, we do not introduce separate learned block or chain embeddings. The block abstraction is instead induced by atom-level encoding, block-wise pooling, position information, and the latent variables themselves.
The decoder factorizes reconstruction into sequence prediction and coordinate generation. The sequence decoder predicts block types from the latent point cloud, \[p_{\xi}(S\mid {\boldsymbol{Z}}) = \prod_{b=1}^{M} p_{\xi}(S_b\mid {\boldsymbol{Z}}_H,{\boldsymbol{Z}}_X),\] and is trained with cross entropy. Given the decoded or ground-truth block types, the structure decoder reconstructs atom coordinates under a bond graph specified by the target topology, peptide chain adjacency, predicted block identities, and terminal capping rules, avoiding a separate bond-generation distribution.
Coordinate reconstruction is trained as a continuous flow-matching problem [22]. For each atom, an initial coordinate \({\boldsymbol{X}}_{\mathrm{prior}}\) is sampled around the corresponding block latent coordinate, and a time \(t \sim \mathcal{U}(0,1)\) is drawn. We define the interpolation \[{\boldsymbol{X}}_t = {\boldsymbol{X}}_{\mathrm{prior}} + (1-t)({\boldsymbol{X}}- {\boldsymbol{X}}_{\mathrm{prior}}), \qquad {\boldsymbol{u}}^{\star} = {\boldsymbol{X}}- {\boldsymbol{X}}_{\mathrm{prior}},\] so that \(t=1\) corresponds to the prior state and \(t=0\) corresponds to the data structure. The structure decoder predicts a vector field \({\boldsymbol{u}}_{\xi}({\boldsymbol{X}}_t,t,{\boldsymbol{Z}},S)\) and is trained to match \({\boldsymbol{u}}^{\star}\). The full VAE objective can be written as \[\begin{align} \mathcal{L}_{\mathrm{VAE}} &= \lambda_{\mathrm{seq}}\, \mathrm{CE}\!\left(S,\hat{S}\right) + \lambda_{\mathrm{coord}}^{\mathrm{lig}}\, \bigl\|{\boldsymbol{M}}_{\mathrm{lig}}\odot({\boldsymbol{u}}_{\xi}-{\boldsymbol{u}}^{\star})\bigr\|_2^2 + \lambda_{\mathrm{coord}}^{\mathrm{poc}}\, \bigl\|{\boldsymbol{M}}_{\mathrm{poc}}\odot({\boldsymbol{u}}_{\xi}-{\boldsymbol{u}}^{\star})\bigr\|_2^2 \notag \\ &\quad + \lambda_{\mathrm{dist}}\,\mathcal{L}_{\mathrm{dist}} + \lambda_H\, D_{\mathrm{KL}}\!\left(q_{\phi}({\boldsymbol{Z}}_H\mid {\boldsymbol{X}},A,S)\,\|\,p({\boldsymbol{Z}}_H)\right) + \lambda_X\, D_{\mathrm{KL}}\!\left(q_{\phi}({\boldsymbol{Z}}_X\mid {\boldsymbol{X}},A,S)\,\|\,p({\boldsymbol{Z}}_X)\right). \end{align}\] Here \({\boldsymbol{M}}_{\mathrm{lig}}\) and \({\boldsymbol{M}}_{\mathrm{poc}}\) select ligand and pocket atoms, and \(\mathcal{L}_{\mathrm{dist}}\) preserves local geometric distances. During VAE decoding, the coordinates are initialized from \({\boldsymbol{X}}_{\mathrm{prior}}\) and updated along a decreasing time grid \(1=t_K>\cdots>t_0=0\) by \[{\boldsymbol{X}}_{t_{k-1}} = {\boldsymbol{X}}_{t_k} + (t_k-t_{k-1})\, {\boldsymbol{u}}_{\xi}({\boldsymbol{X}}_{t_k},t_k,{\boldsymbol{Z}},\hat{S}),\] which transports atoms from the latent prior toward a full-atom structure.
5.0.0.2 Latent diffusion model.
After VAE training, the autoencoder is frozen and a conditional latent diffusion model is trained on the VAE latents. Let \({\boldsymbol{C}}\) denote the target-pocket context encoded by the frozen VAE, and let \({\boldsymbol{Z}}_0=({\boldsymbol{Z}}_{H,0},{\boldsymbol{Z}}_{X,0})\) denote the clean peptide latents. The coordinate latent is centered by the pocket center and normalized by a fixed coordinate scale before diffusion training.
Both \({\boldsymbol{Z}}_H\) and \({\boldsymbol{Z}}_X\) are modeled with continuous-time cosine diffusion paths inspired by DDPM formulations [23], [24]. For each latent field \(r\in\{H,X\}\), the forward process samples \[{\boldsymbol{Z}}_{r,t} = \eta(t){\boldsymbol{Z}}_{r,0} + \sigma(t){\boldsymbol{\epsilon}}_r, \qquad t\sim\mathcal{U}(0,1), \qquad {\boldsymbol{\epsilon}}_r\sim\mathcal{N}({\boldsymbol{0}},{\boldsymbol{I}}),\] where \(\eta(t)\) and \(\sigma(t)\) are the cosine schedule coefficients. The denoising network \({\boldsymbol{\epsilon}}_{\theta}\) is a Meet backbone conditioned on \({\boldsymbol{C}}\) and trained to predict the injected noise. The LDM objective is \[\mathcal{L}_{\mathrm{LDM}} = \mathbb{E}_{t,{\boldsymbol{Z}}_0,{\boldsymbol{\epsilon}}_H,{\boldsymbol{\epsilon}}_X} \left[ \left\| {\boldsymbol{\epsilon}}_{\theta}^{H}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}})-{\boldsymbol{\epsilon}}_H \right\|_2^2 + \left\| {\boldsymbol{\epsilon}}_{\theta}^{X}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}})-{\boldsymbol{\epsilon}}_X \right\|_2^2 \right].\]
At inference time, the target pocket is first encoded into \({\boldsymbol{C}}\). Peptide latents are initialized from Gaussian noise at \(t=1\), and the reverse trajectory is solved with the probability-flow ODE sampler [25], denoted as DiffusionODE in our experiments. For a DDPM probability path, the corresponding forward SDE has drift and diffusion coefficients \[f({\boldsymbol{Z}}_{r,t},t) = -\frac{1}{2}\beta(t){\boldsymbol{Z}}_{r,t}, \qquad g(t) = \sqrt{\beta(t)}.\] Since the denoiser predicts the noise, the score is estimated as \[{\boldsymbol{s}}_{\theta}^{r}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}}) = -\frac{{\boldsymbol{\epsilon}}_{\theta}^{r}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}})}{\sigma(t)}, \qquad r\in\{H,X\}.\] The probability-flow ODE is then \[\frac{\mathrm{d}{\boldsymbol{Z}}_{r,t}}{\mathrm{d}t} = f({\boldsymbol{Z}}_{r,t},t) - \frac{1}{2}g(t)^2 {\boldsymbol{s}}_{\theta}^{r}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}}), \qquad r\in\{H,X\}.\] Using a decreasing time grid \(1=t_K>\cdots>t_0=0\), the sampler applies an Euler update \[{\boldsymbol{Z}}_{r,t_{k-1}} = {\boldsymbol{Z}}_{r,t_k} + (t_{k-1}-t_k) \left[ f({\boldsymbol{Z}}_{r,t_k},t_k) - \frac{1}{2}g(t_k)^2 {\boldsymbol{s}}_{\theta}^{r}({\boldsymbol{Z}}_{H,t_k},{\boldsymbol{Z}}_{X,t_k},t_k,{\boldsymbol{C}}) \right].\] On the final step, the solver returns the corresponding clean-latent prediction \[\hat{{\boldsymbol{Z}}}_{r,0} = \frac{{\boldsymbol{Z}}_{r,t}-\sigma(t){\boldsymbol{\epsilon}}_{\theta}^{r}({\boldsymbol{Z}}_{H,t},{\boldsymbol{Z}}_{X,t},t,{\boldsymbol{C}})}{\eta(t)}.\] This procedure produces peptide block latents at \(t=0\). The frozen VAE decoder then predicts the peptide sequence, constructs the bond graph from the decoded blocks and peptide topology, initializes atom coordinates around \({\boldsymbol{Z}}_X\), and applies the coordinate flow decoder to generate the final full-atom peptide–protein complex. In this way, Meet is used in the encoder, the VAE decoders, and the latent denoiser, making the scalability of the backbone central to the efficiency of the entire pipeline.
6 Evaluation Metrics↩︎
The benchmark tables report five evaluation metrics, including predicted binding free energy \(\Delta G\), PoseBuster pass rate (PB), shape complementarity (Shape), solvation-normalized binding energy \(\Delta G/\Delta\mathrm{SASA}\), and sequence diversity (Seq.Div.). These metrics are chosen to jointly assess binding affinity, physical validity, interface packing, energetic efficiency, and sample diversity.
This metric measures the predicted binding free energy of the generated peptide against the target pocket. For each generated complex, the structure is first relaxed under coordinate constraints, and the peptide–target interface is then scored with a PyRosetta interface energy function [26]. Lower \(\Delta G\) indicates a more favorable predicted binding interaction. We report both the mean and the median across generated samples, since the mean reflects overall sample quality while the median is less sensitive to a small number of very strong or very weak designs.
PB. This metric measures the fraction of generated poses that pass PoseBuster [21]. PoseBuster evaluates whether a generated peptide pose is chemically and geometrically plausible, including molecular parsing, bond geometry, internal clashes, flatness constraints, and intermolecular clashes with the target pocket. A sample is counted as valid only when all selected checks are passed. Higher PB therefore indicates that a method produces fewer physically implausible complexes.
Shape. This metric denotes the PyRosetta shape-complementarity score of the peptide–target interface after the same constrained relaxation used for interface scoring [26]. It measures how well the molecular surfaces of the generated peptide and target pocket pack against each other. Higher Shape values indicate tighter and more geometrically complementary interfaces, independent of whether the peptide sequence matches the reference binder for that target.
This metric normalizes the predicted binding free energy by the buried solvent-accessible surface area at the interface. This metric distinguishes designs that obtain favorable energy through efficient local interactions from those that rely mainly on forming a larger buried interface. Lower values indicate stronger predicted binding per unit buried area. As with \(\Delta G\), we report both the mean and the median across generated samples for each method.
Seq.Div. This metric measures the diversity of generated peptide sequences for the same target. For each target pocket, we compare all pairs of generated sequences and compute their pairwise dissimilarity as one minus their position-wise amino-acid recovery. The score is averaged within each target and then averaged across targets. Higher Seq.Div. indicates that the model can propose a broader set of peptide sequences rather than repeatedly sampling near-identical binders.
7 Hyperparameter Settings↩︎
We list only the hyperparameters that determine model capacity, latent dimensionality, objective balance, and optimization across the reported runs.
| Name | Value | Description |
|---|---|---|
| Backbone | Full-atom encoder with bond-aware layers. | |
| Depth | \(6\) | Number of encoder layers. |
| Hidden size | \(128\) | Scalar and vector channel dimension. |
| Attention heads | \(8\) | Number of attention heads. |
| Latent size | \(8\) | Dimension of the invariant block latent state. |
| Backbone | Decoder for peptide block-type prediction. | |
| Depth | \(3\) | Number of decoder layers. |
| Hidden size | \(128\) | Hidden dimension shared with the encoder. |
| Attention heads | \(8\) | Number of attention heads. |
| Backbone | Full-atom coordinate decoder with specified bonds. | |
| Depth | \(6\) | Number of decoder layers. |
| Hidden size | \(128\) | Hidden dimension shared with the encoder. |
| Attention heads | \(8\) | Number of attention heads. |
| Coordinate prior std. | \(1.0\) | Standard deviation of the coordinate prior. |
| Decode steps | \(10\) | Number of coordinate-flow decoding iterations. |
| Training steps | \(100{,}000\) | Number of optimization steps. |
| Batch size | \(1024\) | Training batch size. |
| Learning rate | \(1.0\times10^{-3}\) | Initial learning rate with cosine decay. |
| Optimizer | AdamW | Weight decay \(1.0\times10^{-5}\). |
| \(\lambda_H\), \(\lambda_X\) | \(0.6\), \(0.8\) | Weights for latent KL terms. |
| \(\lambda_{\mathrm{seq}}\) | \(1.0\) | Weight for block-type cross entropy. |
| \(\lambda_{\mathrm{coord}}^{\mathrm{lig}}\) | \(1.0\) | Weight for peptide-atom vector-field matching. |
| \(\lambda_{\mathrm{coord}}^{\mathrm{poc}}\) | \(1.0\) | Weight for pocket-atom vector-field matching. |
| \(\lambda_{\mathrm{dist}}\) | \(0.5\) | Weight for preserving local geometry. |
5pt
| Backbone scale | Common training and sampling | |||||||
|---|---|---|---|---|---|---|---|---|
| 1-7(l)8-9 Data | Model | L | d | Heads | Steps | LR | Name | Value |
| \(100\mathrm{K}\) | MEET-XS | \(6\) | \(384\) | \(6\) | \(100{,}000\) | \(1.0\times10^{-3}\) | Loss weights | \(1.0\), \(1.0\) |
| \(100\mathrm{K}\) | MEET-B | \(12\) | \(512\) | \(8\) | \(100{,}000\) | \(3.0\times10^{-4}\) | Optimizer | AdamW |
| \(1.2\mathrm{M}\) | MEET-XS | \(6\) | \(384\) | \(6\) | \(100{,}000\) | \(1.0\times10^{-3}\) | Weight decay | \(1.0\times10^{-5}\) |
| \(1.2\mathrm{M}\) | MEET-S | \(12\) | \(384\) | \(6\) | \(100{,}000\) | \(5.0\times10^{-4}\) | LR schedule | Cosine |
| \(1.2\mathrm{M}\) | MEET-B | \(12\) | \(768\) | \(12\) | \(100{,}000\) | \(3.0\times10^{-4}\) | Sampler | Prob.-flow ODE |
| \(1.2\mathrm{M}\) | MEET-L | \(24\) | \(1024\) | \(16\) | \(200{,}000\) | \(1.0\times10^{-4}\) | Sampling steps | \(100\) |
3pt
For all LDM variants, the VAE is frozen during LDM training and the learning rate follows cosine decay. In our runs, we find that larger LDM backbones required smaller learning rates for stable and effective optimization, with XS, S, B and L using \(1.0\times10^{-3}\), \(5.0\times10^{-4}\), \(3.0\times10^{-4}\), and \(1.0\times10^{-4}\), respectively.