No Certificate for Alignment: Two Independent Impossibilities
and the Pareto Frontier of Achievable Safety Guarantees
March 08, 2026
Abstract
We argue that formal certification of AI alignment over open-ended or unbounded input domains is impossible under standard assumptions in computational complexity and learning theory, and characterise what remains achievable. Two structurally independent impossibility theorems support this position. The semantic barrier (Theorem [thm:semantic]): deciding whether a system satisfies any non-trivial alignment property over the full input domain is NP-hard for feedforward networks and undecidable for Turing-complete architectures—a direct consequence of neural-network verification complexity and Rice’s Theorem. The statistical barrier (Theorem [thm:statistical]): any verification procedure that is both sound and tractable cannot satisfy Completeness over the full input domain—a direct consequence of the impossibility of certifying infinite-domain properties from finite observations. These two theorems jointly entail a trilemma: no procedure can simultaneously satisfy soundness (no misaligned system is certified), completeness (no aligned system is rejected), and tractability (polynomial runtime). Each pair is simultaneously achievable; all three are not. We combine these results as a joint framework of two structurally independent barriers, prove their independence, and characterise the achievable Pareto frontier quantitatively via a constructive coverage-gap lower bound.
1 Introduction↩︎
Could we, even in principle, know that an AI system is aligned? The question is not whether a system has performed well on evaluations we have constructed, or optimised objectives we have chosen. It is whether any procedure—under any formalism—can certify alignment with the force of a mathematical guarantee.
The distinction is between measurement and proof. A sorting algorithm is accepted not because it has sorted many lists correctly, but because one can prove that for every admissible input the resulting list is ordered—the correctness follows necessarily. Formal verification of safety-critical software operates in this regime. The question is whether alignment can.
This paper establishes that it cannot, under alignment definitions requiring correct behaviour over open-ended or unbounded input domains, and characterises two structurally independent barriers responsible.
The first is semantic: established results in neural-network verification and computability theory show that deciding any non-trivial alignment property over the full input domain is NP-hard for feedforward networks and undecidable for Turing-complete architectures. The second is statistical: any polynomial-time procedure examines only finitely many inputs, and no finite body of evidence can certify a property required to hold over an unbounded domain.
Together they entail a trilemma (Corollary 1): no procedure can simultaneously satisfy soundness (no false positives), completeness (no false negatives), and tractability (polynomial runtime). Each pair is achievable; all three are not. The trilemma maps the space of achievable partial guarantees.
1.0.0.1 Contribution.
The individual ingredients—NP-hardness of neural-network verification, Rice’s Theorem, and PAC-theoretic limits on finite-sample certification—are each well established. The contributions of this paper are their joint formulation as two structurally independent barriers; a formal proof of that independence, with concrete witness systems for each direction; and a quantitative characterisation of the achievable frontier, including a constructive coverage-gap lower bound and a Pareto frontier characterisation for a canonical model class.
1.0.0.2 Organisation.
Section 2 positions prior work. Section 3 establishes formal definitions. Section 4 states and proves the two impossibility theorems. Section 5 derives the trilemma as a corollary. Section 6 characterises what remains achievable and defines the research agenda. Section 7 draws consequences for alignment practice. Section 8 concludes.
2 Related Work↩︎
Neural-network verification. Katz et al. [1] prove NP-completeness of ReLU-network property verification; Huang et al. [2] and Bastani et al. [3] address bounded-domain verification. Rice’s Theorem [4] establishes undecidability for non-trivial semantic properties of programs. Transformers are Turing-complete under specific regimes [5]–[8], bringing them within Rice’s Theorem’s scope. These results constitute the foundation for Theorem 1.
PAC learning and certification. Valiant [9] establishes that properties of infinite-domain concept classes cannot be certified from finite samples. Blumer et al. [10] connect this to VC dimension. Applied to alignment verification, these results ground Theorem 2.
Alignment methods. RLHF [11], [12], Constitutional AI [13], and DPO [14] define operational objectives without soundness guarantees over the full input domain. They occupy the complete-and-tractable regime characterised in Section 6.
Reward hacking and proxy gaps. Skalse et al. [15] formalise reward hacking; Gao et al. [16] demonstrate scaling laws for proxy overoptimisation; Kwa et al. [17] prove catastrophic Goodhart instances. These confirm that operating without soundness carries concrete, systematic failure modes.
Trilemmas in adjacent formal fields. Soundness-completeness-tractability tradeoffs appear in neighbouring disciplines: the CAP theorem [18] shows that distributed systems cannot simultaneously guarantee consistency, availability, and partition tolerance; tractability-completeness tradeoffs define the complexity landscape of description logics [19]; and abstract interpretation sacrifices completeness for tractability by design [20]. The alignment-certification trilemma differs structurally from these precedents: both barriers are simultaneously active and independent (Proposition 1), yielding a two-cause impossibility not found in prior trilemma analyses. No prior work places both barriers within a unified framework, proves their independence, or derives the achievable-guarantee frontier as a joint consequence.
3 Formal Framework↩︎
Definition 1 (AI System). An AI system is a parameterised function \(f_\theta\colon\mathcal{X}\to\mathcal{Y}\) with parameters \(\theta\in\Theta\subseteq\mathbb{R}^d\).
Definition 2 (Alignment Objective). An alignment objective \(A^*\colon\mathcal{F}\to[0,1]\) maps each system to an alignment score. \(A^*\) is formally specified if it is mathematically defined, though not necessarily computable. A system is aligned if \(A^*(\theta)\geq 1-\delta\) for stated tolerance \(\delta\geq 0\).
Definition 3 (Semantic Property). A property \(\mathcal{P}\) of \(f_\theta\) is semantic if it depends only on the function \(x\mapsto f_\theta(x)\), not on the parameter encoding \(\theta\). It is non-trivial if some system satisfies it and some does not.
Definition 4 (Verification Procedure). A verification procedure \(\mathcal{V}\) takes \(f_\theta\) as input and outputs a binary verdict: aligned or unaligned.
Definition 5 (Verification Properties). Given \(A^*\) and input domain \(\mathcal{X}\), a procedure \(\mathcal{V}\) satisfies:
Soundness (S): \(\mathcal{V}(\theta)=\textsf{aligned}\Rightarrow A^*(\theta)\geq 1-\delta\). (No false positives.)
Completeness (C): \(A^*(\theta)\geq 1-\delta\Rightarrow \mathcal{V}(\theta)=\textsf{aligned}\) over \(\mathcal{D}=\mathcal{X}\). (No false negatives; certificate valid over the full input domain.)
Tractability (T): \(\mathcal{V}\) terminates in time \(\mathrm{poly}(|\theta|)\).
Assumption 1 (Expressive Model Class). \(\mathcal{F}\) contains feedforward ReLU networks of sufficient depth and width to express any computable function over bounded inputs. For undecidability results, \(\mathcal{F}\) additionally contains transformers operating in a Turing-complete regime [5], [7].
Assumption 2 (Full-Domain Alignment). \(A^*\) is defined over the full input domain \(\mathcal{X}\): for any finite \(S\subset\mathcal{X}\), there exist \(\theta_1,\theta_2\) with \(f_{\theta_1}(x)=f_{\theta_2}(x)\) for all \(x\in S\), yet \(A^*(\theta_1)\neq A^*(\theta_2)\).
Remark 1. Assumption 2 holds for any \(A^*\) requiring generalisation beyond a fixed evaluation set. An \(A^*\) certifiable from finite observations reduces alignment to test-passing—certifying systems that behave correctly on the evaluation set but arbitrarily elsewhere, which defeats the purpose of a guarantee. Both impossibility theorems presuppose this assumption; alignment definitions explicitly restricted to a finite evaluation set fall outside their scope.
3.1 Running Example: The 1D Threshold Classifier↩︎
To ground all subsequent results, we introduce a minimal model class where every concept is checkable by hand and all three barriers are visible without computation.
Setup. Let \(\mathcal{X}=[0,1]\), \(\mathcal{Y}=\{0,1\}\), and \(\mathcal{F}_\tau=\{f_\theta:\theta\in[0,1]\}\) where \(f_\theta(x)=\mathbf{1}[x>\theta]\). Define \(A^*(\theta)=1-\theta\), the Lebesgue measure of inputs on which \(f_\theta\) outputs 1. A system is aligned if \(\theta\leq 0.05\) (i.e.it outputs 1 on at least 95% of \([0,1]\)).
Non-triviality witnesses. Two systems witness non-triviality: \(f_{0.04}\) is aligned (\(A^*=0.96\geq 0.95\)) and \(f_{0.06}\) is not (\(A^*=0.94 < 0.95\)). These are the explicit witnesses required by Rice’s Theorem in Part (ii) of Theorem 1.
Semantic barrier. A verifier must determine whether \(\theta\leq 0.05\). The two systems produce identical outputs at every \(x\notin(0.04,0.06)\)—a critical interval of width 0.02. Without a query inside this interval, no verifier can separate them. For a network with \(d\) parameters the analogous critical region shrinks while the number of regions grows super-polynomially.
Statistical barrier. For any finite query set \(S=\{x_1,\ldots,x_n\}\), place all queries outside \((0.04,0.06)\); both systems return identical outputs on \(S\). No finite verifier distinguishes the aligned system from the misaligned one. (Figure 1 shows this directly.)
Trilemma in this example. Table 1 instantiates all three trilemma corners concretely. We call back to this example in every theorem proof below.
| Regime | Procedure | Guarantee | Gap |
|---|---|---|---|
| S+C, not T | Query every rational in \([0,1]\) | Always correct | Infinite time |
| S+T, not C | Bounded-domain abstract interpreter; certify if bound holds | Sound over tested region | Input region beyond \(\mathcal{D}_b\) uncertified |
| C+T, not S | Always output “aligned” | Never rejects aligned | Passes \(\theta=0.99\) |
4 Two Impossibility Theorems↩︎
4.1 The Semantic Barrier↩︎
Sound and complete verification requires deciding a semantic property of \(f_\theta\) over the full input domain \(\mathcal{X}\) for every system simultaneously—a computation the following result shows cannot be tractable.
Theorem 1 (Semantic Barrier). Under Assumption 1, any verification procedure satisfying both Soundness and Completeness cannot satisfy Tractability.
Feedforward ReLU networks: violation of T follows from NP-completeness of neural-network property verification under \(\mathrm{P}\neq\mathrm{NP}\).
Turing-complete architectures: violation of T is unconditional for architectures that simulate arbitrary computation—the problem of deciding \(A^*\) is undecidable regardless of the computational resources available to the verifier.
Proof. (i) Feedforward ReLU networks. Sound and complete verification of any non-trivial alignment property over \(\mathcal{X}\) requires deciding \(\forall x\in\mathcal{X}:\varphi(x,f_\theta(x))\) for semantic specification \(\varphi\). Katz et al. [1] prove this is NP-complete for linear specifications even over bounded domains. Extending to \(\mathcal{D}=\mathcal{X}\) requires reasoning over all linear activation regions, the number of which grows as \(\Omega\bigl((n/L)^{L(d-1)}\bigr)\) for depth \(L\), width \(n\), input dimension \(d\) [21]—super-polynomial in the network’s parameter count under \(\mathrm{P}\neq\mathrm{NP}\). T fails.
(ii) Turing-complete architectures. Transformers in the regimes established by [5]–[8] subsume all Turing machines. We apply Rice’s Theorem [4], which requires verifying that the alignment property is (a) semantic and (b) non-trivial.
Condition (a): \(A^*\) is semantic. By Definition 3 of the framework, \(A^*(\theta)\) depends only on the input-output function \(x\mapsto f_\theta(x)\), not on the parameter encoding \(\theta\). It is therefore a semantic property of the computed function, in Rice’s sense.
Condition (b): \(A^*\) is non-trivial. We exhibit two explicit witness machines. Let \(M_1\) be the constant-1 function (\(f_{M_1}(x)=1\) for all \(x\)); then \(A^*(M_1)=1\geq 1-\delta\) (aligned). Let \(M_2\) be the constant-0 function (\(f_{M_2}(x)=0\) for all \(x\)); then \(A^*(M_2)=0<1-\delta\) for any \(\delta<1\) (misaligned). Both are trivially expressible in any Turing-complete transformer regime [5], [7]. In the running example (Section 3.1), \(f_{0.04}\) and \(f_{0.06}\) play the identical role: one satisfies \(A^*\geq 0.95\), the other does not, and both are in the same model class.
Both conditions satisfied, Rice’s Theorem applies: the language \(L_{A^*}=\{\langle T\rangle\mid T\in\mathcal{F},A^*(T)\geq 1-\delta\}\) is undecidable—no Turing machine halts with the correct verdict for all inputs, and T fails unconditionally.
Remark 2. Deployed LLMs with fixed context windows and finite numerical precision are not Turing-complete in the strict sense required by Rice’s Theorem. For those systems, part (i)—NP-hardness of ReLU-network property verification—is the operative impossibility result. Part (ii) applies to transformer variants that have been shown to operate in a Turing-complete regime [5], [7]; the undecidability claim is scoped to those architectures.
◻
4.2 The Statistical Barrier↩︎
A tractable verifier issues at most polynomially many queries—a finite body of evidence from which no infinite-domain property can be certified, since an indistinguishable misaligned system can always be constructed outside the queried region.
Theorem 2 (Statistical Barrier). Under Assumptions 1 and 2, any verification procedure satisfying both Soundness and Tractability cannot satisfy Completeness.
Proof. Any tractable \(\mathcal{V}\) terminates in \(p(|\theta|)\) steps and issues at most \(p(|\theta|)\) queries, yielding finite support \(S\subset\mathcal{X}\) with \(|S|\leq p(|\theta|)\) [22].
Step 1: a query-free gap always exists. For the threshold family \(\mathcal{F}_\tau\) (Section 3.1), \(n=|S|\) query points partition \([0,1]\) into \(n+1\) intervals; the pigeonhole principle guarantees at least one interval \(I^*\) of width \(\geq 1/(n+1)\) containing no query point. Any pair \(\theta_1\leq\tau<\theta_2\) placed within \(I^*\) responds identically on all of \(S\) yet disagrees in alignment status—a witness derived from the geometry of the model class alone, without invoking any assumption. Assumption 2 extends this structural fact to arbitrary model classes: for any finite \(S\), witnesses \(\theta_1,\theta_2\) exist with \(f_{\theta_1}(x)=f_{\theta_2}(x)\) for all \(x\in S\), yet \(A^*(\theta_1)\neq A^*(\theta_2)\). Take WLOG \(A^*(\theta_1)\geq 1-\delta\) (aligned) and \(A^*(\theta_2)<1-\delta\) (misaligned).
Step 2: identical behaviour forces identical verdicts. Since \(\mathcal{V}\) queries only \(S\) and both systems respond identically on all of \(S\), we have \(\mathcal{V}(\theta_1)=\mathcal{V}(\theta_2)\).
Step 3: soundness forces incompleteness. If both are certified: \(\theta_2\) (misaligned) is falsely certified—Soundness violated. If both are rejected: \(\theta_1\) (aligned) is falsely rejected—Completeness violated. Since \(\mathcal{V}\) is assumed sound, it cannot certify \(\theta_2\), so it rejects \(\theta_1\) as well. Completeness fails. ◻
This is the PAC-theoretic impossibility [9], [10] applied to alignment: for any finite query set, indistinguishable aligned and misaligned systems exist, as the proof above constructs directly.
4.3 Structural Independence of the Two Barriers↩︎
The two barriers arise from entirely distinct sources.
Theorem 1 is a computation barrier. It states that even a verifier with unlimited time cannot decide the required semantic property for all systems in polynomial time. It is activated by the expressivity of the model class (Assumption 1) and holds regardless of how many queries the verifier issues or what sampling strategy it uses. It would apply even if Assumption 2 were dropped.
Theorem 2 is an information barrier. It states that a poly-time verifier cannot gather enough evidence to certify alignment, because the observed input set is always a strict finite subset of \(\mathcal{X}\). It is activated by Assumption 2 and holds regardless of computational power: an oracle-equipped verifier still faces it if restricted to finitely many queries. It would apply even if the model class were far simpler.
We now state independence formally, supplying concrete witness systems for each direction.
Proposition 1 (Formal Independence). The two barriers are structurally independent: resolving either one leaves the other fully intact.
**Resolving the semantic barrier does not resolve the statistical barrier.* Suppose \(\mathrm{P}=\mathrm{NP}\) (the ReLU-network semantic barrier dissolves). A hypothetically sound, complete, tractable semantic verifier still terminates in \(p(|\theta|)\) steps and issues \(\leq p(|\theta|)\) queries. Assumption 2 is unchanged; the proof of Theorem 2 applies without modification, so Completeness still fails. Witness (running example, Section 3.1): even an NP-oracle-equipped verifier cannot distinguish \(f_{0.04}\) from \(f_{0.06}\) without a query inside the critical interval \((0.04,0.06)\)—a gap that \(\mathrm{P}=\mathrm{NP}\) does not close.*
**Resolving the statistical barrier does not resolve the semantic barrier.* Suppose an oracle reveals, for any \(\theta_1,\theta_2\), an input \(x\) on which \(f_{\theta_1}\) and \(f_{\theta_2}\) differ (the statistical gap is closed by assumption). The verifier must still decide the semantic property \(A^*(f_\theta)\geq 1-\delta\) at those inputs. For ReLU networks this decision problem remains NP-hard [1]; for Turing-complete architectures it remains undecidable by Rice’s Theorem. Witness (running example): knowing that the critical interval is \((0.04,0.06)\) does not tell a verifier whether \(\theta\leq 0.05\); it must still evaluate \(f_\theta\) on a continuous input subset, which requires computing the threshold value.*
4.4 Constructive Lower Bound↩︎
The trilemma establishes that a perfect verifier cannot exist. The following result quantifies the minimum coverage gap any sound, tractable verifier must accept.
Proposition 2 (Coverage Gap Lower Bound). Let \(\mathcal{V}\) be any deterministic, sound, tractable verifier issuing at most \(n\) queries. Over the threshold model class \(\mathcal{F}_\tau\), there exists a placement of the alignment boundary \(\tau\) such that \(\mathcal{V}\) fails Completeness: all aligned systems \(f_{\theta'}\) with \(\theta'\) in an interval of measure at least \(\frac{1}{2(n+1)}\) are incorrectly rejected.
Proof. Any \(n\)-query verifier places query points at \(x_1<\cdots<x_n\), partitioning \([0,1]\) into at most \(n+1\) open intervals. By pigeonhole, at least one interval \(I^*=(x_i,x_{i+1})\) has width \(\geq 1/(n+1)\). Place the alignment boundary \(\tau\) at the midpoint of \(I^*\): \(\tau = (x_i + x_{i+1})/2\). Then for all \(\theta\in (\tau, x_{i+1})\) (misaligned) and all \(\theta'\in(x_i,\tau]\) (aligned), both \(f_\theta\) and \(f_{\theta'}\) return identical outputs at every query point. A sound verifier cannot certify the misaligned \(f_\theta\), so it must withhold certification from the entire ambiguous interval—failing Completeness on the aligned half \((x_i,\tau]\), which has width \(= |I^*|/2 \geq \frac{1}{2(n+1)}\), establishing the claimed bound. Running example: with \(n=9\) uniform queries at \(\{0.1,0.2,\ldots,0.9\}\) and \(\tau=0.05\), the leftmost gap is \((0,0.1)\) (width \(0.1\geq 1/10\)); the aligned systems with \(\theta'\in(0,0.05]\) are all incorrectly rejected. ◻
Remark 3. For networks with \(d\) parameters and depth \(L\), the number of critical regions grows as \(R=\Omega\bigl((n/L)^{L(d-1)}\bigr)\) [21]. Each region is a connected, open set in \(\mathcal{X}\); any query set \(S\) of size \(n\) can intersect at most \(n\) regions. The remaining \(R-n\) regions are entirely unqueried, and at least one contains an alignment boundary (by an argument identical to the proof above applied within that region’s parameter slice). Since \(R-n\geq R/2\) for \(n\leq R/2\)—which holds for any poly-\((|\theta|)\) query count \(n\) once \(d\) exceeds a constant threshold—the number of unqueried regions grows super-polynomially in \(d\), and the fraction of aligned parameter space that any poly-query verifier fails to cover remains strictly positive.
5 The Alignment Verification Trilemma↩︎
The trilemma follows directly from the two theorems.
Corollary 1 (Alignment Verification Trilemma). Under Assumptions 1 and 2, no verification procedure \(\mathcal{V}\) can simultaneously satisfy Soundness (S), Completeness (C), and Tractability (T).
Proof. Suppose \(\mathcal{V}\) satisfies any two of the three. S+C: By Theorem 1, T fails. S+T: By Theorem 2, C fails. C+T: \(\mathcal{V}\) must certify \(\theta_1\) (aligned, by C) and is restricted to queries in \(S\) (by T); the construction in Theorem 2 yields identical responses from \(\theta_2\) (misaligned) on all of \(S\), so \(\mathcal{V}\) certifies \(\theta_2\) as well. S fails. ◻
Each pair is achievable by a concrete procedure.
Proposition 3 (Pairwise Achievability). Each pair (S+C), (S+T), (C+T) is simultaneously achievable.
**S+C without T: SMT-based solvers (Reluplex, Marabou) are sound and complete for linear specifications over bounded networks. Worst-case runtime is exponential; T fails.
**S+T without C: Bound-propagation tools (\(\alpha\)-\(\beta\)-CROWN [23]) certify properties over bounded input domains in polynomial time. Coverage is restricted to \(\mathcal{D}_b\subsetneq\mathcal{X}\); full-domain C fails.
**C+T without S: Any uniformly-applied proxy score (e.g.RLHF reward threshold) evaluates all systems tractably. Completeness holds relative to the proxy objective: every system meeting the proxy threshold is certified. The proxy-to-\(A^*\) gap is structural [15], [16], so Soundness relative to \(A^*\) fails.
6 What Remains Possible↩︎
Each relaxation of one trilemma property restores a well-defined certification regime with known tools, known coverage, and known limits.
6.0.0.1 Relaxing T: Sound and Complete Formal Verification.
Dropping the polynomial-time requirement, SMT-based tools—Reluplex, Marabou, \(\alpha\)-\(\beta\)-CROWN in complete mode—provide sound and complete verification for linear specifications over bounded finite-precision networks. Runtime is exponential in the worst case but practical for moderate-sized components: embedded controllers, narrow verifiable submodules, safety-critical classifiers with bounded input domains. The honest deployment of this regime requires that the verified network genuinely represents the deployed system, and that the verified specification genuinely represents the alignment requirement.
6.0.0.2 Relaxing C: Sound and Tractable Bounded Verification.
Restricting to \(\mathcal{D}_b\subsetneq\mathcal{X}\)—a compact region representing expected deployment conditions—makes sound, tractable certification achievable. The resulting guarantee is: the system meets the alignment specification on every input in \(\mathcal{D}_b\). In practice, ensuring that \(\mathcal{D}_b\) accurately covers the relevant deployment inputs is harder to guarantee than the verification computation itself. Distribution shift between \(\mathcal{D}_b\) and runtime inputs falls outside what the certificate covers. The appropriate question for this regime is: how large a \(\mathcal{D}_b\) can be verified soundly and tractably for a given model size, and how much of the alignment-relevant input space does it cover?
6.0.0.3 Relaxing S: Complete and Tractable Statistical Assurance.
Replacing soundness with a probabilistic guarantee—the probability of certifying a misaligned system is at most \(\varepsilon\)—yields statistical assurance from \(n\) test inputs, with \(n\) determined by standard concentration results. All current alignment evaluation methods operate in this regime: RLHF scores, red-teaming evaluations, benchmark performance, constitutional-AI audits. The practical challenge in this regime is that no ground-truth \(A^*\) is available for calibrating \(\varepsilon\); evaluated systems are assessed against proxy objectives whose gap from \(A^*\) is bounded only empirically [16], not formally.
6.0.0.4 The Pareto Frontier.
The three regimes are not isolated points but corners of a continuous achievable space. For the running example (Section 3.1), the miss-rate lower bound is computable in closed form. In the S+T regime, Proposition 2 establishes that any sound, deterministic verifier using \(n\) queries has a miss rate of at least \(\frac{1}{2(n+1)}\) (for adversarial placement of \(\tau\)); Figure 3 (a) plots this lower bound as \(\varepsilon = 1-c\) against coverage fraction \(c\) (since \(n/(n+1)\) uniform queries cover approximately fraction \(c\), giving miss rate \(\approx 1-c\)). In the S+C regime (relaxing T), covering a fraction \(c\) of \(\mathcal{X}\) soundly requires resolving the linear activation regions in the corresponding input subspace; the number of such regions grows as \(\Omega\bigl((n/L)^{L(d{-}1)}\bigr)\) [21], and cost grows at least proportionally, diverging as \(c\to 1\). Figure 3 plots both curves together with empirical measurements on a concrete trained network.
6.0.0.5 Empirical confirmation.
To ground Corollary 1 on a real trained network, we train a two-layer ReLU network (\(\mathrm{Linear}(2{\to}32)\to\mathrm{ReLU}\to\mathrm{Linear}(32{\to}1)\)) on the safety property \(f(\mathbf{x})\geq 0.5\) for all \(\mathbf{x}\in[0.5,1]^2\), achieving 95% grid accuracy. We then apply Interval Bound Propagation (IBP [24]), a sound verifier that propagates axis-aligned intervals through the network layer by layer and returns a guaranteed lower bound on the output over the entire input region—never a false positive. We sweep over ten nested sub-boxes of the safe region, from 4% to 100% coverage, and record whether IBP certifies each.1 Results appear as coloured points in Figure 3. IBP certifies up to 49% of the safe region (S+T confirmed); beyond 64% its interval bounds cross the certification threshold and it correctly returns inconclusive rather than lying (S preserved, C lost). Full-domain certification fails with a lower bound of \(0.0002\)—well below 0.5—quantifying the gap that Theorem 2 guarantees must exist.
![Figure 3: Pareto frontier and trilemma proof (Cor. 1).Network: \mathrm{Lin}(2{\to}32)\to\mathrm{ReLU}\to\mathrm{Lin}(32{\to}1);property: f(\mathbf{x})\geq 0.5, \mathbf{x}\in[0.5,1]^2;verifier: IBP [24].(a) S+T. \varepsilon=1{-}c (theoretical, uniform queries). Green \bullet: IBP certifies; red \times: inconclusive (S kept, C lost).(b) S+C. IBP bound collapses below 0.5 beyond c=49\% (Thm. 2).(c) All three regimes in one plot: S+T (green step) loses C; C+T (blue dashed) loses S; S+C (orange \lozenge) loses T; S+C+T (\star) is unreachable.](/article/2603.08761/_pdflatex/qzsckifl.png "fig:")
The core open questions the frontier defines are: what is the tightest \(\varepsilon\) achievable by a sound tractable verifier over \(\mathcal{D}_b\) for current-scale LLMs; at what coverage fraction does S+C verification become intractable for deployed systems; what joint bound is achievable by combining mechanistic interpretability with bounded verification; and how do all three answers shift as model scale and tooling improve.
7 Implications for Alignment Practice↩︎
7.0.0.1 Every alignment method occupies a known corner.
RLHF, DPO, and constitutional evaluation are C+T methods: broadly applicable and tractable, without soundness. Formal verification tools are S+C methods: sound and complete, without tractability at scale. Bounded verification is the S+T regime: sound and tractable, without full-domain coverage. Knowing which corner one occupies is a prerequisite for accurate safety communication.
7.0.0.2 Formal status of published safety claims.
Table 2 analyses specific, sourced claims from deployed systems. Each claim is an accurate description of the cited evaluation regime. The “Does not establish” column identifies which trilemma property the cited evaluation was not designed to establish—not that any claim is false, but that none constitutes a certificate over full \(\mathcal{X}\).
3pt
| Published claim | Does not establish | Formal status |
|---|---|---|
| GPT-4 is 82% less likely to produce disallowed content vs.GPT-3.5 [25] | S | Measured on internal red-team set \(\mathcal{D}_b\); no bound on behaviour outside \(\mathcal{D}_b\) |
| Gemini Ultra achieves top scores on safety benchmarks [26] | C | Benchmarks are finite \(\mathcal{D}_b\); out-of-benchmark behaviour uncertified |
| InstructGPT is “more helpful, truthful, and harmless” than GPT-3 [12] | S | Comparison is relative to a proxy reward; proxy-to-\(A^*\) gap is structural [16] |
| Constitutional AI is “broadly safe” across evaluation suite [13] | C+S | Safe on tested prompts (C fails on remainder); proxy score used for training (S gap) |
7.0.0.3 Scaling does not dissolve either barrier.
Better hardware makes S+T tools applicable to larger \(\mathcal{D}_b\), but the statistical barrier remains: soundness over full \(\mathcal{X}\) requires completeness to fail for any tractable procedure. Improved proxy objectives narrow the C+T regime’s practical soundness gap, but the gap remains structural so long as \(A^*\) is defined over full \(\mathcal{X}\). Neither barrier is a limitation of current technique that will eventually be engineered away.
7.0.0.4 What a safety claim must specify.
Any safety claim in alignment should specify: (a) which of S, C, T is relaxed; (b) what the remaining guarantee covers—which \(\mathcal{D}_b\), which \(\varepsilon\), which specification; and (c) what falls outside coverage. This is not a counsel of despair. It is the minimum content of an engineering safety argument. Alignment verification is better understood as structured risk management within a well-defined certification space than as progress toward a certificate that the results above show cannot exist.
8 Conclusion↩︎
The question with which this paper began—could we, in principle, know that an AI system is aligned?—has a structured answer.
The semantic barrier shows that no sound and complete verifier can terminate in polynomial time. The statistical barrier shows that no sound and tractable verifier can be complete. Together they entail the trilemma: soundness, completeness, and tractability cannot all hold simultaneously.
The structure matters. An unstructured impossibility offers no guidance. A structured one—with two independent barriers, three achievable pairwise regimes, and a Pareto frontier of partial guarantees— converts impossibility into a map. Each point on the frontier corresponds to a research programme: bounded formal verification, statistical assurance with calibrated soundness, interpretability methods that close the gap between observed behaviour and in-weight goal structure.
The central open problem the trilemma defines is not whether full certification is possible—it is not—but where the strongest achievable guarantee sits on the Pareto frontier: how much soundness can be retained when tractability or completeness is partially relaxed, and how that frontier shifts as model scale, verification tooling, and interpretability methods improve. These questions can now be stated formally. Their answers are not yet known.
References↩︎
Reproducibility details: Adam optimiser, learning rate \(10^{-3}\), 5 000 gradient steps, mini-batches of 128 safe and 64 unsafe samples per step; no fixed random seed (results are stable across runs due to the large training set relative to network size). Code is available via Anonymous GitHub: https://anonymous.4open.science/r/code-7B94/.↩︎