1 Introduction↩︎

Large language models (LLMs) are increasingly accessed through black-box APIs: users submit prompts and observe generated responses, while model weights, gradients, training data, and internal activations remain hidden. In many cases, even token-level probability information is unavailable or only partially exposed [1]–[3]. This interface creates two closely related settings. In benign settings, organizations may distill larger models into smaller students to reduce latency, lower inference cost, enable on-device deployment, or adapt behavior to a specific domain. In adversarial settings, the same prompt-response interface can be used for model extraction, where an external party trains a surrogate model to imitate a deployed teacher. In both cases, the core question is the same: how closely does the student’s observable behavior match the teacher’s?

Black-box LLM distillation is usually evaluated as an output-matching problem: a student is considered successful when its responses are semantically similar to, task-consistent with, or lexically close to those of the teacher. These metrics are useful, but they do not answer a more operational question:

Can an adversary tell whether a response was generated by the teacher or by the distilled student?

This distinction matters because output similarity can hide behavioral differences. A medical assistant may preserve the main answer while omitting cautionary language; a coding assistant may reproduce the algorithm while revealing systematic formatting artifacts; an enterprise chatbot may match factual content while differing in refusal behavior, privacy warnings, or instruction-conflict resolution. In such cases, semantic similarity may overstate behavioral transfer.

We introduce bounded behavioral indistinguishability for black-box LLM distillation. A student \(S\) is \((\epsilon,q,t,\mathbb{A})\)-behavioral indistinguishable from a teacher \(T\) if no adversary in a specified class \(\mathbb{A}\), operating with query budget \(q\) and computational budget \(t\), can distinguish teacher outputs from student outputs with advantage greater than \(\epsilon\) over an explicit prompt distribution. This makes indistinguishability a bounded empirical claim rather than an absolute property of the model: behavioral transfer is measured by the residual advantage of bounded adversaries, not by output similarity alone.

This perspective yields an evaluation methodology based on empirical distinguishing advantage. Instead of relying on a single similarity metric, we evaluate teacher–student pairs using complementary adversarial and behavioral tests: learned discriminators, semantic similarity, category-wise probes, policy-level agreement, and pairwise teacher-identification judges. Each test captures a different residual signal. Response-only discriminators may detect style or length artifacts; prompt-response discriminators test whether responses are teacher-like in context; policy-level evaluators focus on safety-relevant behavior; and pairwise judges ask whether, given two responses to the same prompt, the teacher-generated output can be identified.

We instantiate this approach on Qwen [4] and Llama [5] teacher–student pairs. For each family, we compare the teacher against both the original base student and a LoRA-distilled student trained on teacher-generated black-box responses using Low-Rank Adaptation [6]. Our controlled 5,000-prompt behavioral probe suite covers general question answering, reasoning, coding, summarization, style and format control, ambiguous prompts, safety-boundary prompts, instruction conflict, domain-technical questions, and robustness perturbations. The suite is not intended to model a natural deployment distribution; rather, it provides a controlled probe distribution for measuring where behavioral imitation succeeds and where distinguishable artifacts remain.

Recent work has cautioned that indistinguishability-style measurements should be interpreted within a specified threat model, rather than treated as universal guarantees [3]. Our results support this bounded view: LoRA distillation improves semantic similarity across both Qwen and Llama, and learned discriminators become less effective at separating teacher outputs from student outputs, but residual distinguishability remains.

In a pairwise teacher-identification experiment using a different-family Llama judge with A/B-swap consistency filtering, Qwen distinguishing advantage drops from \(0.158\) for the base student to \(0.081\) after LoRA distillation. This shows that distillation makes teacher identification harder, but not impossible. Category-wise analysis, especially under the pairwise judge, shows that residual artifacts are most visible in style/format, robustness perturbation, and domain-technical prompts. Finally, query-budget experiments show that disagreement-guided acquisition does not consistently outperform stratified random sampling, suggesting that coverage and diversity remain strong baselines for behavioral distillation. Figure 1 summarizes the proposed pipeline.

We summarize our contributions below:

• We introduce bounded behavioral indistinguishability for black-box LLM distillation, formalized as \((\epsilon,q,t,\mathbb{A})\)-behavioral indistinguishability such that \(\epsilon\) bounds distinguishing advantage over an explicit prompt distribution, query budget \(q\), computational budget \(t\), and adversary class \(\mathbb{A}\).

• We develop an empirical evaluation methodology based on distinguishing advantage, combining learned discriminators, semantic similarity, category-wise probes, policy-level measurements, and pairwise teacher-identification judges.

• We evaluate Qwen and Llama teacher–student pairs using a controlled \(5,000\)-prompt behavioral probe suite, showing that LoRA distillation improves semantic similarity and reduces, but does not eliminate, empirical distinguishability.

• We introduce a pairwise teacher-identification evaluation with A/B-swap consistency filtering, showing that Qwen distinguishing advantage decreases from \(0.158\) for the base student to \(0.081\) after LoRA distillation.

• We show that residual distinguishability is category-dependent and that disagreement-guided query acquisition does not consistently outperform stratified random sampling, highlighting the importance of coverage and diversity in black-box behavioral distillation.

Overall, our work argues that semantic fidelity is necessary but insufficient for evaluating black-box LLM distillation. Bounded behavioral indistinguishability provides a formal and empirical lens for measuring how much teacher-like behavior has been transferred, which adversaries can still detect residual artifacts, and where those artifacts are concentrated. Code, data, and evaluation scripts are open source and available on GitHub¹

2 Related Work↩︎

2.0.0.1 LLM distillation and black-box imitation:

Knowledge distillation transfers behavior from a larger teacher model to a smaller student model [7]. Early work focused on model compression for classifiers and transformers, including DistilBERT [8] and TinyBERT [9]. Recent LLM distillation methods have shifted towards training student models on teacher-generated responses [10], [11]. This black-box setting is closely related to model imitation and extraction, where a substitute model is trained from query access to a target model [12], [13]. Our work does not introduce a new distillation objective; instead, it studies how to evaluate whether a distilled student remains distinguishable from the teacher it imitates.

2.0.0.2 LLM evaluation beyond semantic similarity:

Distilled LLMs are commonly evaluated using task accuracy, lexical overlap, embedding similarity, or response-level agreement. Sentence embedding models provide scalable estimates of semantic similarity [14], but semantic closeness does not necessarily imply behavioral equivalence. Two responses may preserve meaning while differing in formatting, refusal behavior, cautionary language, or instruction-following style. Our work therefore treats semantic similarity as one component of behavioral fidelity and adds adversarial distinguishability tests to measure residual teacher–student artifacts.

2.0.0.3 Adversarial and game-based evaluation:

Indistinguishability games are central in cryptography, where security is defined through the advantage of an adversary interacting with a challenger [15]–[17]. We adapt this style of reasoning to empirical LLM distillation. Our notion of \((\epsilon,q,t,\mathbb{A})\)-behavioral indistinguishability bounds distinguishing advantage relative to a prompt distribution, query budget, computational budget, and adversary class. This makes the evaluation claim explicit: a student is indistinguishable only with respect to the adversaries and conditions being tested.

2.0.0.4 LLM-as-judge and pairwise evaluation:

LLM-based judges are widely used to evaluate generative models, especially through pairwise comparisons [18]–[20]. However, automated judges may exhibit position, verbosity, formatting, or model-family bias. In our setting, the pairwise judge is not used as a human preference proxy; it is an explicit adversary that attempts to identify which response is teacher-generated.

2.0.0.5 Policy-level behavior and bounded assurance:

Safety-relevant LLM behavior is often evaluated through refusal behavior, cautionary language, privacy warnings, and policy-boundary compliance [21], [22]. These properties are not fully captured by semantic similarity. Our framework separates textual distinguishability from policy-level behavior through an evaluator \(\pi(x,y)\). This bounded interpretation is consistent with a broader view of AI safety verification, in which safety and policy claims for complex AI systems are stated relative to explicit models, evaluators, and assumptions rather than treated as universal guarantees [23].

2.0.0.6 Query selection for distillation.

Black-box distillation depends on which prompts are used to query the teacher. Active learning and core-set methods study how to select informative examples under labeling or query budgets [24]–[27]. Our query-budget experiments compare stratified random sampling with disagreement-guided acquisition. The results suggest that disagreement alone is not enough: coverage and category diversity remain strong baselines for behavioral distillation.

3 Bounded Behavioral Indistinguishability↩︎

We now formalize black-box LLM distillation as a bounded behavioral indistinguishability problem. The goal is not to prove cryptographic indistinguishability in the absolute sense. Instead, we define an operational notion of indistinguishability under a specified prompt distribution, finite query budget, and bounded class of adversarial evaluators. This framing allows us to evaluate whether a distilled student merely improves average similarity metrics, or whether its outputs become difficult to distinguish from those of the teacher under practical behavioral tests.

3.1 Black-Box Distillation↩︎

Let \(T:\mathcal{X}\rightarrow\mathcal{Y}\) denote the teacher model, where \(x\in\mathcal{X}\) is a prompt and \(T(x)\in\mathcal{Y}\) is the generated teacher response. Let \(S:\mathcal{X}\rightarrow\mathcal{Y}\) denote a candidate student model. In our evaluation, \(S\) is either the original base student \(S_{\mathrm{base}}\) or the LoRA-distilled student \(S_{\mathrm{LoRA}}\).

In the black-box distillation setting, the learner does not have access to the teacher’s weights, logits, hidden states, gradients, training data, or decoding distribution. It only observes prompt-response pairs obtained by querying the teacher. Given a query budget, the learner constructs a distillation set \[\mathcal{Q}_n =\{(x_i,T(x_i))\}_{i=1}^{n},\] where \(x_i\in\mathcal{X}\) are selected prompts and \(T(x_i)\) are the corresponding teacher responses.

The student is trained by supervised fine-tuning on \(\mathcal{Q}_n\). In our LoRA setting, the base student weights are frozen and only the adapter parameters \(\theta\) are updated. The adapter parameters are learned by minimizing the negative log-likelihood of the teacher response: \[\theta^{\star} = \arg\min_{\theta} \sum_{(x,y)\in \mathcal{Q}_n} -\log p_{\theta}(y \mid x).\] The resulting LoRA-adapted model is denoted \(S_{\mathrm{LoRA}}\). The goal of distillation is not exact reproduction. Natural language generation is high-dimensional, stochastic, and sensitive to decoding choices. Thus, exact equality between \(T(x)\) and \(S(x)\) is not an appropriate notion of successful distillation. Instead, we ask whether the student approximates the teacher’s observable behavior under a specified evaluation regime. Let \(\mathcal{P}\) denote an evaluation prompt distribution over \(\mathcal{X}\). In our experiments, \(\mathcal{P}\) is induced by the controlled behavioral probe suite. Thus, sampling \(x\sim\mathcal{P}\) corresponds to drawing an evaluation prompt from the behavioral regions covered by the suite. All empirical indistinguishability claims in this paper are therefore relative to this prompt distribution.

For a behavioral discrepancy function denoted by: \(\Delta:\mathcal{Y}\times\mathcal{Y}\rightarrow\mathbb{R}_{\geq 0}\), one can measure the expected teacher–student discrepancy as \[\mathcal{E}_{\mathcal{P}}(T,S) = \mathbb{E}_{x\sim\mathcal{P}} \left[ \Delta(T(x),S(x)) \right].\] Examples of \(\Delta\) include embedding distance, refusal mismatch, length mismatch, format mismatch, policy-label mismatch, or task-specific behavioral differences. Such metrics are useful, but they remain metric-level summaries: they do not directly answer whether an adversary can detect that a response came from the student rather than the teacher.

3.2 Game-Based Indistinguishability↩︎

We define a game-based notion of bounded behavioral indistinguishability. The game is played between a challenger and an adversary \(\mathcal{A}\). The challenger has access to two models, the teacher \(T\) and the candidate student \(S\), and samples a hidden bit indicating which model will answer the adversary’s queries. The adversary observes responses from the selected model and attempts to identify whether it is interacting with the teacher or the student. The prompt distribution \(\mathcal{P}\) is treated as part of the evaluation setting and is not included explicitly in the game notation. The adversary may issue at most \(q\) prompts from \(\mathcal{X}\), drawn from or selected with respect to \(mathcal{P}\). Prompts may be chosen non-adaptively or adaptively based on previous responses.

The adversary wins the game if \(b'=b\). For a fixed adversary \(\mathcal{A}\), we define its distinguishing advantage as follows: \[\mathrm{Adv}^{\mathsf{dist}}_{T,S}(\mathcal{A},q,t) = \left| \Pr[b'=b] - \frac{1}{2} \right|,\] where the probability is taken over the challenger bit \(b\), any randomness in model generation, the sampled or selected prompts under \(\mathcal{P}\), and any internal randomness of \(\mathcal{A}\).

In the LLM setting, computational budget \(t\) is instantiated by the resources available to the adversarial evaluator, including the discriminator architecture, training procedure, number of epochs, optimization budget, and evaluation budget. Thus, different choices of \(t\) correspond to different strengths of empirical adversaries.

This notion is intentionally bounded. It should not be interpreted as universal indistinguishability between \(T\) and \(S\). Rather, it specifies indistinguishability relative to an explicit evaluation regime i.e., a prompt distribution \(\mathcal{P}\), query budget \(q\), computational budget \(t\), and adversary class \(\mathbb{A}(q,t)\). In practice, \(\mathbb{A}(q,t)\) can be instantiated by empirical evaluators with different information access and strength, including response-only discriminators, prompt-response discriminators, embedding-based classifiers, policy evaluators, and pairwise judges.

3.3 Empirical Distinguishing Advantage↩︎

The game-based definition above specifies an ideal bounded distinguishing advantage with respect to an adversary class \(\mathbb{A}(q,t)\). In practice, we cannot quantify over all adversaries in this class. Instead, we estimate distinguishability using finite held-out prompt sets and a collection of learned discriminators that instantiate particular empirical adversaries. Let \(\mathcal{X}_{\mathrm{test}} = \{x_1,\ldots,x_m\}\) be a held-out prompt set sampled from the evaluation distribution \(\mathcal{P}\). For each prompt \(x_i\), we generate a teacher response \(y_i^T = T(x_i)\), and a student response \(y_i^S = S(x_i)\). This induces two empirical prompt-response sets: \[\widehat{\mathcal{D}}_T = \{(x_i,y_i^T)\}_{i=1}^{m}, \qquad \widehat{\mathcal{D}}_S = \{(x_i,y_i^S)\}_{i=1}^{m}.\] We treat \(\widehat{\mathcal{D}}_T\) and \(\widehat{\mathcal{D}}_S\) as empirical distributions over teacher-generated and student-generated prompt-response pairs, respectively, when training and evaluating discriminators. A learned discriminator \(D_{\phi} : \mathcal{X}\times\mathcal{Y} \rightarrow \{0,1\}\), serves as an empirical adversary. It is trained to predict whether a prompt-response pair was generated by the teacher or the student. We assign label \(0\) to teacher outputs and label \(1\) to student outputs. For a balanced evaluation set, the empirical discriminator accuracy is: \[\begin{align} \widehat{\mathrm{Acc}}(D_{\phi}) = \frac{1}{2m} \sum_{i=1}^{m} \Bigl[ &\mathbf{1}\{D_{\phi}(x_i,y_i^T)=0\} \\ &+ \mathbf{1}\{D_{\phi}(x_i,y_i^S)=1\} \Bigr]. \end{align}\] Under balanced teacher—student labels, the null hypothesis of indistinguishable outputs corresponds to discriminator accuracy \(1/2\). We therefore define the empirical distinguishing advantage of \(D_{\phi}\) as follows: \[\label{eq:empirical-distinguishing-advantage} \widehat{\mathrm{Adv}}^{\mathsf{dist}}_{T,S}(D_{\phi}) = \left| \widehat{\mathrm{Acc}}(D_{\phi}) - \frac{1}{2} \right|,\tag{1}\] where, \(D_{\phi}\) instantiates an adversary under fixed \(q, t\) (refer Definition 2). In Equation 1 , an accuracy of \(0.50\) corresponds to zero empirical advantage, while an accuracy of \(0.60\) corresponds to empirical advantage \(0.10\).

For a finite collection of empirical discriminators: \(\widehat{\mathbb{A}} = \{D_{\phi_1},\ldots,D_{\phi_k}\} \subseteq \mathbb{A}(q,t)\), we report the strongest observed empirical distinguishing advantage: \[\widehat{\mathrm{Adv}}^{\mathsf{dist}}_{T,S}(\widehat{\mathbb{A}}) = \max_{D_{\phi}\in \widehat{\mathbb{A}}} \widehat{\mathrm{Adv}}^{\mathsf{dist}}_{T,S}(D_{\phi}).\]

This discriminator-based evaluation complements conventional distillation metrics. Metrics such as embedding similarity, BLEU, ROUGE, format agreement, and refusal agreement measure predefined dimensions of teacher–student closeness. However, they do not directly test whether residual differences are detectable by an adversarial evaluator. Empirical distinguishing advantage addresses this question by measuring how reliably a discriminator can separate teacher-generated from student-generated responses. Thus, two distilled models may achieve similar semantic similarity while exhibiting different levels of behavioral distinguishability. We instantiate empirical adversaries with different information access as given below:

• Prompt-only discriminators, which receive \(x\) but not the response. These serve as leakage controls and should perform near chance if teacher and student examples are constructed from the same prompt set.

• Response-only discriminators, which receive \(y\) but not the prompt. These test whether student outputs contain prompt-independent artifacts such as differences in style, length, formatting, or generation patterns.

• Prompt-response discriminators, which receive \((x,y)\). These test whether the response is teacher-like in the context of the prompt.

• Embedding discriminators, which operate on embeddings of responses or prompt-response pairs. These test whether teacher and student outputs remain separable in semantic representation space.

This framework also permits stronger pairwise judges, which receive \((x,y_1,y_2)\), where one response is generated by the teacher and the other by the student, and predict which response is teacher-generated. We treat this as a stronger empirical adversary class and evaluate it separately in Section 5.3.

Evaluating multiple empirical adversaries is important because bounded behavioral indistinguishability is relative to the adversary class and to the information available to the adversary. A student may be difficult to distinguish under embedding-based tests while remaining separable under response-style or prompt-conditioned discriminators. Conversely, a student may be textually distinguishable while preserving policy-relevant behavior. We therefore report distinguishability across multiple discriminator families rather than treating empirical indistinguishability as a single model-level scalar.

3.4 Policy-Level Indistinguishability↩︎

For safety-relevant distillation, textual similarity alone is insufficient. A student may produce responses that are semantically close to the teacher while differing in refusal behavior, cautionary language, privacy warnings, or recommendations to seek professional advice. Conversely, a student may be textually distinguishable while preserving the same policy-level decisions. To capture this distinction, let \(\pi : \mathcal{X} \times \mathcal{Y} \rightarrow \mathcal{Z}\) be a policy or behavioral evaluator that maps a prompt-response pair to a policy-relevant label or score. The codomain \(\mathcal{Z}\) may be binary, categorical, or real-valued depending on the evaluator. Examples include:

• whether the response complies or refuses,

• whether it contains medical, legal, or financial caution,

• whether it recommends professional consultation,

• whether it reveals or requests private information,

• whether it satisfies a required output format,

• whether it follows or violates a safety boundary.

For a prompt \(x\), the teacher and student induce policy outcomes as: \[z_T(x) = \pi(x,T(x)), \qquad z_S(x) = \pi(x,S(x)).\] We define policy disagreement over a prompt distribution \(\mathcal{P}\) as \[\label{eq:bound-indistinguishinility:policy-disagreement} \mathcal{E}^{\pi}_{\mathcal{P}}(T,S) = \Pr_{x\sim \mathcal{P}} \left[ \pi(x,T(x)) \neq \pi(x,S(x)) \right].\tag{2}\] When \(\pi\) is real-valued rather than categorical, the disagreement term can be replaced by an appropriate distance or a threshold mismatch. Policy disagreement measures whether the teacher and student induce the same policy-level outcomes under a fixed evaluator. We can also define a policy-level analogue of the distinguishing game in which the adversary observes policy outcomes rather than raw text. Let \(\mathbb{A}_{\pi}(q,t)\) denote a class of policy-level adversaries that make at most \(q\) queries and run within computational budget \(t\). For a fixed policy adversary \(\mathcal{A}_{\pi}\), we define: \[\mathrm{Adv}^{\mathsf{policy}}_{T,S}(\mathcal{A}_{\pi},q,t) = \left| \Pr[b'=b] - \frac{1}{2} \right|,\] where the challenger samples \(b\leftarrow\{0,1\}\), returns \((x,\pi(x,T(x)))\) when \(b=0\), returns \((x,\pi(x,S(x)))\) when \(b=1\), and \(\mathcal{A}_{\pi}\) outputs a guess \(b'\). The corresponding class-level policy distinguishing advantage is \[\mathrm{Adv}^{\mathsf{policy}}_{T,S}(\mathbb{A}_{\pi},q,t) = \sup_{\mathcal{A}_{\pi}\in\mathbb{A}_{\pi}(q,t)} \mathrm{Adv}^{\mathsf{policy}}_{T,S}(\mathcal{A}_{\pi},q,t).\] This notion separates surface-level imitation from safety-relevant behavioral preservation. A distilled model may fail to match the teacher’s exact wording while still preserving policy-level behavior. Alternatively, it may achieve high semantic similarity while failing to preserve safety-critical decisions. In our experiments, we therefore report both textual distinguishability and policy-relevant behavioral agreement.

Bounded behavioral indistinguishability should be read as an empirical and operational notion. A low distinguishing advantage does not imply that the student and teacher are identical. It only indicates that, under the specified prompt distribution, finite sample size, adversary class, query budget, and computational budget, the evaluated adversaries cannot reliably separate teacher outputs from student outputs. This bounded interpretation is essential for avoiding overclaims while still providing a useful security-inspired evaluation lens for black-box LLM distillation.

4 Prompt Suite and Experimental Setup↩︎

We evaluate bounded behavioral indistinguishability using a controlled black-box distillation pipeline. This section describes the behavioral probe suite, teacher and student models, LoRA distillation protocol, empirical adversary instantiations, and evaluation metrics.

4.1 Behavioral Probe Suite↩︎

We construct a controlled behavioral probe suite containing \(5,000\) prompts spanning ten behavioral categories: general question answering, reasoning, coding, summarization, style and format control, ambiguous prompts, safety-boundary prompts, instruction conflict, domain-technical prompts, and robustness perturbations. The suite is designed to cover multiple observable dimensions of LLM behavior rather than to approximate a natural user-query distribution. Further, each prompt is annotated with metadata including category, subtype, domain, difficulty, risk level, constraint type, audience, and paraphrase group where applicable. This metadata supports category-wise evaluation and allows us to test whether residual distinguishability is concentrated in specific behavioral regions. Table 1 summarizes the category composition and behavioral purpose of the suite. We split the prompt suite into \(4,000\) training prompts and \(1,000\) held-out test prompts using a stratified \(80/20\) split over categories.

Since, the suite is generated from controlled templates and metadata, it may contain distributional regularities not present in natural user traffic. We therefore avoid making claims about universal behavioral equivalence. Instead, we treat the suite as a controlled probe distribution for estimating bounded empirical distinguishability.

4.2 Teacher and Student Models↩︎

We evaluate two open model families. In each family, a larger instruction-tuned model serves as the teacher and a smaller instruction-tuned model serves as the student.

1. Qwen family [4]: The teacher is Qwen2.5-3B-Instruct and the student is Qwen2.5-0.5B-Instruct.

2. Llama family [5]: The teacher is Llama-3.2-3B-Instruct and the student is Llama-3.2-1B-Instruct.

For each family of models, we compare two student variants:

1. Base student: the original smaller instruction-tuned model without additional distillation.

2. LoRA-distilled student: the smaller model after LoRA fine-tuning on teacher-generated responses.

This allows us to measure whether black-box distillation reduces teacher–student discrepancy relative to the original base student.

4.3 Teacher Response Generation↩︎

For every prompt \(x\) in the training and test splits, we query the corresponding teacher model to obtain a response \(T(x)\). The resulting prompt-response pairs define the observable black-box behavior of the teacher. Teacher responses are generated using bounded maximum response length and low-temperature decoding. We use a fixed decoding configuration within each model family to reduce variation due to sampling randomness. The student never receives teacher weights, logits, hidden states, gradients, or training data; it only receives the generated prompt-response pairs.

4.4 LoRA Distillation Protocol↩︎

Students are fine-tuned using low-rank adaptation (LoRA) on the 4,000 teacher-generated training responses. Each training example is formatted as an instruction-following conversation consisting of the user prompt and the teacher response. The training objective is standard causal language modeling over the teacher response tokens. Given the distillation set \(\mathcal{Q}_n = \{(x_i,T(x_i))\}_{i=1}^{n}\), the student is trained by supervised fine-tuning (SFT) to minimize: \[\mathcal{L}_{\mathrm{SFT}}(\theta) = -\sum_{(x,y)\in \mathcal{Q}_n} \log p_{\theta}(y \mid x).\] Only LoRA parameters are updated during distillation; the base model weights are kept frozen. \(\theta\) denotes the trainable LoRA adapter parameters. At the evaluation time, responses are generated from the base student and LoRA-distilled student using the same decoding configuration as far as possible. We then compare these responses to the corresponding teacher responses on the held-out test set.

4.5 Empirical Adversary Instantiations↩︎

To estimate bounded behavioral indistinguishability, we evaluate several empirical adversaries. Each discriminator is trained to distinguish teacher outputs from candidate student outputs. The candidate is either the base student or the LoRA-distilled student. These discriminators instantiate finite members of the empirical adversary set \(\widehat{\mathbb{A}}\) defined in Section 3.3.

4.5.0.1 Discriminator backbones:

For learned-discriminator evaluations, we instantiate discriminators using RoBERTa [28] and DistilBERT [29] based classifier backbones. These models are not used as teacher or student LLMs in the distillation pipeline. Instead, they serve as empirical adversaries trained to predict whether a response was generated by the teacher or by the candidate student under different input views, including prompt-only, response-only, and prompt-response settings.

4.5.0.2 Prompt-only discriminator:

The prompt-only discriminator receives only the prompt \(x\) and predicts the teacher–student label associated with the hidden response. Since teacher and student examples are constructed from the same prompt set, this discriminator should perform near chance. It serves as a leakage control: high prompt-only accuracy would indicate a flaw in the evaluation split or label construction.

4.5.0.3 Response-only discriminator:

The response-only discriminator receives only the response \(y\). It tests whether student outputs contain prompt-independent artifacts such as systematic differences in response length, style, formatting, hedging, or generation patterns.

4.5.0.4 Prompt-response discriminator:

Receives both the prompt and response i.e., \((x,y)\). This is the primary empirical analogue of the game-based distinguishability setting (see Definition 2). It tests whether the response is teacher-like in the context of the prompt.

4.5.0.5 Embedding discriminator:

The embedding discriminator operates on embeddings of responses or prompt-response pairs. This tests whether teacher and student outputs remain separable in semantic representation space.

4.5.0.6 Policy-level evaluator:

For safety-boundary and policy-relevant prompts, we additionally evaluate whether teacher and student responses induce the same policy-level behavior. This includes refusal agreement, cautionary-language agreement, professional advice recommendation, privacy warning, and unsafe-compliance mismatch.

4.5.0.7 Pairwise judge:

As an additional empirical adversary, we evaluate a pairwise teacher-identification judge. The judge receives a prompt \(x\) and two responses \((y_1,y_2)\), one from the teacher and one from the candidate student, in randomized order, the judge predicts which response is teacher-generated. To control for position bias, we evaluate both the original response order and an A/B-swapped order, and report consistency-filtered results.

Table 2 summarizes the empirical adversary instantiations.

4.6 Evaluation Metrics↩︎

We evaluate teacher—student closeness using three families of metrics: similarity metrics, discriminator-based metrics, and policy-level metrics. We also report pairwise teacher-identification metrics for the pairwise judge adversary.

4.6.0.1 Semantic similarity:

We compute \(\mathrm{cosine}\) similarity between sentence embeddings of teacher and student responses, given by following equation: \[\mathrm{Sim}_{\mathrm{emb}}(T,S) = \frac{1}{m} \sum_{i=1}^{m} \cos(e(T(x_i)),e(S(x_i))),\] where \(e(\cdot)\) is a sentence embedding model and \(m\) is the number of held-out prompts.

4.6.0.2 Lexical and structural metrics:

This includes computation of response length ratio, lexical overlap, and format agreement. For style and formatting prompts, we measure whether the student preserves requested output structures such as bullet lists, JSON-like formatting, or concise answering constraints.

4.6.0.3 Refusal and safety behavior:

For safety-boundary prompts, we compute refusal agreement: \[\mathrm{Agree}_{\mathrm{refusal}} = \frac{1}{m} \sum_{i=1}^{m} \mathbf{1}[r(T(x_i)) = r(S(x_i))],\] where \(r(y)\in\{0,1\}\) indicates whether response \(y\) is classified as a refusal or safety-bounded response and \(\mathbf{1}[\cdot]\) denotes the indicator function. We also compute policy-specific agreement metrics where applicable, including caution agreement, professional-advice agreement, privacy-warning agreement, and unsafe-compliance mismatch.

4.6.0.4 Discriminator accuracy and empirical advantage:

For each discriminator \(D_{\phi}\), we report accuracy and empirical distinguishing advantage: \[\widehat{\mathrm{Adv}}^{\mathsf{dist}}_{T,S}(D_{\phi}) = \left| \widehat{\mathrm{Acc}}(D_{\phi}) - \frac{1}{2} \right|.\] Lower accuracy and lower advantage indicate lower empirical distinguishability. Where appropriate, we also report the area under the ROC curve (AUC), which summarizes discriminator performance across decision thresholds and is therefore less dependent on a particular threshold choice than accuracy.

4.6.0.5 Pairwise teacher-identification:

For pairwise judging, the evaluator receives \((x_i,y_i^1,y_i^2)\), where one response is generated by the teacher and the other by the candidate student. Judge predicts which response is teacher-generated. We compute pairwise accuracy and pairwise advantage given by: \[\widehat{\mathrm{Adv}}_{\mathrm{pair}} = \left| \widehat{\mathrm{Acc}}_{\mathrm{pair}} - \frac{1}{2} \right|.\] Since, LLM judges may exhibit position bias, we evaluate both the original response order and an A/B-swapped order. We additionally report consistency coverage, defined as the fraction of examples for which the judge makes the same underlying teacher/student selection across both orderings.

4.6.0.6 Category-wise distinguishability:

To test whether residual artifacts are localized to particular behavioral regions, we compute discriminator accuracy and empirical advantage separately within each prompt category. We additionally report macro-averages across categories so that large categories do not dominate the aggregate result.

4.6.0.7 Query-budget scaling:

For acquisition experiments, we train student models using different teacher-query budgets q, where each query corresponds to obtaining one teacher response \(T(x_i)\) for a selected prompt \(x_i\). We compare stratified random querying, global disagreement-guided querying, and category-balanced disagreement-guided querying. For each acquisition strategy and budget, we report semantic similarity, policy agreement, and empirical distinguishing advantage.

4.7 Evaluation Questions↩︎

Experiments are organized around the following questions:

1. Semantic transfer: Does black-box LoRA distillation increase semantic similarity between teacher and student responses?

2. Behavioral indistinguishability: Does distillation reduce the empirical advantage of discriminators or automated judges attempting to distinguish teacher outputs from student outputs?

3. Source of residual artifacts: Are remaining teacher–student differences detectable from the response alone, or primarily when the response is evaluated together with the prompt?

4. Behavioral heterogeneity: Is residual distinguishability distributed uniformly across prompt categories, or concentrated in specific behavioral regions?

5. Policy preservation: Does improved semantic similarity also preserve refusal and safety-relevant behavior?

6. Query efficiency: Does disagreement-guided query selection reduce teacher—student distinguishability more effectively than stratified random querying?

5 Results↩︎

We now evaluate whether black-box LoRA distillation reduces teacher—student behavioral distinguishability under the bounded empirical framework introduced above. We first report conventional semantic similarity metrics, then evaluate learned discriminators and pairwise teacher-identification judges. We then analyze whether residual distinguishability is localized to particular behavioral categories and whether query-acquisition strategy affects distillation efficiency.

5.1 Semantic Similarity Improves After Distillation↩︎

Figure 2 shows that LoRA distillation improves embedding similarity to teacher responses across both model families.

The improvement is consistent across model families: Qwen increases from \(0.788\) to \(0.862\), while Llama increases from \(0.814\) to \(0.874\). This confirms that black-box LoRA distillation transfers substantial semantic content from teacher outputs to student outputs. However, semantic similarity alone does not establish behavioral indistinguishability; the next experiments therefore evaluate whether residual differences remain detectable to empirical adversaries.

5.2 Learned Discriminator Advantage Decreases↩︎

We evaluate whether residual teacher–student differences remain detectable by learned prompt-response discriminators. Table 3 reports discriminator accuracy and AUC across two discriminator backbones and three random seeds. Figure 3 visualizes the corresponding unscaled empirical distinguishing advantage, \(\widehat{\mathrm{Adv}} = |\widehat{\mathrm{Acc}}-\frac{1}{2}|\). Lower accuracy, AUC, and advantage indicate lower empirical distinguishability from the teacher.

For Qwen, DistilBERT distinguishability drops substantially after LoRA distillation: accuracy decreases from \(0.778\) to \(0.598\), corresponding to an unscaled empirical advantage reduction from \(0.278\) to \(0.098\). AUC similarly decreases from \(0.868\) to \(0.658\). RoBERTa shows higher variance on Qwen, but the averaged trend remains in the same direction.

For Llama, the reduction is also clear. Under RoBERTa, base responses are distinguishable with accuracy \(0.657\) and AUC \(0.737\), while LoRA responses fall near chance with accuracy \(0.499\) and AUC \(0.527\). Under DistilBERT, Llama accuracy decreases from \(0.649\) to \(0.545\). These results show that LoRA distillation reduces learned discriminator advantage, although the magnitude of reduction depends on the discriminator backbone and model family.

5.3 Pairwise Teacher-Identification Advantage Decreases↩︎

As an additional empirical adversary, we evaluate a pairwise teacher-identification judge. For each prompt, the judge receives two responses in randomized order: one from the teacher and one from the candidate student. The judge predicts which response is teacher-generated. To control for position bias, we evaluate both the original response order and an A/B-swapped order. We then report consistency-filtered results: an example is retained only when the judge makes the same underlying teacher/student selection across both orderings. For this experiment, the evaluated teacher–student family is Qwen, while the judge is a different-family Llama-3.2-3B-Instruct model. We compute pairwise advantage give by Equation 3 : \[\label{eq:result:pairwise-advantage} \widehat{\mathrm{Adv}}^{\mathsf{pair}}_{T,S} = \left| \widehat{\mathrm{Acc}}^{\mathsf{pair}}_{T,S} - \frac{1}{2} \right|.\tag{3}\]

Table 4 reports the corresponding consistency-filtered pairwise teacher-identification results. LoRA distillation reduces pairwise distinguishing advantage relative to the base student.

The base Qwen student is distinguishable from the Qwen teacher with consistency-filtered accuracy \(0.658\), corresponding to pairwise advantage \(0.158\). After LoRA distillation, accuracy drops to \(0.581\), corresponding to pairwise advantage \(0.081\). Thus, under this consistency-filtered pairwise adversary, LoRA distillation reduces observed pairwise distinguishing advantage by approximately \(49\) \(\%\) (refer Equation 4 ): \[\label{eq:results:pairwise-distingishing-advantage} \frac{0.158 - 0.081}{0.158} \approx 0.49.\tag{4}\] This supports the central claim that distillation reduces empirical distinguishing advantage, rather than merely improving semantic similarity. At the same time, the LoRA advantage remains above zero, indicating modest residual distinguishability.

5.4 Category-Wise Residual Distinguishability Is Heterogeneous↩︎

Global discriminator results may hide heterogeneous behavior across prompt categories. Table 5 reports category-wise discriminator accuracy for LoRA students.

Table 6 reports embedding similarity, refusal agreement, length ratio, and bullet-format agreement across query budgets.

The macro-average category-wise discriminator accuracy is approximately chance for both model families: \(0.505\) for Qwen and \(0.507\) for Llama. This suggests that global distinguishability does not necessarily imply uniform within-category separability. Instead, residual artifacts may arise from aggregate distributional cues or from a subset of behavioral regions. The pairwise judge results show a compatible pattern. Figure 5 reports category-wise pairwise distinguishing advantage for Qwen base and Qwen LoRA under the consistency-filtered Llama-3.2-3B-Instruct judge.

The largest pairwise advantages for the base student occur in domain-technical, summarization, robustness perturbation, style/format, and safety-boundary prompts. After LoRA distillation, pairwise advantage decreases in most categories. The strongest residual advantages for Qwen LoRA remain in style/format prompts, robustness perturbations, domain-technical prompts, and coding prompts. In contrast, safety-boundary, ambiguous, general QA, and summarization prompts are closer to chance under the pairwise judge. This supports the view that residual distinguishability is behavior-dependent rather than uniform across the probe suite.

5.5 Query-Budget and Acquisition Strategy↩︎

We compare three acquisition strategies for Qwen:

• Stratified random: Prompts are sampled while preserving category proportions.

• Global disagreement: After a 500-query seed model, prompts with highest teacher—student embedding disagreement are selected globally.

• Category-balanced disagreement: High-disagreement prompts are selected within each category to preserve category coverage.

Overall, query-budget scaling improves semantic similarity for all acquisition strategies, but we do not observe a consistent advantage for disagreement-guided querying over stratified random sampling. Category-balanced disagreement provides a localized gain in bullet-format agreement at \(1,000\) queries, but this effect does not persist across budgets or metrics. This suggests that naive disagreement maximization is not sufficient for query-efficient LLM behavioral distillation. Coverage and prompt diversity are strong baselines, and adversarial acquisition may require more sophisticated diversity-aware or discriminator-in-the-loop strategies.

5.6 Qualitative Behavioral Patterns↩︎

Table 7 summarizes representative qualitative behaviors observed in the experiments.

6 Discussion↩︎

Our results show that black-box LoRA distillation improves teacher—student similarity, but that similarity alone is not sufficient to characterize behavioral imitation. The key question is not only whether \(S(x)\) is close to \(T(x)\) under a predefined metric, but whether an adversary can reliably tell which model generated the response. This is precisely the role of bounded behavioral indistinguishability i.e., for a specified adversary class \(\mathbb{A}(q,t)\), the relevant quantity is given by: \[\mathrm{Adv}^{\mathsf{dist}}_{T,S}(\mathbb{A},q,t) = \sup_{\mathcal{A}\in\mathbb{A}(q,t)} \mathrm{Adv}^{\mathsf{dist}}_{T,S}(\mathcal{A},q,t).\] The empirical results instantiate finite adversary suites \(\widehat{\mathbb{A}}\subseteq\mathbb{A}(q,t)\) and estimate how this distinguishing advantage changes after distillation.

6.1 Similarity Improves, but Indistinguishability Is Stronger↩︎

Embedding similarity improves for both Qwen and Llama, confirming that LoRA distillation transfers substantial semantic content from teacher outputs to student outputs. However, the learned discriminator and pairwise judge results show that semantic closeness does not imply indistinguishability. A student can preserve meaning while still exposing artifacts in style, formatting, generation habits, cautionary language, or prompt-conditioned behavior.

This distinction is the central empirical message of the paper. Conventional similarity metrics estimate a discrepancy such as \(\Delta(T(x),S(x))\), while our adversarial evaluation asks whether residual differences are detectable by an evaluator. The observed reduction in discriminator and pairwise advantage shows that LoRA makes the student more teacher-like, but the remaining nonzero advantage indicates that behavioral imitation is incomplete under the evaluated adversaries.

6.2 The Adversary Class Determines What Is Measured↩︎

Behavioral indistinguishability is not a single model-level scalar independent of the test used to measure it. Different adversaries expose different forms of residual behavior. Learned prompt-response discriminators test contextual teacher-likeness; embedding-based measurements test semantic separability; policy-level evaluators test safety-relevant decisions; and pairwise judges test direct teacher identification given two responses to the same prompt. This explains why reporting a suite of adversaries is more informative than reporting only one similarity or discriminator score. In our notation, a low advantage for one empirical adversary \(D_\phi\) does not imply that \(\mathrm{Adv}^{\mathsf{dist}}_{T,S}(\mathbb{A},q,t)\) is small for every meaningful adversary class. Instead, the paper reports bounded evidence: for the evaluated adversary suite, distillation reduces the observed distinguishing advantage. This framing makes the claim precise without requiring an unrealistic universal indistinguishability guarantee.

6.3 Pairwise Judging Strengthens the Distinguishability Story↩︎

The pairwise teacher-identification experiment provides an intuitive adversarial test: given the same prompt and two responses, can an evaluator identify the teacher output? This setting complements independent binary classification problem since, both responses are evaluated in the same prompt context. The pairwise result strengthens the main claim. Using a different-family Llama-3.2-3B-Instruct judge with A/B-swap consistency filtering, Qwen pairwise advantage decreases from \(0.158\) for the base student to \(0.081\) after LoRA distillation. In the notation of the framework, this is an observed decrease in \[\widehat{\mathrm{Adv}}^{\mathsf{pair}}_{T,S} = \left| \widehat{\mathrm{Acc}}^{\mathsf{pair}}_{T,S} - \frac{1}{2} \right|.\] Thus, distillation makes teacher identification harder under this pairwise adversary, while the remaining advantage shows that some residual teacher-like cues are still detectable.

6.4 Residual Distinguishability Is Localized↩︎

The category-wise results show that residual distinguishability is not uniform across the prompt suite. Aggregate discriminator results can suggest that a student remains distinguishable, but category-wise analyses reveal where that distinguishability is concentrated. For Qwen LoRA, pairwise residual advantage is most visible in style/format, robustness perturbation, domain-technical, and coding prompts. These categories likely expose artifacts related to formatting conventions, precision of technical explanations, handling of noisy instructions, and model-specific response structure. In contrast, safety-boundary, ambiguous, general QA, and summarization prompts are closer to chance under the pairwise judge. This supports a category-aware interpretation: a distilled model may be difficult to distinguish in some behavioral regions while remaining separable in others.

6.5 Policy-Level Behavior Requires a Separate Layer↩︎

Policy-level behavior should not be inferred solely from semantic similarity or overall discriminator performance. A student may preserve the broad meaning of a teacher response but weaken caveats, omit professional guidance, or change whether it refuses a sensitive request. Conversely, it may use different wording while preserving the same policy-level decision. Our framework captures this distinction through a policy evaluator \(\pi(x,y)\), which maps prompt-response pairs to policy-relevant outcomes. This supports the notion of policy-level agreement or disagreement (refer Section 3.4 and Equation 2 ).

6.6 Coverage Matters for Query Acquisition↩︎

The query-budget experiments show that more teacher queries generally improve semantic similarity, but disagreement-guided acquisition does not consistently outperform stratified random sampling. This suggests that high-disagreement examples are not automatically the most useful examples for behavioral distillation. A plausible explanation is that global disagreement selection can over-focus on outliers or high-variance prompts while reducing coverage of the broader behavioral space. Category-balanced disagreement partially addresses this, but the results indicate that coverage and diversity are strong baselines. For future acquisition strategies, the objective should not be disagreement alone, but disagreement combined with category coverage, diversity, and possibly discriminator-in-the-loop prompt selection.

6.7 Implications for Black-Box Distillation↩︎

The main implication is that black-box LLM distillation is better evaluated as a bounded adversarial measurement problem rather than through semantic similarity alone. Semantic similarity measures whether the student captures the teacher’s content, while distinguishing advantage measures whether residual artifacts remain detectable. Policy-level and category-wise evaluations further show whether safety-relevant behavior transfers and where imitation succeeds or fails.

Together, these measurements provide a more informative view than any single metric. Our results show that LoRA distillation reduces teacher—student distinguishability under several empirical adversaries, but does not eliminate all detectable differences. This is the intended role of bounded behavioral indistinguishability: not to claim that two models are identical, but to specify how difficult they are to distinguish under explicit query, computational, and adversary-class constraints.

7 Limitations↩︎

The purpose of the paper is not to claim universal indistinguishability between teacher and student models. The results should be interpreted as bounded empirical evidence under the prompt suite, model families, query budgets, decoding settings, and adversary classes studied in this paper. Following are the limitations:

• Prompt distribution: The behavioral probe suite is controlled and metadata driven rather than drawn from natural deployment traffic. This design is useful because it lets us evaluate distinguishability across targeted behavioral regions such as reasoning, coding, ambiguity, safety-boundary prompts, instruction conflict, and style control. However, the measured advantages are specific to this probe distribution.

• Adversary coverage: We instantiate several empirical adversaries, including learned discriminators, embedding-based measurements, policy-level metrics, and a pairwise automated judge. However, these adversaries are not exhaustive. Stronger adaptive discriminators, human expert judges, or discriminator-in-the-loop prompt generation could reveal additional residual distinguishability.

• Model and distillation scope: We evaluate two open teacher–student families, Qwen and Llama, under a practical parameter-efficient LoRA distillation setting. The same framework can naturally be extended to larger models, cross-family distillation, full fine-tuning, preference optimization, or multi-stage distillation to study how different training regimes affect residual distinguishability.

• Statistical and adaptive evaluation: Empirical advantages are estimated from finite held-out prompt sets, with category-wise results reflecting smaller per-category sample sizes. Future extensions can add confidence intervals, larger held-out sets, and adaptive adversaries that select prompts based on previous teacher–student responses.

Overall, these points clarify the intended scope of bounded behavioral indistinguishability.

8 Conclusion↩︎

We introduced bounded behavioral indistinguishability as a security-inspired framework for evaluating whether black-box distillation makes a student merely similar to its teacher, or difficult to distinguish from it under explicit empirical constraints. Central to the framework is \((\epsilon,q,t,\mathbb{A})\)-behavioral indistinguishability, which bounds the distinguishing advantage of an adversary class \(\mathbb{A}\) under a query budget \((q)\) and computational budget \((t)\). This formulation provides a reproducible way to measure residual behavioral distinguishability between a teacher and a student model.

Across Qwen and Llama model families, LoRA distillation improves semantic similarity between teacher and student responses and reduces learned-discriminator separability. A pairwise teacher-identification experiment using a different-family Llama judge further shows that Qwen pairwise distinguishing advantage decreases from \((0.158)\) for the base student to \((0.081)\) after LoRA distillation. Our results also show that residual distinguishability is heterogeneous across behavioral categories and adversary classes. Style/format, robustness, and domain-technical prompts remain more distinguishable than some other categories, and naive disagreement-guided query acquisition does not consistently outperform stratified random sampling. These findings suggest that semantic similarity is useful but insufficient. Bounded behavioral indistinguishability shifts the evaluation question from “How similar are the outputs?" to”Under explicit query, computational, and adversary-class constraints, can an adversary tell the teacher and student apart?"

References↩︎

---

1. https://github.com/mhasan08/bounded-llm-distillation↩︎