1 Introduction↩︎

Open-weight large language models (LLMs) bring substantial benefits to society such as enabling downstream customization, self-hosting for privacy, and open research [3]. However, they also significantly increase the potential for adversarial misuse. As a natural worst-case threat, an adversary can fine-tune the model for any harmful task/domain of its choice [4]. Safeguards against these fine-tuning attacks are an emerging line of work: tamper-resistant [1] and self-destructing [2] models are two prominent examples.

Underlying this threat model is an implicit assumption that harmful behavior will be learned through downstream fine-tuning. However, pretrained LLMs contain broad knowledge relevant to many harmful tasks – even in domain-specific evaluations, baseline models achieve non-trivial harmfulness without any fine-tuning [1]. Thus, fine-tuning by adversaries often serves to remove refusal mechanisms rather than introduce new harmful capabilities [5]–[7], raising a broader question about the robustness of safeguarded models:

In this paper, we show that indeed, two gradient-free attacks – Abliteration and Prefilling – can jailbreak safeguarded models without fine-tuning at all. Against two representative fine-tuning safeguards (TAR [1] and SEAM [2]), these attacks achieve attack success rates from 16% to 96% across benchmarks (BeaverTails, HarmBench, and AdvBench) and model architectures (Llama3.2, Qwen2.5, Gemma3).

While Abliteration and Prefilling are individually known attacks, their effectiveness against fine-tuning safeguards has not been characterized — a gap this paper addresses. We find that safeguards meaningfully reduce attack success rates compared to baseline safety-tuned models, in some cases to below 10%. However, gradient-free attacks can recover the aforementioned 16% to 96% attack success rate across various settings, which is a substantial increase over this baseline. These results suggest that current open-weight safeguards, while meaningfully improving baseline safety, do not fully eliminate the harmful knowledge already embedded in the pretrained model. As a result, our attacks can successfully bypass refusal mechanisms to extract this harmful knowledge.

As a step towards defending against the aforementioned attacks, we propose Abliteration-Resistant Tuning (ART), a simple method that incorporates Abliteration into the fine-tuning objective and reduces attack success rates when applied alone or layered with open-weight safeguards. We conclude with a cautionary note that while fine-tuning comprises the most expressive class of attacks, safeguards that suppress refusal without removing underlying harmful knowledge remain vulnerable to simpler gradient-free attacks. As such, it is necessary to evaluate open-weight safeguards using both gradient-free attacks alongside fine-tuning attacks.

2 Preliminaries↩︎

We first introduce two representative open-weight safeguards tested in our work, SEAM [2] and TAR [1]. Standard safety alignment trains models to refuse harmful prompts via supervised fine-tuning (SFT) or preference optimization (e.g. RLHF [8], DPO [9]). However, these methods are brittle: safety alignment can degrade by fine-tuning on a few harmful examples [6], [10]. SEAM and TAR are explicitly designed to address this by making refusal behavior persist under fine-tuning attacks.

Notation. Throughout, we use \(\mathcal{D}^\text{ret}\) and \(\mathcal{D}^\text{aln}\) to denote a retain dataset and alignment dataset respectively. From \(\mathcal{D}^\text{ret}\) we sample prompts and responses \((x^\text{ret}\), \(y^\text{ret})\) which are both benign; from \(\mathcal{D}^\text{aln}\) we sample triples \((x^\text{aln}, y^-, y^+)\), where \(x^\text{aln}\) is a harmful prompt, \(y^-\) is a harmful continuation of \(x^\text{aln}\), and \(y^+\) is a safe refusal of \(x^\text{aln}\). For convenience, we omit prompts from the notation; all computation on \(y^\text{ret}\) or \((y^+,y^-)\) implicitly also uses their respective prompts \(x^\text{ret}\) or \(x^\text{aln}\).

TAR. Tampering Attack Resistance [1] is a safeguard that uses meta-learning to make safety behaviors persist after adversarial fine-tuning attacks. At each iteration, TAR simulates multiple steps of a harmful fine-tuning attack drawn from a set of train-time adversaries \(\mathcal{A}_{\text{train}}\). A tamper-resistance loss \(\mathcal{L}_\text{TR}\) measures how harmful the model is after the simulated attack has been applied to the current parameters \(\theta\). In practice, \(\mathcal{L}_\text{TR}\) is instantiated as a DPO loss over \((y^-, y^+)\) pairs, which encourages the attacked model to prefer refusal responses \(y^+\) over harmful continuations \(y^-\). This steers the model toward a region of weight space from which gradient-based fine-tuning attacks cannot easily recover harmful behavior: \[\label{eq:tar} \mathcal{L}_{\text{TAR}}(\theta;\, y^-, y^+) = \mathbb{E}_{\texttt{attack} \sim \mathcal{A}_{\text{train}}} \left[ \mathcal{L}_{\text{TR}}(f_{\texttt{attack}(\theta)};\, y^+, y^-) \right].\tag{1}\] This is combined with a retain loss on benign sequences \(y^\text{ret}\) that combines a standard cross-entropy (CE) loss and distance to the residual stream hidden states \(h\) of the initial pretrained parameters \(\theta_0\): \[\label{eq:tar95retain} \mathcal{L}_{\text{ret}}(\theta;\, y^\text{ret}) = \mathbb{E}_{y^\text{ret}}\Bigl[\mathrm{CE}(f_\theta(y^\text{ret}), y^\text{ret}) + \|h_\theta(y^\text{ret}) - h_{\theta_0}(y^\text{ret})\|_2^2\Bigr],\tag{2}\]

SEAM. Rather than resisting harmful fine-tuning, SEAM [2] is a safeguard that causes it to backfire. SEAM couples the optimization trajectories of harmful and benign tasks so that any attempt to optimize for harmful objectives inevitably degrades general performance. This is achieved via a self-destructive loss defined as the cosine similarity between the gradient of the loss on \(y^-\) and the gradient of the loss on \(y^\text{ret}\): \[\label{eq:seam95sd} \mathcal{L}_\text{SD}(\theta;\, y^-, y^\text{ret}) = \mathrm{sim}\bigl(\nabla_\theta \mathcal{L}(\theta;\, y^-),\, \nabla_\theta \mathcal{L}(\theta;\, y^\text{ret})\bigr).\tag{3}\] Minimizing this loss encourages the two gradients to point in opposing directions, so that descending on the harmful objective effectively ascends on the benign one. This is combined with an unlearning loss \(-\mathrm{log}(\mathrm{CE}(f_\theta(y^-), y^-))\) that moves the model away from harmful parameters. Notably, unlike TAR, SEAM does not use an explicit retain loss on \(y^\text{ret}\); instead, utility is preserved through a CE loss on safe refusal sequences \(y^+\), which the authors find to be more effective at maintaining stable representations of harmful prompts.

Unified training framework. Despite their different objectives, both safeguards share a common training structure, summarized in Algorithm 2. At each step, a safeguard-specific loss \(\mathcal{L}_\text{def}\) is combined with a utility preservation term: \[\label{eq:unified} \mathcal{L}(\theta) = \mathcal{L}_\text{def}(\theta;\, y^-, y^+) + \lambda\, \mathcal{L}_\text{ret}(\theta;\, y^\text{ret}, y^+),\tag{4}\]

3 Attack Methodology↩︎

We propose using two simple, gradient-free attacks against TAR and SEAM, which require no fine-tuning at all: (1) Abliteration, which identifies and removes a refusal direction from the model’s residual stream at inference time, and (2) Prefilling, which bypasses refusal by injecting a compliant prefix before generation begins. Surprisingly, we show that an adversary can use these attacks to jailbreak safeguarded models without fine-tuning.

Abliteration. Refusal behavior in safety-tuned models has been shown to be mediated by a single direction in the residual stream [11], and subtracting (ablating) this direction from activations at inference time is sufficient to jailbreak the model. We generate candidate refusal directions using a set of harmful prompts \(y^-\) and benign prompts \(y^+\) and computing the mean difference in hidden states at each layer \(\ell \in [L]\), where \(h^{(\ell)}\) is the hidden state at layer \(\ell\): \[\label{eq:abliterate} r^{(\ell)} = \mathrm{normalize}\Bigl(\mathbb{E}_{y^-}\bigl[h^{(\ell)}\bigr] - \mathbb{E}_{y^+}\bigl[h^{(\ell)}\bigr]\Bigr),\tag{5}\] Rather than subtracting \(r^{(\ell)}\) at inference time, we permanently project it out of the model’s weight matrices. Specifically, for each output projection matrix \(W\) (attention \(W_O\) and MLP \(W_\text{down}\)) in a single layer, we apply the following transformation: \[\label{eq:weight95proj} W' = W(I - \alpha\, r^{(\ell)} {r^{(\ell)}}^\top),\tag{6}\] which removes the component of any residual stream input along \(r^{(\ell)}\) before it is processed by the layer. Here \(\alpha\) is a scaling factor tuned to balance attack effectiveness against output coherence.

In practice, a refusal direction can be computed for every layer and token position, and an attacker could potentially test multiple attacks (using different subsets of these refusal directions) in parallel. Since the effectiveness of abliteration is highly sensitive to the choice of layer [11], we abliterate each layer individually and consider a prompt to be successfully attacked if any attack on a given layer was successful. For computational efficiency, we only consider the hidden state at the last prompt token rather than searching over all tokens.

Prefilling. The user-assistant input structure of instruction-tuned models can be exploited by injecting a prefill (partial response) into the assistant turn before generation begins. We use a simple static prefill (“Sure, here are some ideas. First, …”) without any optimization. This results in a stronger attack than naively prompting within the user turn [12], [13], as it bypasses refusal behavior that typically appears in the first tokens of a response.

Compositional Attacks. Open-weight access exposes a large attack surface: an adversary can freely combine, adapt, and search over attacks in ways that are not possible against API-served models. We demonstrate a simple instance of this by first abliterating the model and then adding a prefill on top. While this attack improves over both individual attacks in a few settings, the gains are inconsistent, suggesting that such an approach would be more beneficial as part of a larger adaptive attack.

4 Experiments and Results↩︎

In this section, we investigate two key research questions: First, can an adversary use Abliteration and Prefilling to jailbreak models safeguarded by TAR and SEAM? Second, how do additional factors (architecture, model size, and tradeoffs with benign utility) affect the attack success rate? We then explore potential defenses against these attacks in Section 5.

4.0.0.1 Model Setup.

We evaluate the effectiveness of Abliteration and Prefilling against open-source refusal-trained models (Base) with additional fine-tuning using SEAM and TAR. We experiment with 1B, 3B, and 8B Llama models, as well as 1-2B scale models across Llama, Qwen, and Gemma families. We train both safeguards using the Anthropic HH RLHF dataset [14], which is the original data used by TAR; we train SEAM on the same data for consistency. We train TAR for 750 outer loop steps and 4 to 16 inner loop steps (to balance benign refusals with harmful compliance), and train SEAM for 2000 steps. We use a learning rate of 2e-5 and batch size of 8 for fine-tuning.

Attacks. For Abliteration, we compute refusal directions using 128 harmful and benign prompts each from two public HuggingFace datasets (mlabonne/harmful_behaviors and mlabonne/harmless_alpaca). We sweep the scaling factor \(\alpha \in \{1.0, 1.5, 2.0\}\) and all layers in the model, reporting the best result per harmful evaluation prompt. For Prefilling, we use a fixed static prefill (“Sure, here are some ideas. First, …”) across all settings.

Evaluation. We evaluate attack success rate on three benchmarks: BeaverTails [15], AdvBench [16], and HarmBench [17]. To evaluate whether an attack is successful, we input the concatenated prompt and response into the 13B Llama-2-chat judge from HarmBench, which outputs a “Yes” or “No” answer in response to whether the interaction is harmful [17]. We first compare safeguards on Llama-3.1-8B-Instruct across all benchmarks (Table 1), then examine how attack effectiveness varies with model size (Table 2) and family (Table 3) on BeaverTails. We additionally evaluate model utility on the Alpaca benchmark [18], reporting ROUGE-L against reference outputs and the rate of incorrect refusals on benign prompts (Table 4). For incorrect refusals, we search the response for a fixed set of key phrases such as “Sorry, I cannot...” or “I’m just an AI...”.

4.1 Results↩︎

Non-fine-tuning attacks bypass open-weight safeguards. Table 1 shows attack success rates for Llama3.1-8B-Instruct models across all three benchmarks. Baseline attack success rates are low across all models and benchmarks, confirming that the safeguards can effectively instill refusal behavior. However, Abliteration alone significantly increases attack success rates for Base and SEAM to above 70% on all three benchmarks and above 90% on AdvBench and HarmBench. While the rates for TAR are lower, they are typically several times larger than the Baseline rate (no attack applied).

We find that Abliteration is much more effective than Prefilling, especially for the 8B models tested in Table 1. Prefilling alone has a modest effect; it generally produces single-digit increases over Baseline, and the combination of Abliteration + Prefill can also produce single-digit increases over Abliteration alone. Interestingly, in both cases, Prefilling can also lower the attack success rate. While we use a very simple static prefill (“Sure, here are some ideas. First, ...”), more complex approaches (e.g. prefill optimization) can potentially further boost attack success rate.

Jailbroken prompts shift toward harmless representations. In Figure 3, we visualize PCA projections of last-token hidden states at layer 15 for the abliterated Base model (Llama-3.2-1B-Instruct), where Abliteration is applied at layer 7, before and after generating 128 tokens on 100 harmful and 100 harmless prompts. Before generation, harmful (red) and harmless (green) inputs are clearly distinguishable in activation space. During generation, there are three key observations. First, the representations of successfully jailbroken prompts (\(\times\)) shift towards those of harmless prompts. Second, harmful prompts whose representations remain distinct are all refused by the model. Third, and most interestingly, several harmful prompts’ representations shift toward the harmless distribution, but still result in refusal. These observations lead us to hypothesize that a model’s ability to distinguish between harmful and harmless inputs is a key factor in its refusal capability.

TAR is more effective than SEAM. While Abliteration and Prefilling significantly increase attack success rate against TAR (up to multiple times), TAR is substantially more resistant to attacks than Base or SEAM. In Table 1, the ASR on TAR ranges from as low as 9% (BeaverTails, Prefill) up to 62% (AdvBench, Abliterate). We hypothesize that this is because the objective of TAR minimizes harmful behavior more strongly through an inner meta-learning optimization, while the objective of SEAM combines vanilla gradient ascent on harmful data with a self-destructive loss.

In Figure 4, we provide intuition on the effectiveness of SEAM and TAR compared to Base by visualizing the last-layer representations of each method. We compare last-token hidden states at layer 15 after abliterating the models and generating 128 tokens. TAR exhibits clearer separation between harmful and harmless inputs than SEAM, followed by Base. Because this is the same ranking we observe in terms of ascending ASR, this matches our expectation that more effective safeguards (i.e. TAR) exhibit a stronger ability to identify (and handle) harmful representations.

Attacks are effective across model size. Table 2 shows how attack effectiveness varies with model size on BeaverTails for Base and SEAM. First, Prefilling is more effective at smaller scales: for Base, it raises ASR from 15% to 30% at 1B but only from 13% to 20% at 8B; SEAM follows the same pattern, increasing from 5% to 14% at 1B but only from 2% to 6% at 8B. Second, Abliteration is effective across all scales: Base reaches 68%, 59%, and 85% at 1B, 3B, and 8B respectively, while SEAM reaches 38%, 41%, and 73%. The combined attack yields the highest ASR in most cases, with Base 3B reaching 89% and SEAM 3B reaching 80%. Finally, for smaller Base and SEAM models, the combined attack can significantly boost performance over either attack alone.

Attacks are effective across model architecture. Table 3 reports attack success rates on BeaverTails across three model families: Llama, Qwen, and Gemma. Across all families, Abliteration is a stronger attack than either baseline and prefill-only. Gemma is particularly vulnerable to Abliteration, reaching 93% and 96% under Abliteration and the combined attack respectively. One notable difference across families is the effect of Prefilling on Qwen SEAM, where Prefilling alone raises ASR from 1% to 34%, a much larger effect than seen in Llama or Gemma. Overall, the best attacks increase attack success rates over the Baseline by several times.

Safeguards sacrifice benign performance. A potential concern with any defense is that it may degrade the quality of the model’s responses non-harmful inputs. Table 4 reports utility on the Alpaca benchmark, which we use to evaluate whether the safeguards degrade general instruction-following ability on benign inputs. We report two metrics: ROUGE-L, which measures the quality of instruction-following relative to a reference model, and Benign Refusal, which measures the rate at which the model incorrectly refuses harmless instructions. Benign refusal rates are generally low, but the safeguards do increase refusal rates compared to the base model. ROUGE-L scores are largely comparable between Base and SEAM across all sizes, while TAR shows a more notable utility cost, particularly at 1B where ROUGE-L drops to 22.9 compared to 29.0 for the Base model.

5 Defense: Abliteration-Resistant Tuning (ART)↩︎

Since we find that abliteration is an effective attack across multiple settings, a natural defense is to train models to remain safe even after abliteration. Our key insight is that standard defenses optimize refusal behavior in the original model \(f_\theta\), but place no constraint on the abliterated model \(f_{\tilde{\theta}}\). We propose abliteration-resistant tuning (ART), which addresses this by simulating the worst-case abliteration attack on the current parameters and performing gradient ascent on harmful outputs \(y^-\). While prior work has proposed defenses against abliteration based on synthetic data generation [19], ART requires no additional data beyond the alignment dataset already used by existing safeguards, making it a lightweight and composable complement to such approaches.

ART is described in Algorithm 5 and uses Algorithm 6 (abliteration) as an inner loop. Note that in ART, abliteration with Algorithm 6 uses a batch of training data (i.e. Anthropic HH), while at evaluation time, we use the data and evaluation procedure in Section 4.1. At each training step, we first identify the single most attackable layer by computing a refusal direction at every layer and selecting the layer whose abliteration most reduces the model’s harmful CE loss. We then compute the gradient of the CE loss on harmful sequences with respect to the abliterated parameters \(\tilde{\theta}\). We apply gradient ascent on this direction to the original parameters \(\theta\), while also incorporating a standard CE retain loss on benign inputs in order to retain general performance. Unlike SEAM or TAR, we do not use \(y^{aln}\) for DPO or gradient ascent, as we find that significantly increases benign refusal rates.

Experimental Setup. We apply ART to Base, SEAM, and TAR on Gemma-1B-IT, fine-tuning each defended model for an additional \(200\) steps on the Anthropic HH dataset [14]. At each step, we use a small batch of harmful and benign examples for both the abliteration direction computation and the \(\mathcal{L}_{\text{AR}}\) loss. After training, we repeat the same attack pipeline described in Section 4.1: we evaluate attack success rate on BeaverTails under the same four attack conditions and report benign utility on the Alpaca benchmark. ART introduces two additional hyperparameters, \(\gamma_\text{ar}\) and \(\gamma_\text{ret}\), which we set to \(\gamma_\text{ar}=1\) and \(\gamma_\text{ret}=2\) in all experiments. We train with a batch size of 4, 200 steps, and a learning rate of 2e-5. A key detail omitted from the pseudocode is that we recompute the optimal layer \(l^*\) every 10 steps rather than every step. While increasing the amount of data used and frequency for abliteration could potentially improve the robustness of ART, we keep our setup intentionally lightweight to demonstrate both its accessible and clear benefits.

ART reduces attack success rate at modest utility cost. Table 5 reports attack success rates and utility for ART applied on top of each safeguard on Gemma3-1B-IT. ART substantially reduces abliteration ASR across all three safeguards: Base drops from 93% to 49%, SEAM from 80% to 64%, and TAR from 76% to 55%. Combined abliteration and prefilling follows a similar trend. Interestingly, baseline and prefill-only ASRs are reduced to near zero in all cases, suggesting that abliteration resistance could have benefits against more general attacks. Furthermore, these results show that ART may be complementary to other safeguards and that there is a rich design space over which safeguards can potentially be layered together.

Crucially, none of the defenses evaluated fully close the gap from the baseline model. This reflects a broader challenge in open-weight safety: defenses that suppress refusal behavior without eliminating underlying harmful knowledge will remain exploitable by simple jailbreaking attacks.

6 Related Work↩︎

Open-weight safety evaluation. Because adversaries have direct access to open-weight model parameters, they can modify weights to remove safety behavior entirely [20]. These white-box attacks are an accessible and effective attack, as we discuss below. A growing line of work has developed defenses specifically designed to persist after adversarial modification [19], [21]–[23]. TAR [1] and SEAM [2] are two safeguards that defend specifically against fine-tuning.

Fine-tuning attacks. Fine-tuning attacks pose a threat both to closed models via fine-tuning-as-a-service APIs and, more acutely, to open-weight models where adversaries have direct access to weights [20], [24]. Such attacks are highly effective: safety alignment can be broken by fine-tuning on a handful of harmful examples [5], [10], [25], [26], or even on benign data [6], [27]–[29]. Furthermore, recent work has shown simple changes in the fine-tuning pipeline can break safeguards such as TAR [30], [31].

Activation and weight-space attacks. Safety behavior can also be bypassed via editing the model’s activations or projecting model weights without fine-tuning. In the model’s representation space, certain directions correspond to behaviors of interest; these directions can be added or subtracted at inference time to amplify or suppress these behaviors respectively [32]–[35]. [11] leverage this insight and propose a simple search over layers, tokens, and scaling factors to maximize the effect of refusal suppression.

Prompting and prefilling attacks. Prompt-based jailbreaks require only black-box access and can elicit harmful outputs without modifying model weights [16]. Prefilling attacks are a more targeted variant: [36] show that safety alignment concentrates its effect on the first few output tokens, so supplying a non-refusal prefix at the start of generation can bypass the refusal mechanism [13], [37].

Our work. TAR and SEAM are designed to prevent adversarial misuse of open-weight models, but recent work has shown that they remain brittle to simple variations of fine-tuning attacks [30], [31], raising the question of whether they are robust to gradient-free attacks at all. We show that both safeguards are vulnerable to abliteration and prefilling, and propose ART as a complementary defense that can be layered onto existing safeguards to reduce their vulnerability.

7 Conclusion↩︎

Open-weight safeguards such as TAR and SEAM are designed to prevent adversarial misuse by resisting fine-tuning attacks. However, as shown in prior work and supported by our results, pretrained models contain significant amounts of harmful knowledge which cannot be easily removed via downstream fine-tuning safeguards. Thus, gradient-free attacks can simply elicit this knowledge without needing fine-tuning at all. To validate this, we show that Abliteration and Prefilling recover success rates of 16–97% compared to below 10% under the safeguard alone; the additional access available in the open-weight setting could be exploited to design stronger variants of both attacks.

As a step toward closing this gap, we propose ART, which defends against abliteration by simulating abliteration during training and performing gradient ascent on harmful outputs. While ART reduces attack success rates across all settings, no defense evaluated fully closes the gap to the no-attack baseline. This suggests that truly robust open-weight safety will require defenses that eliminate harmful knowledge rather than relying on refusal mechanisms. We hope our results motivate both stronger defenses and more comprehensive evaluation practices that consider both fine-tuning and gradient-free attacks in open-weight settings.

8 Technical appendices and supplementary material↩︎

Licenses and Attribution. Our figures use images from Flaticon.com. As per their free license, we state that “This cover has been designed using resources from Flaticon.com.”

References↩︎