1 Introduction↩︎

3pt

Hybrid-thinking language models are only useful if their control tokens actually control behavior. In principle, \think should invoke deliberate reasoning when a problem requires it, while \no_think should return a direct answer when extended reasoning is unnecessary [2]–[4]. This distinction matters because long reasoning traces come with an associated cost: they increase latency, consume token budget, and occupy context-window space [5]–[7]. A practical \no_think mode therefore must not be a weakened version of \think; it must operate as a genuine direct-answer mode.

In current hybrid-thinking systems, this separation is weak. Even under explicit \no_think instructions, models often continue to emit signs of deliberation: responses become unnecessarily long, reflective markers such as wait or hmm appear, and outputs drift toward partial chain-of-thought generation. Prior literature defined this failure mode as reasoning leakage. Under this failure mode, \no_think becomes an unstable halfway mode between direct answering and explicit reasoning, thereby the interface no longer delivers the efficiency, predictability, and behavioral control that motivate hybrid thinking in the first place.

Most existing attempts to fix leakage operate through training alone. Prior work shows that changing data scale, think/no-think ratios, and multi-stage training schedules can reduce leakage substantially [1]. However, lower leakage alone does not guarantee a strong \no_think mode: answers may remain verbose, partially reflective, or weaker in accuracy. This suggests that the limitation is not only in data or optimization. A single dense decoder is being asked to realize two competing output behaviors with the same feed-forward parameters; one mode should externalize reasoning, while the other should suppress it. We therefore hypothesize that weak mode separation is partly an architectural interference problem.

We address this with Path-Lock Expert (PLE), a minimal architecture that makes the mode choice part of the architecture itself. PLE replaces each decoder MLP with two semantically locked experts, one dedicated to \think and one dedicated to \no_think, and uses deterministic control-token routing to activate exactly one expert path for the entire sequence. Attention, embeddings, normalization, and the language-model head remain shared. This design separates the parameters most directly tied to generation behavior while preserving a shared representational backbone, and it does so without a learned router, auxiliary balancing losses, or a complex RL-based training pipeline.

Across mathematical and scientific reasoning benchmarks, PLE yields a substantially stronger \no_think mode than dense baselines and prior training-only hybrid approaches: no-think outputs are shorter, exhibit far less reasoning leakage, and achieve better no-think accuracy in our primary evaluations. At the same time, PLE maintains strong \think-mode performance. Our claim is intentionally specific: we do not argue that PLE solves hybrid thinking in general or establishes a new overall state of the art; rather, we show that architecture-level mode separation can materially improve the usefulness of the \no_think mode while preserving the value of \think. Our contributions are as follows:

• We motivate an architectural view of reasoning leakage in hybrid-thinking models, arguing that training heuristics alone may be insufficient because shared feed-forward parameters interfere with clean mode separation.

• We introduce PLE, a deterministic dual-expert decoder architecture that separates \think and \no_think behavior at the MLP level while preserving a shared Transformer backbone and dense-model-like per-token inference computation.

• We show that, across our evaluated settings, PLE strengthens the \no_think mode through lower leakage, shorter outputs, and improved no-think accuracy in the primary evaluations, while maintaining strong \think performance.

2 Motivation↩︎

We use a simple empirical lens: compare a pure instruction-tuned model, an off-the-shelf hybrid-thinking model, and the strongest prior training-level mitigation, and ask whether \no_think behaves like a genuine direct-answer mode. First, \no_think should suppress explicit reasoning artifacts such as long deliberative outputs and reflective phrases. Second, it should preserve the behavioral character of direct answering rather than merely producing a weaker version of \think.

2.0.0.1 Off-the-shelf hybrid thinking leaks by default.

1 shows that released hybrid-thinking models do learn to respond differently under \think and \no_think prompts, but the separation is incomplete. Relative to the pure Instruct baseline, the hybrid model’s \no_think outputs remain longer and contain nonzero reflective-token counts across benchmarks. This means the issue is not that hybrid models fail to react to the control token at all; rather, they react only partially. \no_think behaves like a softened form of thinking, not like a truly distinct direct-answer mode.

2.0.0.2 Training-only fixes help, but they stop short of true separation.

[1] systematically explored the most natural mitigation strategies: scaling training data, changing the think/no-think ratio, and applying multi-stage training. Their best recipe significantly reduces leakage, confirming that data and optimization matter. However, it still does not match the behavioral target defined above. Reflective tokens remain nonzero, and the resulting \no_think behavior does not consistently recover the direct-answer profile of the pure Instruct model across benchmarks. This is the key empirical observation: even after substantial recipe engineering, the model remains only partially controllable.

3 Method: Path-Lock Expert↩︎

Path-Lock Expert (PLE) aims to achieve three primary objectives in hybrid-thinking language models: 1)Better reasoning mode separation than conventional hybrid-thinking architectures; 2) Performance preservation, ensuring not to degrade the model’s core reasoning capabilities; and 3) Parameter efficiency, avoiding the prohibitive cost of training/deploying two independent models. To achieve these, PLE introduces a minimal architectural modification to the standard decoder-only Transformer: the attention weights remain fully shared, but the MLP component in each layer is replaced by dual MLP experts——one handling the \think pathway and the other dedicated to the \no_think mode.

The selection of the expert is determined one-time by the control token (‘\think’or’\no_think’) present in the input. Once the route is established, the entire inference process is "locked" to the chosen expert across all layers. This stands in contrast to traditional Mixture-of-Experts (MoE) models that perform routing at every layer, which is why we refer to our architecture as Path-Lock Expert.

PLE avoids the prohibitive overhead of full model duplication by maintaining shared attention, embeddings, and normalization layers, while enforcing hard separation exclusively at the feed-forward network. This design allows both modes to leverage a unified representational backbone and shared attention base. Notably, PLE is highly parameter-efficient: by sharing the attention mechanisms and only replicating the MLP experts, we save approximately 33% of the total parameters compared to deploying two independent Qwen3-4B models, achieving robust mode separation with minimal architectural expansion.

Path-locked dual-expert decoder. In every layer, the single MLP is replaced by two structurally identical copies—one for \think and one for \no_think. All other components (self-attention, positional encoding, normalization, embeddings, LM head) remain shared. Because exactly one expert is active per response, per-token computation is unchanged;

Deterministic control-token routing. The active expert is selected by the last /think or /no_think token appearing in the input; the choice is resolved once and locked for all layers and all decoding steps. No learned router or balancing loss is involved.

Routing-conditioned fine-tuning. PLE is fine-tuned with standard causal LM loss on chat examples tagged with either mode. The inactive expert receives zero gradient; the shared backbone is updated by both modes. This removes direct parameter coupling between the two MLP pathways while preserving a common representational substrate.

We provide a rigorous definition in Appendix 9

3.0.0.1 Relation to Mixture-of-Experts.

While previous work such as Metis-HOME [8] has explored using a two-expert MoE for mode separation, their approach still relies on the conventional MoE framework with learned, token-level routers. In contrast, PLE shifts from token-level competition to deterministic, sequence-level routing. By tying expert identity directly to the \think or \no_think interface via control tokens, PLE eliminates the need for router parameters, balancing losses, or complex dispatch decisions. This ensures a “locked” path that is structurally stable and specifically designed for behavioral control.

4 Experiments↩︎

In this section, we evaluate the effectiveness of PLE in achieving architectural mode separation. Our experiments are designed to verify whether physical parameter isolation can improve mode separation by addressing three key questions: 1) Leakage Reduction: Does PLE significantly reduce reasoning leakage (reflective tokens) in \no_think mode compared to dense baselines? 2) Mode Performance: Does PLE improve the performance (accuracy and conciseness) of the \no_think mode by reducing gradient interference? 3) Reasoning Preservation: Does the architectural separation maintain the original \think performance without degradation?

4.1 Experimental setup↩︎

Dataset Construction We construct our training set by augmenting the original Superior-Reasoning[9] corpus with synthesized \no_think targets. Specifically, we use Qwen3-235B in \no_think mode to generate direct solutions for the same problem set. These candidates undergo three rigorous filters: (1) Correctness: matching ground-truth answers; (2) Length: adhering to domain-specific token limits; and (3) Style: excluding any reflective tokens (e.g., wait, hmm). The final dataset, totaling 54k samples, combines the original reasoning traces with the filtered direct answers in a 1:1 ratio, ensuring balanced supervision for both modes.

Base model weights. We evaluate PLE across two representative model families to demonstrate its generalizability. For the Qwen2.5 family, we use Qwen2.5-7B-Instruct as the backbone. For the Qwen3 family, we employ Qwen3-4B, which serves as our primary testbed due to its native hybrid-thinking capabilities. For each base model, we initialize the PLE architecture by cloning their MLP weights, and fine-tune on our superior-reasoning dataset.

Baselines. We compare PLE against three primary baselines: (1) the Original Model (vanilla Qwen3-4B), representing the default hybrid-thinking behavior and its inherent leakage; (2) the Hybrid SFT-only method [1], a training-level mitigation strategy that uses a dense Transformer with a 140k-sample SFT recipe; and (3) the Pure Instruct represents Qwen2.5-7B-Instruct, which has zero reasoning behavior.

Benchmarks and Metrics. We evaluate all models in both \think and \no_think modes across four reasoning benchmarks: MATH500 [10], AIME24[11], MMLU-STEM[12], and GPQA-Diamond [13]. To measure performance and mode separation, we report: (1) Accuracy (%) for task proficiency; (2) Average output length to monitor verbosity; and (3) Reflective tokens per answer (#Ref.)—the mean count of self-reflective markers (e.g., wait, hmm, alternatively)—as our primary metric for reasoning leakage.

4.2 Main Results↩︎

We present a comprehensive evaluation of PLE across two model families to verify its effectiveness in mitigating reasoning leakage while maintaining task performance. Overall, our results demonstrate that PLE provides a superior Pareto frontier for hybrid thinking: it delivers a \no_think mode that is cleaner and more proficient than prior training-level recipes, while simultaneously safeguarding (or even enhancing) the model’s capacity for deep, multi-step reasoning. Our primary analysis focuses on the challenging AIME24 benchmark (3), which serves as a stress test for hybrid thinking due to its requirement for deep deliberation; results for MATH500 are consistent with these findings and are provided in 6 (with additional details in Appendix 8.1).

4.2.0.1 Effective Mode Separation with Minimal Leakage.

The core strength of PLE is its ability to achieve robust architectural mode separation. As illustrated in 3, PLE drastically reduces the emission of reflective tokens in \no_think mode compared to the Hybrid baseline and previous SFT-only methods. Specifically, in Qwen3-4B, the average reflective tokens per answer drop from 2.54 to 0.39. Crucially, this separation is maintained even as the \think mode’s complexity increases; while the \think mode produces significantly longer reasoning chains to solve AIME24 problems, the \no_think mode successfully remains concise and artifact-free. This decoupling demonstrates that physical parameter isolation effectively prevents the "thinking" behavior from leaking into the direct-answer path.

4.2.0.2 Significant Enhancement of \no_think Proficiency.

Beyond artifact reduction, PLE substantially boosts the inherent accuracy of the direct-answer mode. As shown in 3, the \no_think accuracy of PLE models consistently outperforms all baselines. Notably, for Qwen3-4B, our \no_think mode achieves a +19.3% absolute accuracy gain over the native hybrid baseline. This suggests that by dedicating a specialized MLP expert to the \no_think task, the model can produce more reliable direct answers without being confused by the high-entropy reasoning patterns required for the \think mode.

4.2.0.3 Preservation and Synergy of Reasoning Capabilities.

A vital requirement for PLE is that architectural isolation must not degrade the model’s core reasoning power. Our results confirm that the \think mode’s performance remains robust across both model families. Interestingly, we observe a synergistic effect in the Qwen2.5-7B-Instruct family: the \think mode of PLE actually outperforms the original model on AIME24 accuracy (3). This indicates that sharing the attention backbone preserves the underlying logic while the dual-expert MLP structure allows the \think mode to focus on complex deliberation without the need to compromise its internal representations for direct-answering tasks.

5 Ablation Studies↩︎

Beyond the main evaluations in 4, we conduct a series of ablation studies to dissect the impact of various design choices on PLE’s performance and mode separation behavior. Our analysis reveals that two primary factors significantly dictate the model’s efficacy: the selection of base model weights for initialization and the composition of the training dataset. In the following subsections, we explore how these factors influence the delicate balance between reasoning power and leakage mitigation.

5.1 Effect of base model weights↩︎

The choice of which pretrained model to use as the several source model for PLE is a critical design decision. We compare three base models: 1) Qwen2.5-7B-Instruct, 2) Qwen3-4B, and 3) Qwen3-4B-Base (the pretrained model without any post-training.)

All variants are trained with the same dataset (superior-reasoning 54k) and identical hyperparameters to isolate the effect of base model weights. 4 provides the results on AIME24 task, more comprehensive experimental results are provided in Appendix 8.2.1

Hybrid vs. Pure Instruct: The “Reasoning Contamination” Trade-off. Comparing Qwen3-4B (a hybrid-thinking model) with Qwen2.5-7B-Instruct (a pure instruction-tuned model) reveals a clear divergence in behavior. We find that using a pure Instruct model as the base generally yields superior mode separation, characterized by a “cleaner” \no_think mode with virtually zero leakage. In contrast, using the hybrid Qwen3-4B leads to higher absolute performance (accuracy) but carries a persistent residue of reasoning artifacts.

We hypothesize that hybrid-thinking models have been “contaminated” by reasoning-heavy thought patterns during their intensive post-training. This contamination is deeply embedded in the shared attention representations, making it difficult to fully extinguish reflective behaviors in the \no_think mode, even with architectural isolation. However, because the hybrid base already possesses a stronger foundation for the \think mode, it ultimately achieves higher task scores, suggesting that the initial “reasoning literacy” of the weights sets a higher ceiling for performance.

The Necessity of Post-training: Qwen3-4B vs. Qwen3-Base. To isolate the impact of post-training, we evaluated Qwen3-4B-Base (the raw pretrained weights). While this base shows reasonable performance on simpler benchmarks like MATH500, its performance collapses on the high-difficulty AIME24 tasks. In many cases, the base-initialized PLE fails to converge on a coherent reasoning strategy, often exceeding token limits without reaching a solution. This result underscores that PLE requires a base model that has already acquired basic instruction-following and logical structuring capabilities. Relying solely on the PLE to “learn” reasoning from scratch is insufficient for complex domains.

5.2 Effect of training dataset↩︎

We investigate the impact of dataset quality and composition on PLE training. Specifically, we comparOpenR1 dataset used in [1], constructed from OpenR1-Math [14] and Superior-reasoning: our improved dataset featuring harder reasoning questions and higher-quality responses.

The choice of initialization weights for PLE’s shared backbone and dual experts is a primary determinant of the final model’s behavior. We ablate this factor by comparing backbones with varying degrees of post-training and architectural lineage. Our findings, summarized in 5 and [tab:ablation95dataset95by95base95weight] in Appendix 8.2.2, lead to several key insights on the trade-off between task proficiency and mode purity.

We investigate the impact of dataset quality and difficulty on PLE training by comparing two distinct sources: the OpenR1 dataset (a relatively simpler collection) and our Superior-reasoning dataset (characterized by higher difficulty and longer reasoning chains). Our analysis, supported by [tab:ablation95dataset95by95base95weight] and 5, reveals how data composition interacts with architectural experts.

Difficulty-Capacity Alignment. A key observation from our ablation is that the effectiveness of a dataset is closely tied to the inherent capability of the base model. As shown in [tab:ablation95dataset95by95base95weight], the simpler OpenR1 dataset yields better relative gains when paired with the Qwen2.5-7B-Instruct backbone. Conversely, the more challenging Superior-reasoning dataset excels when paired with the more capable Qwen3-4B model.

For the Qwen3-4B backbone, the advantage of the superior dataset is particularly pronounced in the \no_think mode, where it significantly outperforms the OpenR1 baseline in accuracy. This suggests that as base models become more proficient, they require higher-quality, “harder” data to fully specialize their dual experts. We conclude that dataset “difficulty” must be aligned with the base model’s capacity to achieve optimal results.

The Leakage-Performance Trade-off. While the Superior-reasoning dataset provides clear performance benefits, it also induces more noticeable reasoning leakage compared to OpenR1. We attribute this to the nature of the data: the superior dataset contains significantly longer and more complex chain-of-thought (CoT) traces. These high-intensity reasoning patterns appear to be more “contagious,” exerting greater pressure on the architectural separation and making the \no_think expert more prone to emitting reflective artifacts.

This represents a fundamental trade-off in hybrid thinking: achieving the highest levels of reasoning proficiency often requires sacrificing a degree of mode-control purity. In most practical scenarios, we argue that this trade-off is acceptable, as the gains in \no_think accuracy and \think depth outweigh the presence of minor leakage artifacts.

6 Related Work↩︎

LLM Reasoning. Reasoning in LLMs has been advanced through reinforcement learning (RL) [15]–[21], supervised fine-tuning (SFT) [22]–[25], and methods that add formal structure, uncertainty estimates, neurosymbolic components, or verification procedures to reasoning procedures [26]–[29]. These approaches primarily aim to improve the quality, validity, or trustworthiness of reasoning once reasoning is invoked. Our work is orthogonal: rather than making reasoning stronger or more verifiable, we focus on controlling when reasoning occurs through architectural design.

Hybrid Thinking and Efficient Reasoning. Hybrid thinking [5], [6], [30] lets models switch between chain-of-thought and direct answering via control tokens, and has been adopted by several recent systems [2]–[4], [31]. However, reasoning behaviors still leak into \no_think mode. [1] showed that training-level interventions, including data scaling, think/no-think ratio tuning, and multi-stage schedules, can reduce but cannot eliminate this leakage. Other efficiency-oriented approaches include CoT compression [7], [32]–[34], early stopping [35], preference optimization [36], and token-level or intermediate-budget reasoning control [37]. Unlike all of these, we address leakage at the architectural level.

Mixture-of-Experts. MoE architectures [38]–[42] use learned token-level routers for conditional computation, but face challenges including load imbalance, training instability, and the need for auxiliary balancing losses.

7 Conclusion↩︎

In this paper, we argued that reasoning leakage in hybrid-thinking models is partly an architectural problem. We introduced Path-Lock Expert (PLE), a minimal modification that replaces each decoder MLP with two semantically locked experts routed deterministically by control tokens. Across math and science benchmarks on two model families, PLE substantially strengthens the \no_think mode—reducing reflective tokens, shortening outputs, and improving accuracy—while preserving \think performance. Our ablations further show that PLE’s effectiveness depends on base model initialization and dataset–capacity alignment. These results demonstrate that separating mode-specific feed-forward pathways is a straightforward and effective complement to training-level interventions for achieving more controllable hybrid thinking models.

Acknowledgements↩︎

This work was supported in part by NSF award 2117439 and 2112606.

8 Appendix↩︎

8.1 Supplemental Results for Main Experiment Results↩︎

8.2 Supplemental Results for Ablation Studies↩︎

8.2.1 Supplemental Results for Base Model Weight Ablation↩︎

[tab:appendix_ablation_base_openr1_mmlu_gpqa,tab:appendix_ablation_base_superior_mmlu_gpqa,tab:appendix_ablation_dataset_mmlu_gpqa] extend the ablations beyond the mathematical benchmarks and clarify the scope of our claims.

8.2.1.1 MMLU-STEM often favors direct answering more than long externalized reasoning.

Across several initializations, \no_think matches or even exceeds \think accuracy on MMLU-STEM while remaining short and nearly artifact-free. This is consistent with the structure of the benchmark: many questions reward factual recall or short derivations rather than long-form search. These results reinforce the practical motivation for hybrid thinking. A well-behaved \no_think mode should not be interpreted as a fallback setting; on knowledge-heavy multiple-choice problems, it may be the more appropriate operating mode.

8.2.1.2 GPQA-Diamond is more backbone-dependent.

The GPQA results are more mixed. \no_think leakage remains low, but accuracy depends strongly on the initialization and training dataset. We view this as an important boundary of our claim. PLE addresses mode interference; it does not by itself supply missing scientific knowledge or guarantee uniformly strong direct answering on every benchmark. The most stable conclusion on GPQA is therefore about control rather than universal accuracy gains.

8.2.1.3 Why these supplementary results matter.

Taken together with the main math results, the appendix suggests a simple task-dependent picture. Architectural separation is most valuable when a model must clearly choose between “reason deeply” and “answer directly.” On benchmarks that mainly test retrieval or short reasoning, the main benefit of PLE is a clean and efficient \no_think mode; on long-horizon mathematical benchmarks, it can also substantially improve \no_think accuracy.

5pt

5pt

5pt

5pt

8.2.2 Supplemental Results for Dataset Ablation↩︎

4pt

4pt

9 Formal Architecture Specification↩︎

Path-locked dual-expert decoder Consider a standard pre-norm decoder layer: \[\begin{align} \mathbf{h}^{(l)}_{\text{attn}} &= \mathbf{h}^{(l)} + \mathrm{Attn}^{(l)}\!\left(\mathrm{LN}^{(l)}_1(\mathbf{h}^{(l)})\right); \mathbf{h}^{(l+1)} = \mathbf{h}^{(l)}_{\text{attn}} + \mathrm{MLP}^{(l)}\!\left(\mathrm{LN}^{(l)}_2(\mathbf{h}^{(l)}_{\text{attn}})\right). \end{align}\]

PLE changes only the second part. Each decoder MLP is replaced by a pair of experts, and the selected route \(r \in \{0,1\}\) determines which expert is active: \[\begin{align} \mathbf{h}^{(l+1)} = \mathbf{h}^{(l)}_{\text{attn}} + \mathrm{MLP}^{(l)}_{r}\!\left(\mathrm{LN}^{(l)}_2(\mathbf{h}^{(l)}_{\text{attn}})\right), \end{align}\] where \(r=0\) denotes the \no_think expert and \(r=1\) denotes the \think expert. For the Qwen backbones used in our main experiments, each expert preserves the source model’s original SwiGLU-style MLP parameterization: \[\begin{align} \mathrm{MLP}^{(l)}_{r}(\mathbf{x}) = W^{(l,r)}_{\mathrm{down}} \Big( \phi\!\left(W^{(l,r)}_{\mathrm{gate}}\mathbf{x}\right) \odot W^{(l,r)}_{\mathrm{up}}\mathbf{x} \Big), \end{align}\] where \(\phi(\cdot)\) is the activation function and \(\odot\) is elementwise multiplication. The two experts are structurally identical; they differ only in parameters and in which control mode activates them. All non-MLP components remain shared across modes, including self-attention, positional encoding, normalization, token embeddings, and the LM head. We duplicate only the MLP because the two modes should share the same contextualization and stored knowledge, but need not share the final feed-forward transformation that turns those representations into long-form reasoning or concise direct answers. This makes PLE a targeted intervention rather than a full model duplication scheme.

Deterministic control-token routing PLE does not use a learned router. Instead, routing is determined by control tokens in the input. We register two tokenizer-level special tokens, /no_think and /think, each represented by a single token ID. Given an input sequence \(\mathbf{x}\), the routing function is \[\begin{align} r(\mathbf{x}) = \begin{cases} 1, & \text{if the final control token in } \mathbf{x} \text{ is } \texttt{/think},\\ 0, & \text{if the final control token in } \mathbf{x} \text{ is } \texttt{/no\_think},\\ r_{\mathrm{default}}, & \text{if no control token is present}, \end{cases} \end{align}\] where \(r_{\mathrm{default}}=0\) routes to the \no_think expert. The last control token wins rule serves two purposes. First, it makes routing fully deterministic and interpretable from the prompt alone. Second, it allows a later template or system token to override an earlier user-supplied token without introducing any learned gating behavior.

During autoregressive generation, the route is determined from the accumulated prompt at the first decoding step and then cached and reused for all later steps. This ensures that the active expert remains fixed throughout decoding. Our current implementation uses one scalar routing index per forward pass. Accordingly, we do not mix \think and \no_think examples within the same batch; each forward pass uses a single mode. This keeps the routing logic simple while preserving the intended turn-level control semantics.

Routing-conditioned supervised fine-tuning We fine-tune PLE with standard causal language modeling loss on chat-formatted examples tagged with either /think or /no_think. Let \(\mathcal{B}_r\) denote a minibatch routed to mode \(r \in \{0,1\}\). The training objective is \[\begin{align} \mathcal{L}(\theta;\mathcal{B}_r) = - \sum_{(\mathbf{x},\mathbf{y}) \in \mathcal{B}_r} \sum_{t=1}^{|\mathbf{y}|} \log p_{\theta}\!\left(y_t \mid \mathbf{x}, \mathbf{y}_{<t}; r\right). \end{align}\]

Let \(\theta_{\mathrm{sh}}\) denote the shared parameters and \(\theta_0,\theta_1\) denote the two expert parameter sets. Because only the routed expert is executed, for any batch using route \(r\), \[\begin{align} \frac{\partial \mathcal{L}}{\partial \theta_{1-r}} = 0, \qquad \frac{\partial \mathcal{L}}{\partial \theta_r} \neq 0, \qquad \frac{\partial \mathcal{L}}{\partial \theta_{\mathrm{sh}}} \neq 0. \end{align}\]

The inactive expert therefore receives no gradient, while the shared backbone is updated by both modes. This is the central mechanism of PLE: it removes direct parameter sharing between the two MLP pathways while still allowing both modes to benefit from a common representational substrate. At the same time, because attention and output layers remain shared, PLE should be understood as targeted architectural separation, not as full model duplication.

9.0.0.1 Active-compute scope.

Because exactly one expert is active for a routed response, PLE does not double the active MLP computation per generated token relative to the source dense model. It instead trades additional parameters for cleaner mode control. Detailed GPU-level throughput and distributed-training effects depend on systems factors outside the scope of this architecture study [43].

9.0.0.2 Filtered supervision and route purity.

The filtered \no_think targets are used only to make the routed supervision mode-pure. We require correctness, concision, and absence of reflective markers so that examples assigned to the \no_think expert do not update it toward long-form deliberation. This use of filtering is narrower than broader data-curation or post-hoc reliability methods that identify problematic examples, atypical representations, visual concepts, logs, or generations after they arise [44]–[48]. In PLE, filtering prepares the supervision, while the route-locked architecture is the mechanism that enforces mode separation during generation.

10 Theoretical account: Path-Lock as partial block-diagonalization↩︎

This section formalizes PLE as a partially decoupled two-objective optimization problem. The derivation is guided by three observations from prior work: (i) multi-task learning is inherently multi-objective and can suffer from conflicting gradients [49]–[51]; (ii) expert routing is useful when different objectives prefer different submodels [52], [53]; and (iii) Transformer feed-forward layers are a privileged locus for token-level prediction shaping [54]–[56].

10.0.0.1 Setup.

We model hybrid thinking as conditional language modeling with an observed mode variable \(r \in \{0,1\}\), where \(r=0\) denotes \no_think and \(r=1\) denotes \think. Let \[\begin{align} \alpha &= \text{shared parameters (embeddings, attention, norms, LM head)}, \\ \beta_0 &= \text{no-think expert parameters}, \\ \beta_1 &= \text{think expert parameters}. \end{align}\] Thus the full parameter vector is \[\begin{align} \theta = (\alpha,\beta_0,\beta_1). \end{align}\]

For an example \((x,y,r)\), the PLE model defines \[\begin{align} p_\theta(y \mid x,r) = \prod_{t=1}^{|y|} p_\theta\!\left(y_t \mid x,y_{<t}; \alpha,\beta_r\right). \label{eq:cond95lm} \end{align}\tag{1}\]

10.0.0.2 Observed-route formulation.

Introduce an expert variable \(z \in \{0,1\}\). Because routing is deterministic from the control token \(c\), \[p(z \mid c) = \mathbf{1}\{z = r(c)\}, \label{eq:observed95route}\tag{2}\] where \(r(c)\) is the route selected by the control-token rule. Therefore \[\begin{align} p_\theta(y \mid x,c) &= \sum_{z \in \{0,1\}} p(z \mid c)\, p_\theta(y \mid x,z) \\ &= \sum_{z \in \{0,1\}} \mathbf{1}\{z = r(c)\}\, p_\theta(y \mid x,z) \\ &= p_\theta(y \mid x,r(c)). \end{align}\] Hence the route variable is observed, not latent. In this observed-route setting, a learned router is not statistically necessary for identifying the routed conditional model; outside this setting (e.g., if the route is unobserved or must be inferred at test time), a learned router may still be useful or necessary.

10.0.0.3 Modeling assumption.

The reduction from the control token \(c\) to the observed route \(r(c)\) assumes that, within this theoretical abstraction, \(c\) affects the conditional distribution only through route selection: \[\begin{align} p_\theta(y \mid x,c,z) = p_\theta(y \mid x,z). \label{eq:control95only95via95route} \end{align}\tag{3}\] Equivalently, once the route \(z\) is fixed, the control token carries no additional route-independent signal for prediction. If, in the implemented model, \(c\) is also processed by the shared backbone through a distinct embedding or any other route-specific component, then the identities below should be read as an abstraction that isolates routing from those additional effects.

10.0.0.4 Dataset decomposition.

Let \[\begin{align} \mathcal{D} = \mathcal{D}_0 \sqcup \mathcal{D}_1, \end{align}\] where \(\mathcal{D}_0\) contains no-think examples and \(\mathcal{D}_1\) contains think examples. Define mode frequencies \[\begin{align} \pi_r = \frac{|\mathcal{D}_r|}{|\mathcal{D}|}, \qquad \pi_0 + \pi_1 = 1. \end{align}\] For each mode, define the expected autoregressive loss \[\mathcal{L}_r(\alpha,\beta_r) = \mathbb{E}_{(x,y)\sim \mathcal{D}_r} \left[ -\sum_{t=1}^{|y|} \log p_\theta(y_t \mid x,y_{<t}; \alpha,\beta_r) \right]. \label{eq:mode95loss}\tag{4}\] Then the full PLE objective is \[\begin{align} \mathcal{L}_{\mathrm{PLE}}(\alpha,\beta_0,\beta_1) &= \mathbb{E}_{(x,y,r)\sim \mathcal{D}} \left[ -\sum_{t=1}^{|y|} \log p_\theta(y_t \mid x,y_{<t}; \alpha,\beta_r) \right] \\ &= \sum_{r\in\{0,1\}} \pi_r \mathcal{L}_r(\alpha,\beta_r) \\ &= \pi_0 \mathcal{L}_0(\alpha,\beta_0) + \pi_1 \mathcal{L}_1(\alpha,\beta_1). \label{eq:ple95decomposition} \end{align}\tag{5}\]

Unless stated otherwise, all exact statements below refer to the example-weighted objective in 5 , with \(\pi_r = |\mathcal{D}_r|/|\mathcal{D}|\). The later length-asymmetry discussion explicitly switches to a token-summed training heuristic. If training instead uses token-weighted normalization, or adds auxiliary router losses, cross-expert regularizers, or other route-coupling terms, then the gradient and Hessian identities below acquire additional additive terms.

10.0.0.5 Pointwise consequence.

For a single example \((x_i,y_i,r_i)\), define \[\begin{align} \ell_i(\alpha,\beta_0,\beta_1) = -\sum_{t=1}^{|y_i|} \log p_\theta(y_{i,t} \mid x_i,y_{i,<t}; \alpha,\beta_{r_i}). \end{align}\] Then for any \(k\in\{0,1\}\), \[\begin{align} \frac{\partial \ell_i}{\partial \beta_k} = \begin{cases} -\displaystyle\sum_{t=1}^{|y_i|} \frac{\partial}{\partial \beta_{r_i}} \log p_\theta(y_{i,t}\mid x_i,y_{i,<t};\alpha,\beta_{r_i}), & k=r_i,\\[10pt] 0, & k\neq r_i. \end{cases} \label{eq:pointwise95grad} \end{align}\tag{6}\] So the inactive expert receives exactly zero gradient from the route-conditioned likelihood term on that example. So the inactive expert receives exactly zero gradient on that example.

All curvature-based statements below assume the relevant local second derivatives exist and that differentiation may be interchanged with expectation in the usual way.

10.0.0.6 Partial block-diagonalization of the curvature.

Let \[\begin{align} H_{ab}^{(r)} := \nabla^2_{a,b}\mathcal{L}_r, \qquad a,b \in \{\alpha,\beta_0,\beta_1\}. \end{align}\] Because \(\mathcal{L}_0\) does not depend on \(\beta_1\) and \(\mathcal{L}_1\) does not depend on \(\beta_0\), \[\begin{align} \nabla^2_{\beta_0\beta_1}\mathcal{L}_{\mathrm{PLE}} = 0, \qquad \nabla^2_{\beta_1\beta_0}\mathcal{L}_{\mathrm{PLE}} = 0. \end{align}\] Hence the Hessian takes the form \[\begin{align} \nabla^2 \mathcal{L}_{\mathrm{PLE}} = \begin{bmatrix} \pi_0 H^{(0)}_{\alpha\alpha} + \pi_1 H^{(1)}_{\alpha\alpha} & \pi_0 H^{(0)}_{\alpha\beta_0} & \pi_1 H^{(1)}_{\alpha\beta_1} \\[3pt] \pi_0 H^{(0)}_{\beta_0\alpha} & \pi_0 H^{(0)}_{\beta_0\beta_0} & 0 \\[3pt] \pi_1 H^{(1)}_{\beta_1\alpha} & 0 & \pi_1 H^{(1)}_{\beta_1\beta_1} \end{bmatrix}. \label{eq:block95hessian} \end{align}\tag{7}\] Equation 7 is the precise sense in which PLE partially block-diagonalizes the hybrid-thinking optimization problem: the expert blocks are exactly decoupled from each other, while both remain coupled to the shared backbone.

This block structure is exact only for the direct expert–expert interactions in the stated objective. It does not imply full disentanglement over training, because both modes remain coupled through the shared parameters \(\alpha\): an update from one mode can change \(\alpha\), which in turn changes future gradients for the other mode.

10.0.0.7 Dense hybrid training as a conflicting shared-objective problem.

Now consider a dense baseline with the same shared backbone \(\alpha\) but a single shared MLP parameter vector \(\beta\). Its objective is \[\begin{align} \mathcal{L}_{\mathrm{dense}}(\alpha,\beta) = \pi_0 \mathcal{L}_0(\alpha,\beta) + \pi_1 \mathcal{L}_1(\alpha,\beta). \label{eq:dense95obj} \end{align}\tag{8}\] Let \[\begin{align} g_0 := \nabla_\beta \mathcal{L}_0(\alpha,\beta), \qquad g_1 := \nabla_\beta \mathcal{L}_1(\alpha,\beta). \end{align}\] Then the dense expert gradient is \[\nabla_\beta \mathcal{L}_{\mathrm{dense}} = \pi_0 g_0 + \pi_1 g_1. \label{eq:dense95grad}\tag{9}\]

By contrast, under the same full-objective weighting, the expected PLE update for the no-think expert is \[\begin{align} \Delta \beta_0 = -\eta \pi_0 g_0, \end{align}\] so \[\begin{align} \mathcal{L}_0(\beta_0+\Delta\beta_0) = \mathcal{L}_0(\beta_0) - \eta \pi_0 \|g_0\|^2 + O(\eta^2), \end{align}\] with no cross-term involving \(g_1\). If one instead conditions on a mode-\(0\) minibatch, the same calculation gives \(\Delta\beta_0=-\eta g_0\). Thus PLE removes direct cross-mode interference exactly in the expert parameters, while indirect coupling through the shared backbone remains.

10.0.0.8 A local quadratic theory of the dense compromise.

To isolate the mechanism, fix the shared parameters at some local value \(\bar{\alpha}\) and write \[\begin{align} L_r(\beta) := \mathcal{L}_r(\bar{\alpha},\beta). \end{align}\]

Let \(\beta_r^\star\) denote a local minimizer of \(L_r\), and let \(H_r \succeq 0\) be the local Hessian at that point. A second-order Taylor expansion gives \[L_r(\beta) \approx L_r(\beta_r^\star) + \frac{1}{2} (\beta-\beta_r^\star)^\top H_r (\beta-\beta_r^\star). \label{eq:quadratic95mode}\tag{10}\]

For the remainder of this subsection, it is convenient to promote 10 to the local surrogate \[\begin{align} \widetilde{L}_r(\beta) := L_r(\beta_r^\star) + \frac{1}{2} (\beta-\beta_r^\star)^\top H_r (\beta-\beta_r^\star), \label{eq:quadratic95surrogate} \end{align}\tag{11}\] and to interpret all equalities below in this subsection as statements about the surrogate \(\widetilde{L}_r\), not the exact nonquadratic loss \(L_r\).

The dense expert objective becomes \[\begin{align} L_{\mathrm{dense}}(\beta) &= \pi_0 L_0(\beta)+\pi_1 L_1(\beta) \\ &\approx \pi_0 L_0(\beta_0^\star)+\pi_1 L_1(\beta_1^\star) + \frac{\pi_0}{2}(\beta-\beta_0^\star)^\top H_0(\beta-\beta_0^\star) + \frac{\pi_1}{2}(\beta-\beta_1^\star)^\top H_1(\beta-\beta_1^\star). \label{eq:dense95quad95start} \end{align}\tag{12}\]

Expand the first quadratic term: \[\begin{align} (\beta-\beta_0^\star)^\top H_0(\beta-\beta_0^\star) = \beta^\top H_0 \beta - 2\beta^\top H_0 \beta_0^\star + (\beta_0^\star)^\top H_0 \beta_0^\star. \end{align}\] Expand the second: \[\begin{align} (\beta-\beta_1^\star)^\top H_1(\beta-\beta_1^\star) = \beta^\top H_1 \beta - 2\beta^\top H_1 \beta_1^\star + (\beta_1^\star)^\top H_1 \beta_1^\star. \end{align}\] Substitute both expansions into 12 : \[\begin{align} L_{\mathrm{dense}}(\beta) &\approx C + \frac{1}{2} \beta^\top(\pi_0 H_0+\pi_1 H_1)\beta - \beta^\top(\pi_0 H_0 \beta_0^\star + \pi_1 H_1 \beta_1^\star), \label{eq:dense95quad95expanded} \end{align}\tag{13}\] where \[\begin{align} C = \pi_0 L_0(\beta_0^\star)+\pi_1 L_1(\beta_1^\star) + \frac{\pi_0}{2}(\beta_0^\star)^\top H_0 \beta_0^\star + \frac{\pi_1}{2}(\beta_1^\star)^\top H_1 \beta_1^\star. \end{align}\]

Differentiate 13 with respect to \(\beta\): \[\begin{align} \nabla_\beta L_{\mathrm{dense}}(\beta) = (\pi_0 H_0+\pi_1 H_1)\beta - (\pi_0 H_0 \beta_0^\star+\pi_1 H_1 \beta_1^\star). \end{align}\] Setting the gradient to zero gives \[\begin{align} (\pi_0 H_0+\pi_1 H_1)\beta_{\mathrm{dense}}^\star = \pi_0 H_0 \beta_0^\star+\pi_1 H_1 \beta_1^\star, \end{align}\] hence \[\begin{align} \beta_{\mathrm{dense}}^\star = (\pi_0 H_0+\pi_1 H_1)^{-1} (\pi_0 H_0 \beta_0^\star+\pi_1 H_1 \beta_1^\star), \label{eq:dense95optimum} \end{align}\tag{14}\] assuming \(\pi_0 H_0+\pi_1 H_1\) is invertible.

Equation 14 shows that the dense MLP is forced to sit at a curvature-weighted compromise between the two mode-specific optima.

Equation ?? is the formal version of the core intuition for the local quadratic surrogate: if think and no-think prefer different MLP parameters in directions with nonzero local curvature, then a dense hybrid MLP pays a nonzero compromise penalty. PLE removes that surrogate penalty in the expert subspace by allowing each mode to occupy its own local optimum.

Equation ?? is especially useful for intuition: the dense compromise penalty grows with (i) the frequency of both modes, through \(\pi_0\pi_1\), (ii) the geometric separation between their preferred optima, and (iii) the curvature in the directions where they disagree.

10.0.0.9 Why sequence-level routing matters.

For sequence-level routing, each example has a single route \(r_i\), so \[\begin{align} \ell_i = \sum_{t=1}^{|y_i|} \ell_{i,t}(\alpha,\beta_{r_i}). \end{align}\] Thus \[\begin{align} \nabla_{\beta_k}\ell_i = \mathbf{1}\{k=r_i\} \sum_{t=1}^{|y_i|} \nabla_{\beta_k}\ell_{i,t}. \label{eq:sequence95level95grad} \end{align}\tag{15}\] A single response updates at most one expert.

If routing were token-level, with \(r_{i,t}\in\{0,1\}\), then \[\begin{align} \ell_i^{\mathrm{token}} = \sum_{t=1}^{|y_i|} \ell_{i,t}(\alpha,\beta_{r_{i,t}}), \end{align}\] and hence \[\begin{align} \nabla_{\beta_k}\ell_i^{\mathrm{token}} = \sum_{t=1}^{|y_i|} \mathbf{1}\{k=r_{i,t}\} \nabla_{\beta_k}\ell_{i,t}. \label{eq:token95level95grad} \end{align}\tag{16}\] Now the same response can update both experts, weakening the clean decoupling in 6 and 15 . This is why sequence-level routing is the natural match to a turn-level \think/\no_think interface.

10.0.0.10 Why identical initialization matters.

PLE initializes both experts from the same dense source MLP: \[\beta_0^{(0)} = \beta_1^{(0)} = \beta_{\mathrm{src}}. \label{eq:identical95init}\tag{17}\] If the two routes differ only through the duplicated expert weights at initialization, then before fine-tuning \[\begin{align} p_\theta(y \mid x,r=0) = p_\theta(y \mid x,r=1) = p_{\mathrm{src}}(y \mid x). \end{align}\] When newly introduced control-token embeddings or other route-specific components are present, this equality need not hold exactly. In that case, the intended claim is narrower: the two experts start from identical weights, so specialization is not driven by random asymmetry in the expert parameters.

The update identities below are exact for plain SGD with matched learning rates and no optimizer state. With momentum, Adam, gradient clipping, or asynchronous minibatch schedules, the same conclusion is qualitative rather than algebraically exact.

After one SGD step on a no-think batch and one SGD step on a think batch, \[\begin{align} \beta_0^{(1)} &= \beta_{\mathrm{src}} - \eta g_0^{(0)}, \\ \beta_1^{(1)} &= \beta_{\mathrm{src}} - \eta g_1^{(0)}. \end{align}\] Subtracting yields \[\begin{align} \beta_1^{(1)} - \beta_0^{(1)} = -\eta\big(g_1^{(0)} - g_0^{(0)}\big). \end{align}\] After \(T\) paired updates, \[\begin{align} \beta_1^{(T)} - \beta_0^{(T)} = -\eta \sum_{s=0}^{T-1} \big(g_1^{(s)} - g_0^{(s)}\big). \label{eq:specialization95trajectory} \end{align}\tag{18}\] Thus expert divergence is driven purely by the cumulative difference between mode-specific gradients.

10.0.0.11 Why duplicating only the MLP can be sufficient.

Let \(u_t\) denote the shared residual-stream representation entering a routed MLP block, and let \(f_{\beta_r}(u_t)\) denote the routed expert output. Abstract the remainder of the network (all later layers plus the LM head) as a differentiable map \(G\). Then the route-\(r\) logits can be written locally as \[\begin{align} o_t^{(r)} = G\big(u_t + f_{\beta_r}(u_t)\big). \end{align}\] A first-order expansion around \(u_t\) gives

\[o_t^{(r)} \approx G(u_t) + J_G(u_t)\, f_{\beta_r}(u_t), \label{eq:mlp95linearization}\tag{19}\] where \(J_G(u_t)\) is the Jacobian of \(G\) at \(u_t\). This linearization should be interpreted as a local sensitivity argument rather than a global proof of sufficiency. It is accurate when the downstream map \(G\) is locally smooth around \(u_t\) and the routed residual update \(f_{\beta_r}(u_t)\) remains within a neighborhood where higher-order terms are small; in affine or shared linear-region regimes, the relation is exact. Hence the think/no-think logit difference is approximately \[\begin{align} o_t^{(1)} - o_t^{(0)} \approx J_G(u_t)\Big(f_{\beta_1}(u_t)-f_{\beta_0}(u_t)\Big). \label{eq:route95logit95difference} \end{align}\tag{20}\] Equations 19 –20 make the architectural intuition explicit: the shared backbone can keep constructing a common contextual representation \(u_t\), while the mode-specific MLPs directly control how that representation is pushed toward different token distributions. In this sense, PLE separates expression policy more than core competence.

10.0.0.12 A length-induced asymmetry heuristic.

Let mode \(r\) have \(N_r\) examples with average target length \(m_r\), and let \(g_{i,t}^{(r)}\) denote the per-token gradient contribution in the expert subspace. A token-summed dense objective produces an epoch-level update \[\begin{align} \Delta \beta_{\mathrm{dense}} \propto - \sum_{r\in\{0,1\}} \sum_{i=1}^{N_r} \sum_{t=1}^{m_i^{(r)}} g_{i,t}^{(r)}. \end{align}\] If think targets are much longer than no-think targets, then even with similar example counts, the dense expert receives more cumulative update mass from think-mode tokens. PLE changes this to \[\begin{align} \Delta \beta_0 \propto - \sum_{i=1}^{N_0} \sum_{t=1}^{m_i^{(0)}} g_{i,t}^{(0)}, \qquad \Delta \beta_1 \propto - \sum_{i=1}^{N_1} \sum_{t=1}^{m_i^{(1)}} g_{i,t}^{(1)}, \end{align}\] so long think traces can no longer wash over the no-think expert. This gives a conditional explanation for why PLE may especially strengthen the cleanliness and stability of \no_think mode when optimization effectively scales with token count and think targets are substantially longer.

References↩︎