1 Introduction↩︎

Neural sentence embeddings serve as the backbone of modern retrieval-centric applications, including Retrieval-Augmented Generation (RAG) pipelines [1], yet they suffer from limited interpretability [2]. According to the superposition hypothesis, dense vectors often compress sparse features into fewer dimensions, rendering them polysemantic [3], [4]. This opacity creates a fundamental alignment gap [5], making it difficult to verify or intervene when retrieval processes diverge from human intent.

To bridge this gap, we propose applying Sparse Autoencoders (SAEs) [6] to the final output of sentence encoders, such as the E5 model [7]. While SAEs are typically used to study internal LLM residual streams [8], [9], their potential to interpret dense retrievers remains underexplored. Specifically, we utilize a Top-k SAE architecture [10] to map dense embeddings into a higher-dimensional, sparse latent space. This methodology allows us to decompose entangled representations into monosemantic features that are human-interpretable, expanding the 1,024-dimensional input into a significantly larger latent space to resolve feature superposition.

In this work, we demonstrate that Top-k SAEs can effectively align dense sentence embeddings with human conceptual frameworks. Our experiments on the WikiText-103-v1 dataset show that the learned latent features correspond to granular concepts. Furthermore, we go beyond static analysis to demonstrate semantic steering. By manually clamping specific latent neurons, we show that it is possible to control the retrieval mechanism—filtering out unwanted concepts without retraining the backbone model. This capability addresses the limitations of macro-steering by enabling micro-level control over semantic features. This work suggests that SAE-based decomposition is a viable path toward transparent, aligned, and steerable neural search systems.

2 Methodology↩︎

We propose a framework to disentangle and align the dense representations of neural retrievers using a Top-k Sparse Autoencoder (SAE). This approach maps entangled dense embeddings into a high-dimensional sparse latent space where individual dimensions represent monosemantic concepts.

2.1 Top-k Sparse Autoencoder Architecture↩︎

We adopt the Top-k SAE architecture to overcome the limitations of traditional \(L_1\)-regularized autoencoders. While \(L_1\) penalties effectively induce sparsity, they often introduce a shrinkage bias that suppresses feature activation magnitudes, degrading reconstruction quality. [8], [11] The Top-k approach addresses this by directly enforcing a hard constraint [12]: for every input embedding \(\mathbf{x}\), only the \(k\) most active latent neurons are retained, while the rest are set to zero.

Our model projects the input embedding into a latent space \(\mathbb{R}^{d_{latent}}\) (where \(d_{latent} \gg d_{model}\)), applies the Top-k activation function, and reconstructs the original vector from these sparse features. Following the approach of [10], we tie the encoder and decoder weights to stabilize training. This ensures that the model learns a consistent and interpretable decomposition of the semantic space without the trade-offs inherent in soft sparsity constraints.

For the detailed mathematical formulation, and the auxiliary training objective used to mitigate dead neurons, please refer to Appendix 6.

2.2 Latent Feature Steering via Clamping↩︎

A key contribution of our work is the ability to intervene in the retrieval process via latent feature clamping. Since each dimension \(i\) of \(\mathbf{z}\) corresponds to an interpretable concept, we can surgically modify the latent vector to filter out unwanted semantic attributes from the retrieval results.

Given an input query \(\mathbf{x}\), we compute its sparse representation \(\mathbf{z}\). We define a clamping operation \(C(\mathbf{z}, \mathcal{I}_{\text{clamp}})\) where \(\mathcal{I}_{\text{clamp}}\) is the set of target neuron indices to deactivate: \[\mathbf{z}_{\text{steered}}^{(i)} = \begin{cases} 0 & \text{if } i \in \mathcal{I}_{\text{clamp}} \\ \mathbf{z}^{(i)} & \text{otherwise} \end{cases}\] By strictly zeroing out specific activations, this operation effectively removes corresponding concepts without affecting other semantic components. The steered embedding \(\hat{\mathbf{x}}_{\text{steered}} = \mathbf{W}_{\text{dec}}\mathbf{z}_{\text{steered}}\) is then used for similarity search, enabling precise, inference-time control over the retrieval mechanism without fine-tuning the backbone model.

2.3 Automated Feature Annotation Pipeline↩︎

To scale the interpretation of thousands of latent features, we utilize an automated pipeline powered by GPT-4o-mini. The process follows three stages:

1. Orthogonality-based Filtering. We prioritize distinct features by calculating the Decoder Orthogonality (DO) for each neuron. Only neurons with a mean pairwise cosine similarity below a strict threshold are selected, ensuring that the annotated features are geometrically and semantically well-separated.

2. Context Extraction. For each target neuron, we retrieve the top-\(N\) sentences (e.g., \(N=10\)) from the corpus that yield the highest activation values, providing a representative context of the learned pattern.

3. LLM Labeling with Coherence Check. An LLM acts as a linguist to identify common patterns across the retrieved sentences. Crucially, a Coherence Check is enforced: if the sentences lack a clear common thread, the neuron is classified as Unlabelable. Otherwise, the LLM generates a specific Label and assigns a Category (e.g., Entity, Topic, Syntactic, or Pragmatic). Full details of the annotation prompts are provided in Appendix 9.

3 Experiments↩︎

3.1 Experimental Setup↩︎

Dataset & Model. We utilize the WikiText-103-v1 dataset [13] as our primary corpus. To ensure the quality of semantic representations, we applied a rigorous preprocessing pipeline involving sentence segmentation and length-based filtering, resulting in a final dataset of approximately 3.8 million sentences. Further details on the data preprocessing are provided in Appendix 7.

For the backbone model, we employ intfloat/e5-large-v2, which generates 1,024-dimensional dense embeddings.

SAE Configuration. We train a Top-k SAE with a latent expansion factor of \(12\times\), resulting in a latent dimension of \(d_{latent} = 12,288\) given the input dimension of \(d_{model}=1,024\). The sparsity level is fixed at \(k=32\). To mitigate the dead neuron problem and ensure efficient feature utilization, we employ an auxiliary loss with \(\alpha=0.1\).

3.2 Quantitative Analysis: Disentanglement Quality↩︎

To verify that the SAE effectively disentangles the dense embedding space, we analyze two key metrics: Decoder Orthogonality and Reconstruction Fidelity. For detailed mathematical definitions of these metrics, please refer to Appendix 8. Based on the trade-off between semantic separability and information retention, we selected the model with a latent dimension of \(d_{latent}=12,288\) (\(12\times\) expansion) for our subsequent steering experiments.

Decoder Orthogonality (DO). A key prerequisite for interpretability is that learned features should represent distinct, non-overlapping concepts. We quantify this by measuring the mean pairwise cosine similarity between the columns of the decoder matrix \(\mathbf{W}_{\text{dec}}\). As presented in Table 1, our \(12\times\) model achieves a remarkably low mean orthogonality score of 0.0408. Figure 1 further illustrates the distribution of these pairwise similarities. The log-scale histogram reveals that the vast majority of feature pairs exhibit near-zero similarity, confirming that the learned semantic directions are structurally independent rather than redundant.

Reconstruction Fidelity. Despite the aggressive sparsity constraint (\(k=32\)), the model maintains high fidelity with an Explained Variance (EV) of 0.9259. This corresponds to a Fraction of Variance Unexplained (FVU) of less than 0.08, demonstrating that the sparse decomposition effectively preserves the essential semantic information of the original dense embeddings.

Feature Vitality and Sparsity. We analyze the activation frequency across the corpus to evaluate latent space utilization. As shown in Figure 2, the model exhibits a healthy long-tailed distribution with zero dead neurons (0.74% mortality). The median activation frequency of \(\approx 10^{-5}\) confirms that features capture granular, rare contexts rather than generic noise, providing a robust foundation for subsequent steering experiments.

3.3 Analysis of Auto-Labeled Features↩︎

We applied our automated pipeline to the trained Top-k SAE using GPT-4o-mini. Out of the neurons satisfying the orthogonality threshold, the LLM successfully generated coherent labels for the majority of features, confirming that the low-orthogonality criterion effectively filters for monosemantic concepts.

Granularity of Concepts. Unlike standard clustering which often stops at broad topics (e.g., “Politics” or “Arts”), the auto-labeled features exhibit remarkable granularity. For instance, the model captures specific abstract topics such as “Critical Reception of Media” (#7400) rather than general “Reviews,” and distinguishes specific syntactic tokens like “Reference to Number 21” (#230) from general numeric values. As shown in Table 2, these features span across diverse categories including specific entities, pragmatic functions, and syntactic patterns.

3.4 Latent Steering for Alignment↩︎

Building on the interpretability confirmed above, we demonstrate the capability to align retrieval results with human intent via activation steering. By manually clamping specific latent neurons, we can surgically intervene in the retrieval process to modify semantic priorities without retraining the backbone model.

Consider a complex query: “The updated safety guidelines applied only to heavy machinery operators.” We intervene by identifying and suppressing a specific semantic dimension:

• Original Retrieval: The baseline retrieval prioritizes the entangled context of safety and labor.

  • This was largely done in consideration of safety grounds and usually applied to those conducting maintenance or the repair of equipment.

  • To ensure safety during the Stack period, the organizers maintained a perimeter around the working area, and allowed only safety-trained students through.

• Steering Intervention: We identified neuron #4940 (Industrial Safety) via our auto-labeling pipeline and clamped its activation to zero (\(z_{4940} = 0\)).

• Steered Retrieval: Neutralizing the ‘safety’ component shifts results toward the regulatory scope; the original top result dropped to 9th place, while regulatory sentences ascended:

  • The final rule adopted several changes to the HOS regulations, including a new provision requiring drivers to take a rest break during the work day under certain circumstances.

  • The policy has been changed so permits are only required for large scale film, video and photography requiring 10 person crews .

These results demonstrate that Top-k SAEs effectively bridge dense vectors and human concepts. By modulating these semantic atoms, we align neural retrieval with specific intent, offering a principled framework for steerable, transparent search. See Appendix 10 for further case studies.

4 Conclusion and Future Work↩︎

In this work, we presented a framework for aligning the latent representations of dense retrievers with human conceptual structures utilizing Top-k Sparse Autoencoders. We demonstrated that applying the Top-k architecture allows for the successful decomposition of 1,024-dimensional embeddings into granular, monosemantic features, effectively bridging the gap between high-dimensional vector spaces and human interpretability.

Our automated analysis pipeline confirmed that these features correspond to specific entities, topics, and pragmatic functions. Crucially, we demonstrated that this disentanglement enables latent steering, which serves as a mechanism to intervene in the retrieval process and re-rank results according to user intent without model retraining.

Limitations. Our study focuses on a single backbone model (E5-large) and English benchmarks. Furthermore, we acknowledge that our hyperparameter selection is not exhaustive. While prior research indicates that scaling the latent dimension—often accompanied by increasing \(k\)—can improve performance up to the point of model collapse, we did not optimize for the ideal latent dimension or sparsity level. Since our primary objective was an exploratory investigation to verify if sentence embeddings could be decomposed into interpretable concepts and effectively steered, we maintained a fixed sparsity of \(k=32\). Consequently, the current performance may not represent the upper bound of the proposed framework.

Future Work. Our primary focus for future research is to integrate this steering mechanism into RAG pipelines to dynamically filter biased or unsafe concepts at inference time. We also plan to extend this methodology to multilingual settings to investigate cross-lingual conceptual alignment.

Disclosure of AI Usage↩︎

We acknowledge the use of Gemini-3 for linguistic polishing to improve the manuscript’s readability. Furthermore, GPT-4o-mini was employed within our research pipeline to perform automated feature annotation (Section 3.3). The authors maintain full responsibility for the content and ensure that all AI-generated outputs were critically evaluated and verified by the authors.

5 Qualitative Analysis of Learned Latent Features↩︎

In this section, we provide a detailed examination of the features learned by our Top-k Sparse Autoencoder. Table 3 presents a curated list of neurons, their human-interpreted labels, and representative activating sentences from the WikiText-103-v1 dataset. These examples demonstrate the model’s ability to disentangle diverse linguistic and semantic concepts, ranging from specific entities to abstract topics and structural patterns.

6 Detailed SAE Architecture and Training Objective↩︎

In this section, we provide the mathematical formulation of our Top-k Sparse Autoencoder and the training objectives employed.

6.1 Forward Pass Formulation↩︎

Let \(\mathbf{x} \in \mathbb{R}^{d_{model}}\) be the dense sentence embedding generated by the backbone model. The encoding process consists of a linear transformation followed by a Top-k non-linearity: \[\mathbf{z}_{\text{pre}} = \mathbf{W}_{\text{enc}}(\mathbf{x} - \mathbf{b}_{\text{dec}})\] \[\mathbf{z} = \text{TopK}(\text{ReLU}(\mathbf{z}_{\text{pre}}), k)\] where \(\mathbf{W}_{\text{enc}} \in \mathbb{R}^{d_{latent} \times d_{model}}\) is the encoder weight matrix, and \(\mathbf{b}_{\text{dec}}\) is the decoder bias. The \(\text{TopK}(\cdot, k)\) operation keeps only the \(k\) largest values of the pre-activations and zeroes out the rest, ensuring a fixed sparsity level.

The reconstruction \(\hat{\mathbf{x}}\) is obtained via the decoder: \[\hat{\mathbf{x}} = \mathbf{W}_{\text{dec}}\mathbf{z} + \mathbf{b}_{\text{dec}}\] We enforce tied weights such that \(\mathbf{W}_{\text{enc}} = \mathbf{W}_{\text{dec}}^T\), which acts as a regularization technique to improve training stability and feature coherence.

6.2 Training Objective with AuxK Loss↩︎

The primary objective is to minimize the reconstruction error, measured by the Mean Squared Error (MSE): \[\mathcal{L}_{\text{recon}} = ||\mathbf{x} - \hat{\mathbf{x}}||_2^2\]

A common challenge in training large-scale sparse autoencoders is the "dead neuron" problem, where certain latent features never activate across the entire dataset. To mitigate this, we incorporate an auxiliary loss term, \(\mathcal{L}_{\text{aux}}\), often referred to as the AuxK loss. This objective encourages inactive neurons to predict the residual error \((\mathbf{x} - \hat{\mathbf{x}})\) of the main autoencoder, effectively "reviving" them by forcing them to explain the information missed by the currently active features.

The total loss function is defined as: \[\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{recon}} + \alpha \mathcal{L}_{\text{aux}}\] where \(\alpha\) is a hyperparameter that controls the contribution of the auxiliary task. In our experiments, we set \(\alpha\) to a non-zero value (e.g., 0.1) to proactively prevent feature collapse.

7 Dataset Preprocessing Details↩︎

We constructed our training corpus using the WikiText-103-v1 dataset, which consists of over 100 million tokens extracted from verified “Good and Featured” articles on English Wikipedia. To adapt this document-level dataset for sentence embedding tasks, we performed the following preprocessing steps:

1. Sentence Segmentation: We split the raw articles into individual sentences using the NLTK sentence tokenizer [14], [15] as a standard natural language processing tool.

2. Filtering: To remove noise and ensure semantic completeness, we applied several filters:

   • We removed sentences shorter than 5 words to exclude section headers, list items, and fragmented text.

   • We excluded extremely long sentences (e.g., \(>128\) words) to prevent excessive truncation by the backbone model’s tokenizer.

   • We filtered out non-natural language artifacts, such as URLs and wiki-markup boilerplate.

After this process, the final dataset consists of 3,837,611 unique sentences, which serve as the inputs for training our Top-k Sparse Autoencoders.

8 Evaluation Metrics↩︎

Following the framework of [16], we employ quantitative metrics to assess the quality of learned sparse representations.

8.1 Explained Variance (EV) and FVU↩︎

To measure reconstruction quality, we report the Explained Variance (EV). This metric is directly related to the Fraction of Variance Unexplained (FVU), where \(\text{EV} = 1 - \text{FVU}\).

\[\text{EV} = 1 - \frac{\sum_{i=1}^{N} ||\mathbf{x}_i - \hat{\mathbf{x}}_i||_2^2}{\sum_{i=1}^{N} ||\mathbf{x}_i - \bar{\mathbf{x}}||_2^2}\]

A value close to 1.0 (or FVU close to 0.0) indicates that the sparse decomposition retains nearly all information present in the original dense embeddings.

8.2 Decoder Orthogonality (DO)↩︎

To evaluate the disentanglement of the learned features, we measure the orthogonality of the decoder weights. In an ideal disentangled representation, distinct latent features should correspond to distinct (orthogonal) directions in the semantic space.

Let \(\mathbf{w}_i\) denote the \(i\)-th column vector of the decoder matrix \(\mathbf{W}_{\text{dec}} \in \mathbb{R}^{d_{model} \times d_{latent}}\), representing the direction of the \(i\)-th latent feature. We first normalize all column vectors to unit length: \(\hat{\mathbf{w}}_i = \frac{\mathbf{w}_i}{||\mathbf{w}_i||_2}\). We then compute the mean absolute pairwise cosine similarity between all unique pairs of features:

\[\mathcal{M}_{\text{ortho}} = \frac{1}{d_{latent}(d_{latent}-1)} \sum_{i \neq j} |\langle \hat{\mathbf{w}}_i, \hat{\mathbf{w}}_j \rangle|\]

where \(\langle \cdot, \cdot \rangle\) denotes the dot product. A lower \(\mathcal{M}_{\text{ortho}}\) score indicates that the features are more independent and less redundant. A score of 0 implies perfect orthogonality among all feature directions.

9 Automated Labeling Prompt↩︎

To ensure transparency and reproducibility, we provide the full system and user prompts used for the automated feature annotation pipeline powered by GPT-4o-mini. The prompt was designed to enforce strict coherence checks and ensure specific, granular labels.

None

10 Additional Case Studies on Latent Feature Steering↩︎

To further validate the precision of semantic steering, we present additional case studies across various domains. Each case highlights how deactivating a single monosemantic neuron (identified via our automated pipeline) redirects the retrieval mechanism to prioritize alternative semantic facets of the query.

In the following tables, the values in brackets represent the cosine similarity scores between the query (original or steered) and the retrieved sentences.

10.1 Case Study 1: Neutralizing “Jumping Activities” (Neuron #2039)↩︎

The query “The record for the longest jump was broken during the Olympic trials” initially retrieves sentences focused exclusively on jumping events. By clamping neuron #2039, the focus shifts to the general concept of breaking records at Olympic trials across diverse sports.

10.2 Case Study 2: Neutralizing “Bridge Infrastructure” (Neuron #6188)↩︎

The query “The bridge construction was delayed due to an update in safety requirements” is suppressed by deactivating the bridge-specific context, shifting the results toward general construction delays and safety compliance.

10.3 Case Study 3: Neutralizing “Climate Change Awareness” (Neuron #9054)↩︎

Suppression of the “Climate Change Awareness” feature in the query “The government announced a major policy shift regarding carbon tax credits” shifts focus from environmental activism to general fiscal and administrative policy changes.

References↩︎