1 Introduction↩︎

Text retrieval involves searching for relevant information in vast text collections based on user queries. Efficient and effective methods for this task have revolutionized how we interact with information systems. Recently, there has been growing interest in augmenting large language models (LLMs) with text retrieval for question answering (QA) [1]–[6]. These approaches harness retrieval models to obtain external knowledge and ground LLM outputs, reducing hallucinations and the need for frequent LLM updates. Interestingly, augmenting an LM with a retrieval helps reduce the number of parameters required to achieve similar performance as larger LMs [7].

Text retrieval methods can be broadly categorized into two main approaches: sparse and dense retrievals. Sparse methods, such as BM25, exploit the frequency of words to measure the relevance between a passage and a query. While efficient, these methods often fall short of capturing intricate relationships and contextual nuances of language. In contrast, dense retrieval methods aim to learn meaningful representations from the semantic content of passages and queries effectively. These models can be trained based on a pretraining model (e.g. BERT, RoBERTa) as well as fine-tuned for downstream QA tasks [8], offering easy integration. In addition, it is possible to apply approximate nearest neighbors (ANN) with dense retrieval [9] for efficient retrieval.

This paper focuses on dense retrieval, where contrastive learning is often employed to train passage and query encoders. The core principle of contrastive learning is to encode passages and queries such that relevant passages are closer to their corresponding query in the embedding space, while irrelevant passages are farther away. To train such encoders, we need a labeled dataset with queries annotated with relevant passages (positive samples). However, due to the vast number of candidate passages and the complexity of questions, it is common for annotators to miss relevant information (texts) during data preparation, leading to unlabeled positive examples (false negatives) in the training set. Recent studies support this assumption. For instance, [10] found that over half of 50 answerable questions from the IIRC dataset [11] had at least one missing piece of evidence. Similarly, [12] manually reviewed top-retrieved passages not labeled as positives in MSMARCO [13] and detected a 70% false negative rate. On the other hand, it is essential to sample hard negatives for effective contrastive learning. Here, hard negatives refer to passages obtained from the top results of a pre-trained dense retrieval model or BM25. Unfortunately, hard negative sampling is susceptible to higher false negative rates in noisy datasets because such negative samples are more likely to be mislabeled ones. Therefore, mitigating the impact of false negatives can potentially improve the performance of dense retrieval.

Several strategies have recently emerged to address the problem of false negatives. [12] use a highly effective but inefficient reranker based on a cross-encoder to identify high-confidence negatives as true negatives, which were then used to train the retrieval model. [10] leverage answers in the downstream QA task and design several heuristics to detect valid contexts such as lexical overlapping (between a gold answer and a candidate passage). Recently, [14] suggests selecting samples that are highly similar to positive samples but not too close to the query. These samples are considered informative negatives and unlikely to be false negatives. Despite the recent progress, these current methods are primarily based on heuristics and lack a theoretical guarantee.

This paper formalizes the problem of training dense retrieval with false negatives into the peer loss framework [15], [16], a theoretical sound approach to learning with label noise. We extend this framework by developing a confidence regularizer for NCE, a commonly used loss for training dense retrieval models. Our regularized loss function increases the model’s confidence and proves to be robust against false negatives. By encouraging confident scoring, we prevent the model from overfitting to noise, resulting in a more robust retrieval model. We then propose a new passage sieve algorithm, that makes use of a confidence regularized retrieval model to select true hard negatives. The clean dataset after the passage sieve is then used to train a stronger retrieval model. We prove that our method can successfully filter out false negatives from hard negatives under mild assumptions. Through experiments on three datasets, we demonstrate that our method achieves better retrieval performance compared to existing state-of-the-art dense retrieval systems that rely on heuristic false negative filtering. Supplementary materials (the appendix, codes) can be found in our GitHub².

2 Related Works↩︎

2.1 Dense Retrieval↩︎

The dual-encoder (two-towers or biencoder) architecture [17], [18] is the common choice for dense retrieval thanks to its high efficiency. However, vanilla dual encoders have several challenges such as the limited expressiveness compared to cross-encoders, and the suboptimal performance due to non-informative negative samples. As a result, various solutions have been introduced to improve vanilla dual encoders from different perspectives such as knowledge distillation [19], [20], lightweight interaction models [21], [22], sophisticated training procedure [12], [19], [23], negative sampling strategies [9], [12], [14], [24]. In this paper, we focus mostly on negative sampling strategies, particularly targeting the false negative issue. Unlike the closely related works [10], [12], [14], which exploit heuristic strategies, our method leverages the peer-loss approach [15], effectively combining practical application with a theoretically-informed perspective.

2.1.0.1

2.2 Label-Noise Robust Machine Learning↩︎

Developing machine learning models that are robust against label noise is important for supervised learning. Existing methods tackle label noise based on the type of noise, such as random noise [25], [26], class-dependent noise [27]–[29], or instance-dependent noise [16], [30]–[33]. Unfortunately, these methods are mainly designed for multi-class classification, and thus cannot be directly applied to our task.

Also relevant to our work are [34], [35] which consider the issue of bias in contrastive learning for multi-class classification. These studies rely on the assumption of a fixed uniform noise probability across different classes (i.e. queries in our case) to approximate the positive and negative distributions. In contrast, our work concerns noise in supervised contrastive learning for query-dependent ranking, where the distributions of positives and negatives are not the same for different queries. For example, it is more likely for queries with high recall to be associated with false negatives. In other words, the noise probability is not uniform across queries.

3 Preliminaries↩︎

3.1 Problem Formalization↩︎

Let \(C\) be a collection of textual passages, and \(\tilde{\mathcal{D}} =\{(q_n,p_n^+,\tilde{\mathcal{N}}_{q_{n}})\}\) indicates the noisy set of tuples, each consists of a query \(q_n\), an annotated positive \(p_n^+\in C\), and a set of sampled negatives \(\mathcal{N}_{q_{n}}\subset C\) with possible false negatives. Our objective is to mitigate the impacts of such false negatives and learn a robust retrieval model that can closely match the model trained on the clean dataset \(\mathcal{D}=\{(q_n,p_n^+,\mathcal{N}_{q_{n}})\}\) without false negatives.

3.2 Dual-Encoders for Dense Retrieval↩︎

A question or a passage is encoded as dense vectors separately, and the similarity is determined as the dot product of the encoder outputs. \[sim(q, p) = <E_{qry}(q), E_{psg}(p)>\] where \(E_{qry}\), \(E_{psg}\) represent distinct encoders that map the query and the passage into dense vectors, and \(<.>\) is the similarity function such as dot product, cosine or Euclidean distance. The encoders are often built based on a pre-trained language model such as BERT-based [9], [24], [36]–[38], ERNIE-based [12], [19], [23] or RoBERTa-based [37] etc. Since passages and queries are encoded separately, passage embeddings can be precomputed and indexed using Faiss [39] for efficient search.

Contrastive learning is commonly used to train dual encoders [9], [12], [14], [24]. Typically, it is assumed that we have access to the clean training set \(\mathcal{D}=\{(q_n,p_n^+,\mathcal{N}_{q_{n}})\}\). Using this training set, we measure the NCE (Noise-Contrastive Estimation) loss for a given query \(q_n\) as follows: \[\begin{align} \ell_{NCE}(p_n^+,q_n)&=-\ln(f(p_n^+,q_n))\\ &=-\ln(\frac{e^{sim(p_n^+,q_n)}}{\sum_{p_i^- \in \mathcal{N}_{q_n}}e^{sim(p_i^-,q_n)}+e^{sim(p_n^+,q_n)}}) \label{eq:nce} \end{align}\tag{1}\] The total loss function can be calculated as follows: \[\frac{1}{N}\sum_{n\in [N]}\ell_{NCE}(p_n^+,q_n) \label{eq:nce95total}\tag{2}\] where \(N\) is the total number of queries in the dataset, and \([N]=\{1,2,\ldots, N\}\).

Negative sampling aims to select samples for \(\mathcal{N}_{q_n}\) and plays an important role in learning effective representations with contrastive learning. In the context of dense retrieval, two common strategies for negative sampling are in-batch negatives and hard negatives [9], [12], [24]. In-batch negatives involve selecting positive passages from other queries in the same batch as negative samples. Generally, increasing the number of in-batch negatives improves dense retrieval performance. On the other hand, hard negatives are usually informative samples that receive a high similarity score from another retrieval model (e.g., BM25, a pre-trained DPR) [24]. Hard negatives can result in more effective training for DPR, yet including such samples may exaggerate the issue of false negatives.

3.3 Peer Loss and Confidence Regularization↩︎

The problem of learning with label noise has been extensively researched in the context of classification tasks [15], [16], [30]–[33]. Recently, two effective methods called Peer Loss [15] and its inspired Confidence Regularizer [16], [30] have been introduced to train robust machine learning models. One advantage of these methods is that they can work without requiring knowledge of the noise transition matrix or the probability of labels being flipped between classes.

The concept of peer loss is initially introduced for the binary classification problem. In this setting, we use \(x\) to indicate an instance (a feature vector), \(y \in \{-1,1\}\) and \(\tilde{y}\in \{-1,1\}\) to represent the clean label and noisy labels, respectively. For each sample \((x_n,\tilde{y}_n)\), the peer loss is then defined for cross-entropy loss as follows: \[\ell_{PL}(g(x_n),\tilde{y}_n) = \ell_{CE}(g(x_n),\tilde{y}_n)-\ell_{CE}(g(x_{n1}),\tilde{y}_{n2}) \label{eqn:original95pl}\tag{3}\]

where \(g\) indicates the classification function, \(x_{n1}\) and \(\tilde{y}_{n2}\) are derived from different, randomly selected peer sample for \(n\). Here, the first term measures the loss of the classifier prediction, whereas the second term penalizes the model when it excessively agrees with the incorrect or noisy labels [15].

Inspired by peer loss, [16] develops \(\text{CORES}^2\), which extends the framework for multi-class classification and uses the first-order statistic instead of randomly selecting peer samples. Specifically, the new loss in \(\text{CORES}^2\) is defined as follows: \[\begin{align} &\ell(g(x_n),\tilde{y}_n) = \ell_{CE}(g(x_n),\tilde{y}_n)-\beta \mathbb{E}_{\tilde{D}_{\tilde{Y}}}[\ell_{CE}(g(x_n),\tilde{Y})] \end{align} \label{eqn:cores}\tag{4}\] where \(\beta\) is a hyper-parameter, \(\tilde{Y}\) denotes the random variable corresponding to the noisy label and \(\tilde{D}_{\tilde{Y}}\) indicates the marginal distribution of \(\tilde{Y}\). It has been shown that learning with an appropriate \(\beta\) will make the loss function robust to instance-dependent label noise with theoretical guarantee [16].

4 Contrastive Confidence Regularizer↩︎

We can adopt the above confidence regularization by introducing a binary label \(y\) associated with each pair of queries and passages, where \(y=+1\) indicates the positive pair and \(y=-1\) vice versa. Subsequently, \(\text{CORES}^2\) can be used with pairwise cross-entropy to mitigate the impacts of false negatives on training the retrieval model. Unfortunately, the cross-entropy loss is not as effective as the NCE loss (Eq. 2 ) for dense retrieval since the latter can learn a good ranking function by contrasting a positive sample over a list of negatives [24]. As a result, we aim to extend the framework of peer loss and tailor the confidence regularization to the NCE loss.

To adopt the peer-loss framework, we extend NCE loss to measure loss values associated with negative pairs \((p_i^-,q_n)\) besides positive ones. It is noteworthy that the original NCE loss (Eq. 2 ) incorporates all positive and negative pairs as normalization factors but only calculates loss values for positive pairs \((p^+_n, q_n)\) while disregarding the negative pairs. Formally, we use \(p\) without superscript as a generic term for positive passage \(p_n^+\) and negative passage \(p_i^- \in \mathcal{N}_{q_n}\), and define a general NCE loss as follows: \[\begin{align} \ell_{NCE}(p,q_n)&=-\ln(f(p,q_n))\\ &=-\ln(\frac{e^{sim(p,q_n)}}{\sum_{p_i^- \in \mathcal{N}_{q_n}}e^{sim(p_i^-,q_n)}+e^{sim(p_n^+,q_n)}}) \end{align}\] where p could be either positive or negative passages for query \(q_n\). By this convention, the peer-loss framework can be applied by introducing randomly selected pairs \((p_{n_1}, q_{n_1})\), \((p_{n_2}, q_{n_2})\) (\({n_1}\neq {n_2}\)) as peer samples to regularize the NCE loss. We then obtain the contrastive peer loss as follows: \[\ell_{PL}(p_n^+,q_n) = \ell_{NCE}(p_n^+,q_n)-\ell_{NCE}(p_{n_1},q_{n_2}) \label{eqn:contrastive95pl}\tag{5}\]

While our regularized loss appears similar to the original peer-loss outlined in Eq. 3 , the major difference lies in the structure of the loss functions. Specifically, the first term of Eq. 5 is calculated only for positive pairs whereas the first term of Eq. 3 involves both positive (\(\tilde{y}=+1\)) and negative (\(\tilde{y}=-1\)) samples.

Similar to \(\text{CORES}^2\), we introduce \(P_{n_1}\) and \(Q_{n_2}\) as the random variables corresponding to the peer passage \(p_{n_1}\) and the peer query \(q_{n_2}\), respectively. Let \(\tilde{\mathcal{D}}_P\) be the distribution of \(P_{n_1}\) given distribution \(\tilde{\mathcal{D}}\). Note that \(Q_{n_2}\) is a uniform random variable, or \(\mathbb{P}(Q_{n_2}=q_{n'}|\mathcal{\tilde{D}})=1/N\), the contrastive peer loss has the following form in expectation: \[\begin{align} &\frac{1}{N}\sum_{n\in[N]}\mathbb{E}_{\tilde{\mathcal{D}}_{P_{n_1},Q_{n_2}}}\left[\ell_{NCE}(p_n^+,q_n)-\ell_{NCE}(P_{n_1},Q_{n_2})\right]\\ =&\frac{1}{N}\sum_{n\in[N]}[\ell_{NCE}(p_n^+,q_n)-\\ &\sum_{n' \in[N]} \mathbb{P} (Q_{n_2}=q_{n'}|\tilde{\mathcal{D}})\mathbb{E}_{\tilde{\mathcal{D}}_{P}}\left[\ell_{NCE}(P_{n_1},q_{n'})\right]\\ = &\frac{1}{N}\sum_{n\in[N]}\left[\ell_{NCE}(p_n^+,q_{n})-\mathbb{E}_{\tilde{\mathcal{D}}_{P}}\left[\ell_{NCE}(P,q_{n})\right]\right]\\ \approx &\frac{1}{N}\sum_{n\in[N]}\left[\ell_{NCE}(p_n^+,q_{n})-\mathbb{E}_{\tilde{\mathcal{D}}_{P|q_{n}}}\left[\ell_{NCE}(P,q_{n})\right]\right] \end{align}\] The last approximation equation is obtained because when considering the batch training in dense retrieval, \(P_{n_1}\) is drawn from in-batch passages. In addition, all the passages in batch form the conditional passage distribution of query \(q_n\). From this derivation, we can get the new noise robust contrastive loss function (i.e. \(\ell_{RCL}\)) with contrastive confidence regularizer denoted by \(\ell_{CCR}\) as follows: \[\begin{align} \ell_{CCR}(p_n^+,q_n) &= \mathbb{E}_{\tilde{\mathcal{D}}_{P|q_{n}}}[\ell_{NCE}(p,q_{n})]\\ \ell_{RCL}(p_n^+,q_n) &=\ell_{NCE}(p_n^+,q_n)-\beta*\ell_{CCR}(p_n^+,q_n) \label{l95RCL} \end{align}\tag{6}\]

In the following, we first empirically prove that our regularized loss function makes the retrieval model more confident, hence being more robust against false negatives. We then present the main theories that guarantee the robustness of our regularized NCE loss.

4.1 Analysis on Simulated Data↩︎

The RCL loss (eq. 6 ) is minimized by making the first term smaller and the expectation term bigger. In other words, we pull the loss associated with positive passages further from the average (the expectation) loss, subsequently making the model more confident in predicting positive passages. Intuitively, as label noise distorts the learning signal in clean data, the model trained on noisy datasets often fits noises, hence becoming less confident [16]. By making the model more confident, we can partially reverse it and obtain a more robust retrieval model. We verify this intuition by conducting a simulation on the Natural Question (NQ) dataset [40]. Specifically, we randomly convert some positive passages to false negatives and observe the effects of contrastive confidence regularizer \(\ell_{CCR}\) on the distributions of the similarity scores in different groups of passages including false negatives, hard negatives and in-batch negatives. The experimental results in Figure 1 indicate that if we do not include the \(\ell_{CCR}\) term (i.e., \(\beta=0\)), the distributions are close to each other. On the other hand, incorporating the \(\ell_{CCR}\) term makes it easier to separate these distributions. It should be noted that a small overlapping between the false negative and hard negative distributions is expected. This is because while all samples in the simulated “false negatives” category are certainly positive passages, there exist (unknown) false negatives in the “hard negatives” category.

4.2 Theoretical Analysis↩︎

Theoretical analysis shows that our RCL loss enjoys similar properties with \(CORES^2\). The detailed proof can be found in the Appendix, where the main idea is that we introduce pseudo-labels to bridge NCE loss and cross-entropy loss and follow a similar proof sketch with [16]. The main result is summarized in the following theorem.

Theorem 1 indicates that the contrastive confidence regularizer effectively addresses the adverse effect of false negatives on the loss function. Specifically, we can decompose the regularized loss so that the impact of false negatives is mostly captured by the product of \(\ell_{CCR}\) and a scaling factor (the last term in Eq.@eq:eq:e95rcl , Appendix). Here, the scaling factor is intuitively related to the difference between the relative clean rate of a sample (compared to the average clean rate) and \(\beta\). When \(\beta\) is large enough, this scaling factor is negative, thus reversing the (term) gradient associated with instances with labeled noise. Note that \(\beta\) should not be too large, otherwise it will affect learning from clean data. This raises a crucial question regarding the existence of parameter \(\beta\). Analyzing the existence of \(\beta\) becomes challenging when the nature of the noise is uncertain. Fortunately, in the context of false negatives in dense passage retrieval, we can derive the following theorem.

Theorem 2 shows that under the setting of dense passage retrieval with false negatives, one can select a suitable \(\beta\) between the interval [0,1]. However, this theorem only applies when the similarity function is bounded and there is no other source of noise besides false negatives. In practical scenarios, many dense passage retrieval algorithms employ dot-product similarity instead of cosine similarity. When the loss function is not bounded, and there is an excessive number of negative passages, the confidence regularizer can cause the model to overly optimize the loss associated with negative cases. Our experiments indicate that in such situations, the contrastive confidence regularizer still helps as long as a sufficiently small value for \(\beta\) is selected. In general, as the batch size increases and in-batch negative sampling is utilized, we should decrease the value of \(\beta\).

5 Passage Sieve Method↩︎

The previously proposed regularization is helpful towards dense retrieval models based on contrastive NCE loss, but not applicable to sophisticated methods such as AR2 [23], which do not make use of NCE loss. As a result, we design a novel passage sieve algorithm, which can be used as pre-processing for the noisy datasets \(\tilde{\mathcal{D}}\) to obtain relatively clean datasets for training.

Our method is presented in Algorithm 2. We first train DPR [24], a dual-encoder based retrieval model on the noisy dataset. We then refine the retrieval model using the contrastive confidence regularizer. Afterward, we identify the hard negative passages with a loss function value \(\ell_{NCE}(p,q_n)\) higher than the average loss value \(\frac{1}{|N_{q_n}|+1}\sum_{p \in N_{q_n} \cup \{p_n^+\} }\ell_{NCE}(p,q_n)\), and set them as confident negatives. Finally, we discard all other passages except for the confident negatives to obtain a “clean” dataset. It is noteworthy that we employ cosine similarity to satisfy the assumptions outlined in Theorem 2.

Lemma 1 provides the sufficient condition for accurately selecting true negatives in Algorithm 2. In this algorithm, the DPR model is trained with robust contrastive loss and cosine similarity. According to Theorems 1 and 2, by selecting a suitable value of \(\beta \in [0,1]\), minimizing \(\mathbb{E}_{\tilde{D}}[\ell(p,q)+\ell_{CCR}(p,q)]\) is equivalent to minimizing \(\mathbb{E}_\mathcal{D}[\ell(p,q)]\). This implies that false negatives are gradually given scores closer to the positive passage, and higher compared to hard negatives and in-batch negatives. Considering the abundance of true negatives, the scores assigned to false negatives will exceed random guesses at some point. Consequently, false negatives will be excluded from the sieved dataset in Algorithm 2, resulting in a clean dataset \(\tilde{D}\).

Compared to vanilla DPR, DPR with CCR introduces additional computation associated with the expectation calculation, the second term in Eq. 6 . Fortunately, this can be approximated by taking the mean value of the NCE losses of all passages given the query. In addition, the calculation of NCE losses is dominated by the calculation of the similarity matrix (Eq. 1 ), which is also needed in vanilla DPR. Consequently, the contrastive confidence regularizer only slightly increases the time complexity when implemented appropriately. It is worth noting that when using retrieval models like DPR, ANCE, or RocketQA that employ NCE loss, using the robust contrastive loss in section 4 should be sufficient.

6 Experiment↩︎

6.1 Experimental Setup↩︎

6.1.0.1 Datasets

We conduct experiments on three public QA datasets: Natural Question (NQ) [40], Trivia QA (TQ) [49] and MSMARCO Passage Ranking (MS-pas) [13]. The detailed statistics are given in the supplementary material.

6.1.0.2 Evaluation Metrics

Following previous works, we report R@k (k=5, 20, 100) for NQ and TQ, and MRR@10, R@k (k=50, 1K) for MS-pas. Here, MRR refers to the Mean Reciprocal Rank that calculates the reciprocal rank where the first relevant passage is achieved, and R@k measures the proportion of relevant passages (recall) in top-k results.

6.2 Effects of Passage Sieve Method↩︎

6.2.1 Experimental Design↩︎

We implement our passage sieve method on top of AR2 [23], which is referred to as AR2+passage sieve. To evaluate the effectiveness of our passage sieve method, we compare it with the original AR2 along with other contemporary baselines. The details on the baselines are given in the supplementary material.

During the passage sieve procedure in AR2+passage sieve, we set \(\beta=0.5\) and epochs \(T=1\), learning rate \(\alpha=1e-7\). Additionally, we leverage the cosine similarity function and set the initial number of hard negative passages to be two times the number of hard negatives to be used in the downstream AR2 model. As for AR2 training, all settings are set the same way as in [23]. Specifically, the batch size is set to 64 and the number of hard negatives is 15.

6.2.2 Overall Results↩︎

Table 1 presents the results of AR2+passage sieve on NQ and TQ test sets, as well as MS-pas development set. It can be observed that AR2 outperforms most of the baseline models on all three datasets across different evaluation metrics. Furthermore, the proposed passage sieve approach enhances the performance of AR2 on all three datasets and evaluation metrics. The findings suggest that the sieve algorithm significantly contributes to improving the quality of hard negative samples in the dataset, enabling better results even with less GPU memory requirement. It is worth mentioning that our passage sieve procedure only accounts for one-tenth of the training time compared to AR2, the downstream retrieval model.

SimANS [14] is the recently proposed method to address the issue of false negatives based on some useful heuristic assumptions. When combined with AR2, AR2+SimANS achieves state-of-the-art results in dense retrieval. In contrast, we start from theory in noise-robust machine learning algorithms and develop a new loss function. The results in Table 2 show that both SimANS and our method enhance the performance of AR2. However, compared to AR2+SimANS, the passage sieve achieves better results in all terms of TQ and NQ datasets. The advantage is more clearly seen on NQ dataset. It should be noted that we do not include the result on the MS-pas dataset of SimANS because they leverage a 4 times larger batch size than AR2 which makes the comparison unfair.

6.3 Detailed Analysis↩︎

6.3.0.1 Influence of limited batch size

We investigate the impact of smaller batch size on the performance of AR2+passage sieve. Table 2 shows that AR2+passage sieve achieves better results in terms of R@5, R@20, and R@100 on TQ dataset, even with a smaller batch size. On NQ dataset, our method achieves comparable results with AR2+SimANS while having only a slight decrease in R@20.

6.3.0.2 Influence of the number of hard negatives

To evaluate the impact of hard negatives, we experiment with varying numbers of hard negatives during training. To speed up the experiment, the batch size is set to 32 and we only work on NQ dataset. The results in Table 3 demonstrate that the passage sieve approach consistently enhances the performance of AR2 across all settings. Particularly, with only 5 hard negatives, our method can reach a performance near that of 15 hard negatives. Intuitively, when the quality of hard negative samples is higher, we do not need many negative samples, thus reducing computing resources for training.

6.3.0.3 Influence of hyper-parameter \(\beta\)

To study the influence of the hyper-parameter \(\beta\) on the passage sieve algorithm, we conduct experiments on NQ dataset with different \(\beta\) and all other settings remain the same. Specifically, we vary the values of \(\beta\) to be 0, 0.25, 0.5. Figure 3 indicates the passage sieve method is not sensitive to \(\beta\). This is because DPR in the passage sieve method is trained with cosine similarity, which satisfies our assumptions in Theorem 2.

6.4 Effects of Contrastive Confidence Regularizer↩︎

6.4.1 Exterimental Design↩︎

We evaluate the performance of the Contrastive Confidence Regularizer on the DPR model [24]. We incorporate the regularizer into the existing NCE loss function during the final 5 epochs of training. Other parameters and configurations are kept consistent with those in the original DPR paper. Since the DPR model utilizes a dot-product similarity function, we select the values of \(\beta\) from the range of 0.001 to 0.0001. The selection of \(\beta\) is based on the performance observed on the validation set.

6.4.2 Experiment Results↩︎

Table 4 shows results on NQ and TQ datasets. It is observable that the proposed contrastive confidence regularizer helps DPR get a better performance across all metrics on NQ and TQ datasets. The experimental results also show that the reduction of batch size has a greater impact on R@20 and R@5, but a smaller impact on R@100. On both datasets, the proposed CCR helps achieve the same R@100 value as DPR with only half the batch size.

7 Conclusion↩︎

This paper aims to mitigate the impact of false negatives on dense passage retrieval. Toward such a goal, we extend the peer-loss framework and develop a confidence regularization for training robust retrieval models. The proposed regularization is compatible with any base retrieval model that uses NCE loss, a widely used contrastive loss function in dense retrieval. Through empirical and theoretical analysis, it is demonstrated that contrastive confidence regularization leads to more robust retrieval models. Building on this regularization, a passage sieve algorithm is proposed. The algorithm leverages a dense retrieval model trained with confidence regularized NCE loss to filter out false negatives, thereby improving any downstream retrieval model including those that do not exploit NCE loss. The effectiveness of both the passage sieve algorithm and the confidence regularization method is validated through extensive experiments on three commonly used QA datasets. The results show that these methods can enhance base retrieval models, even when fewer negative samples are used.

Acknowledgments↩︎

We thank all the reviewers for their helpful comments. This work is supported by the National Key R&D Program of China (2022ZD0116600).

Appendix↩︎

8 Missing Details↩︎

8.1 Limitations and future work↩︎

This paper proposed a novel contrastive confidence regularizer and a passage sieve algorithm to handle the false negative problem in dense retrieval training. The theory analysis and experimental results have shown the effectiveness of our methods, however, there are still several limitations of our work:

1. We only test our methods on top of AR2 [23] and DPR [24], more retrieval models can be tested to see the performance variance of the passage sieve and contrastive confidence regularizer in different cases.

2. Theorem 1 only assumes the loss function to be NCE loss and does not require the task to be dense retrieval. Thus, the proposed contrastive confidence regularizer has the same power on any other self-supervised tasks using NCE loss, e.g. self-supervised image classification, sentence embedding, etc. More experiments on these benchmarks should be conducted in future work.

3. The proposed methods are built upon \(\text{CORES}^2\) [16], which consider the first-order statistic(estimation) of peer loss. However, [30] has extended this into considering the second-order statistics and achieving better performance in CV benchmarks. More work is needed to extend the contrastive confidence regularizer into second-order statistics as well.

8.2 Details about baselines↩︎

Details about the baselines in Table 1 in the main content are shown in this subsection.
DPR [24] employs a dual-encoder architecture, leveraging BM25 for the identification of hard negatives and utilizing the in-batch negatives for efficiency.
ANCE [9] constructs hard negatives from an Approximate Nearest Neighbor (ANN) index.
COIL [41] integrates semantic with lexical matching, storing contextualized token representations in inverted lists.
ME-BERT [36] merges the efficiency of dual encoders with the expressiveness of attentional architectures, further venturing into sparse-dense hybrids.
Individual top-k [42] utilizes unsupervised pre-training, focusing on the inverse cloze task and masked salient spans within question-context pairs.
RocketQA [12] explores the ability of the cross-encoder model to denoise hard negatives.
RDR [43] seeks to distill knowledge from the reader into the retriever, aiming for the retriever to gain reader strengths without loss of inherent advantages.
RocketQAv2 [19] distills knowledge from the cross-encoder model by narrowing the relevance distribution between the retrieval model and the cross-encoder model.
PAIR [44] considers both query-centric similarity relations and passage-centric similarity relations.
DPR-PAQ [37] introduces the PAQ dataset, concentrating on the question-answering pretraining task.
Condenser [45] boosts optimization readiness by performing mask language model predictions actively condition on dense representation.
coCondenser [46] adds a corpus-level contrastive learning goal to the existing Condenser pre-training architecture.
ERNIE-Search [20] refines the distillation process by sharing the encoding layers with the dual-encoder and conducting a cascaded approach.
MVR [38] offers multiple representations for candidate passages and proposes a global-local loss to prevent multiple embeddings from collapsing to the same.
PROD [47] enhances the cross-encoder distillation by cyclically improving teacher models and tailoring sample selections to student needs.
COT-MAE [48] utilizes an asymmetric encoder-decoder architecture with self-supervised and context-supervised masked auto-encoding tasks.
AR2 [23] jointly optimizes the retrieval and re-ranker model according to a minimax adversarial objective.

8.3 Example implementation of contrastive confidence regularizer↩︎

In section 5, we mentioned that CCR (eq.8 in the main content) only slightly increases the time complexity when implemented properly. In this subsection, we will demonstrate the implementation to prove this claim.

The example code is shown in listing [lst:python]. The dimension of \(s\) is the number of queries in the batch times the number of passages (including positive, in-batch negatives, and hard negatives) of each query. In order to calculate the NCE loss, a log softmax function on the passage dimension is required. And after that, an NLL loss with only positive passages labeled as 1 is calculated with the after-softmax matrix \(s\_\). The only additional code for CCR is to calculate the NLL loss for all elements in the \(s\_\) and get the mean value of each row of it.

[language=Python, caption= NCE loss with Contrastive Confidence Regularizer, label=lst:python, frame=single, escapeinside={(*@}{@*)}]
s = get_scores(q, p) #get the similarity matrix
s_ = F.log_softmax(s, dim=1)#calculate softmax on passage dimension
loss = F.nll_loss(s_,positive_idx,reduction="none") #the original NCE loss
loss=loss-beta*(-1)*s_.mean(dim=1)#the only additional code to calculate CCR
loss=loss.mean()

8.4 Efficiency analysis of passage sieve algorithm↩︎

Overall speaking, The time taken to integrate the passage sieve (algorithm 1) into AR2 is equivalent to that of training a vanilla DPR[24] model. Compared with the time consumption of AR2 training itself, the additional training time is relatively small. We record the time consumption of one epoch training on the TQ dataset and show it in table 5.

It should be noted that the passage sieve has nothing to do with the inference time consumption. Adding the passage sieve only requires an additional data preprocessing time during training.

8.5 Intermediate experiment result↩︎

8.5.0.1 Sieve out rate of passage sieve

In the passage sieve algorithm, we perform passage sieves according to their loss value and the average loss value (i.e. threshold). And we prove that those who get a score \(f(p,q)\) more than the average value will be sieved out for sure, however, it remains unknown what percentage of all negative passages will be sieved out (we called it sieve out rate). The sieve out rate directly affects how many passages we need to examine during the passage sieve in order to meet the hard negatives requirement of the downstream retrieval model.

In the main content, we mentioned that we choose two times the number of hard negatives that the downstream retrieval model (AR2 in this case) will use. This ratio is actually chosen by some preliminary experiments, in which we observed that the passage sieve algorithm usually sieves out examples less than half. We show the sieve-out rate during training on the NQ dataset in figure 4. The figure shows that the sieve rate is usually smaller than but close to 0.5 and usually when one epoch achieves a sieve-out rate near 0.5, the rate of the next epoch will drop immediately. It should be noted that this rate has nothing to do with the estimated noise pair possibility. The strategy adopted by the passage sieve algorithm is to filter aggressively and only retain confident ones, in which only the sufficient conditions for filtering are specified (see Lemma 1).

8.5.0.2 Learning curves

We also show the visualization of the training procedure. Specifically, the Recall@1 Recall@5 of the TQ test set and Recall@1 MRR@10 of the MS-Pas dataset are shown in the figure. And we can see from fig.5 that the metrics on both TQ and MS-pas datasets gradually increase until convergence.

8.6 Annotation Table↩︎

We conclude the annotation we used in the main content of the paper and show them in table 6 to help read our paper.

9 Important Proofs↩︎

Before we start the proof, we will first introduce the pseudo-label and corresponding loss function we used during the proof. Let’s consider a positive pair \((p^+,q)\) as label \(y=1\) and negative pair \((p^-,q)\) as label \(y=-1\). Remind that \(p\) means passages and \(q\) stands for queries. In the following context, instead of using superscript, we will leverage this label to indicate the relation between passages and queries. And similar to the context in the label-noise robust machine learning papers, \(y\) represents the real label and \(\tilde{y}\) for the noisy label in the dataset. Thus we can represent the false negatives in the dataset as \((p,q;\tilde{y}=-1,y=1)\). We then re-define the contrastive loss function as : \[\ell(f(p,q);y)= \begin{cases} \ell_{NCE}(p,q), & y=1,\\ 0, & y=-1, \end{cases} \label{eq:pseudo-loss}\tag{7}\] where \[\begin{align} \ell_{NCE}(p,q)&=-\ln(f(p,q))\\ &=-\ln(\frac{e^{sim(p,q)}}{\sum_{p_i^- \in \mathcal{N}_{q}}e^{sim(p_i^-,q)}+e^{sim(p^+,q)}}) \end{align}\] Here, \(f\) is the model that gives the score between queries and passages. \(\ell_{NCE}\) is the broadened NCE loss as we discussed in the preliminary part. This newly defined pseudo loss is for proof purposes only which helps better analyze the performance of the proposed methods during proof. Pay attention that such a definition will not affect the original contrastive learning in dense retrieval as the loss for negative pairs is always equal to 0. And there is only a coefficient difference between them: \[\scalebox{0.9}{ \mathbb{E}_{(P,Q;Y)\in D}\left[\ell(f(P,Q);Y)\right] = \mathbb{P}(Y=1) \mathbb{E}_{(P^+,Q)\in D}\left[\ell_{NCE}(P,Q)\right] }\] In the following context, unless otherwise specified, \(\ell\) stands for this defined loss, and \(\ell_{NCE}\) stands for the original NCE loss.

9.1 Proof for theorem 1 and theorem 2↩︎

To make the reading more natural and smooth, we simultaneously prove Theorems 1 and 2 in this section.
Theorem 1. With the assumption that the possibility of a noisy pair is smaller than a clean pair and a suitable selection of \(\beta\), we have: minimizing \(\mathbb{E}_{\tilde{\mathcal{D}}}[\ell(p^+,q)-\beta*\ell_{CCR}(p^+,q)]\) is equivalent to minimizing \(\mathbb{E}_\mathcal{D}[\ell(p^+,q)]\)
Theorem 2. When the assumption in Theorem 1 holds, the similarity function \(sim\) is bounded (e.g. cosine similarity) and there is no false positive in the dataset \(\tilde{\mathcal{D}}\), the \(\beta\) that satisfies Theorem 1 must exist and in the interval [0,1].

9.2 Proof for Lemma 1↩︎

lemma 1. The passage sieve algorithm (1) ensures that a negative sample \(p^-\) in hard negatives will NOT be selected into the sieved dataset \(\mathcal{D}^*\) if its score \(f(p^-,q)\) given by the model f is more than a random guess, i.e. its similarity score after softmax is bigger than the average value \(1/(|\mathcal{N}_k|+1)\)

References↩︎