1 Introduction↩︎

Open-domain dialogue agents have received increasing attention in recent years [1], [2]. In an engaging open-domain dialogue, a large amount of knowledge, such as commonsense [3] and factual knowledge [4], is involved. To integrate knowledge into dialogue agents, KB-based methods have been proposed to explicitly acquire knowledge from knowledge bases [3], [4]. However, KB-based methods suffer from problems of retrieval error [5] and inefficiency [6]. Meanwhile, recent years have witnessed a rapid development of pre-trained language models (LMs) [7], [8] and their applications to dialogue tasks [9]. Large LMs implicitly store knowledge in their parameters during the pretraining stage [10], [11] and thus, to some extent, they can serve as knowledge bases to ground open-domain dialogues [12]. Such approaches, known as LM-based methods, achieve promising performance in generating informative responses and obviate the drawbacks of KB-based methods. However, LM-based methods have the problem of “hallucination” [13], [14]: they generate plausible-looking statements that are irrelevant or factually incorrect.

To understand the severity of hallucinations of LMs, we conduct a pilot experiment. We sample 200 responses generated by BART [15] on the Wizard-of-Wikipedia dataset [4] for various topics and conversation turns. These responses are annotated by three well-informed experts in terms of knowledge relevancy and factuality. Based on the results, we group the hallucinations of LMs into two types: intrinsic hallucinations and extrinsic hallucinations. Intrinsic hallucinations are non-factual statements, such as incorrectly predicting a celebrity’s birthday. Extrinsic hallucinations are irrelevant or out-of-context responses, such as the a description of the history of football when the user asks the number of teams currently in the NFL. Fig. 1 summarizes the outcomes: intrinsic and extrinsic hallucinations account for 24% and 27% of the responses, respectively.

The problem of hallucinations is mainly attributable to the optimization recipes: the commonly used maximum likelihood estimation (MLE) with teacher forcing training encourages the model to imitate the training data blindly, leading to model hallucinations at inference time [16]. Most studies on tackling hallucination in conversations focus on KB-based methods and use pre-retrieval [13] or post-editing techniques [17] to improve faithfulness; the hallucination of LM-based agents in eliciting knowledge inside LMs’ parameters is still underexplored.

In this paper, we propose Mixed Contrastive Learning(MixCL) to alleviate the hallucinations of LM-based dialogue agents. MixCL explicitly samples the most confusing knowledge to the model and reduces its generation probability by contrasting it with the ground-truth. To this end, two novel steps are used by MixCL:

In the former, we sample the most confused negative knowledge by retrieving from the corpus or deriving via model bootstrapping. In the latter, we propose mixed-contrastive learning under the inspiration of mix-up data augmentation [18], which mixes the positive and negative at span level. Moreover, we propose two mixed strategies regarding the two types of hallucination: entity-based mix-up and constituency-based mix-up. Finally, MixCL is optimized in an end-to-end manner, thus avoiding the retrieval step during inference and instead using the knowledge inside its parameters.

We conduct experiments on Wizard-of-Wikipedia [4], an open-domain, knowledge-grounded dialogue dataset. Extensive experiments show that MixCL improves the informativeness and relevancy of the responses. Compared with previous LM-based methods [5], [6], [12], MixCL achieves improvements by 5% to 15% in terms of response quality and relevancy. Moreover, MixCL achieves comparable performance as state-of-the-art KB-based methods (e.g., KnowledGPT [19]), while speeding up 5\(\times\) in model inference and showing superior scalability. The effectiveness of MixCL is also verified through human evaluation and ablation experiments.

Our contributions are as follows:

2 Related work↩︎

2.1 Knowledge-grounded dialogues↩︎

In open-domain knowledge-grounded dialogues (KGDs), people respond to each other’s utterances in a meaningful way by integrating knowledge [2], [3]. To integrate knowledge, KB-based methods have been explored [3], [4], [20]; they retrieve knowledge from a corpus through additional information retrieval (IR) modules. Studies on KB-based methods focus on knowledge selection [13], [21] and knowledge-grounded response generation [19], [22]. However, KB-based methods suffer from the problems of retrieval errors [5], inefficiencies [6], and multi-granularity knowledge integration [23].

2.2 Language models as knowledge bases↩︎

Recent years have witnessed a rapid development of language models (LMs) [8] and LM-based dialogue agents [9]. Large LMs store knowledge into their parameters during pre-training and can generate informative responses in conversations [12]. [10] show that LMs can serve as knowledge bases for downstream tasks (e.g., question answering [24]). On this basis, [12] show that LMs can ground open-domain dialogues using their implicit knowledge. [25] embed knowledge bases into model’s parameters for end-to-end task-oriented dialogues. [26] finetune LMs on KGD data. [27] propose knowledge-enhanced finetuning methods to handle unseen entities. [6] propose a topic-aware adapter to adapt LMs in KGDs. [5] propose a multi-stage prompting approach for triggering knowledge in LMs. [23] propose lexical knowledge internalization to integrate token-level knowledge into the model’s parameters. However, existing LM-based methods suffer from the problem of hallucination. In this paper, we optimize the implicit knowledge eliciting process, i.e., reduce hallucination of LMs in KGD, via the proposed contrastive learning framework MixCL.

2.3 Contrastive learning↩︎

Contrastive learning (CL) [28], [29] is based on the idea that similar samples should also be close in representation space, and has seen applications in NLP [30]. CL has been used for optimizing knowledge retrieval processes [31], [32], where the model learns to identify positive knowledge from negatives. On the task of neural text generation, CL [33], a.k.a.unlikelihood training [34] or negative training [35], alleviates undesirable properties of the generated output, e.g., repetition [33], [36], maliciousness [35], dullness [37], [38], or inconsistency [39]. Moreover, [40] propose a sentence level contrastive learning method to reduce the hallucinations of text summarization model. Unlike existing studies, we propose a mixed contrastive learning framework MixCL that eliminates the hallucination at the span level with effective negative sampling strategies.

3 Problem formulation↩︎

Let \(x\), \(y\), and \(k\) be the dialogue context, the corresponding response, and the ground-truth knowledge, respectively. As illustrated in Fig. 2, given a knowledge corpus \(\mathcal{K}\), a dialogue agent learns to predict an informative response \(y\) based on the dialogue context \(x\) using the knowledge in \(\mathcal{K}\). As discussed earlier, two approaches are studied in KGD, KB-based methods and LM-based methods. In this paper, we focus on the latter one.

3.0.0.1 KB-based methods.

KB-based dialogue agents [4] ground the response generation by explicitly retrieving knowledge from \(\mathcal{K}\). Two sub-modules, i.e., knowledge retriever and response generator, are employed by KB-based approaches, as shown in Fig. 2 (a).

3.0.0.2 LM-based methods.

In this paper, we explore language models as knowledge bases for dialogue agents [6], [12], as illustrated in Fig. 2 (b). In LM-based approaches, the LMs are first pre-trained on \(\mathcal{K}\) to store the knowledge in their parameters. Then, the models directly generate \(y\) given \(x\) using the knowledge in their parameters and getting rid of the explicit retrieval step.

4 Preliminaries↩︎

We propose a LM-based dialogue agent for open-domain KGD. The proposed model \(p_\theta(y|x)\) is based on a transformer-based language model with encoder-decoder architecture. The model is first pre-trained on the corpus \(\mathcal{K}\) and then finetuned on dialogue data to generate informative responses.

4.0.0.1 Pre-training on knowledge corpus.

We employ BART [15] as the pre-trained transformer, which is pre-trained by denoising self-supervised learning: \[\label{eq:lm} \mathcal{L}_{\text{LM}} = -\mathbb{E}_{k \sim \mathcal{K}}~\log p_\theta(k|\hat{k}),\tag{1}\] where \(\mathcal{K}\) is a text-based knowledge corpus (e.g., Wikipedia), \(k\) is a text sampled from knowledge corpus \(\mathcal{K}\), and denotes corrupted text by corruption functions (e.g., masking, deletion, infilling, etc.; [15]).

4.0.0.2 Finetuning on dialogue datasets.

With the pre-trained LM, the model generates the response \(y\) given \(x\) without explicit knowledge retrieval step [6], [12]. Maximum likelihood estimation (MLE) training loss on dialogue data with paired \((x, y)\) is employed by previous methods. In MLE, the model learns to predict the ground-truth tokens for each step in a teacher forcing paradigm [6], [12]: \[\label{eq:mle} \mathcal{L}_{\text{MLE}} = -\log p_\theta(y|x) = -\sum_{t=1}^{|y|} \log p_\theta(y_t|y_{<t},x).\tag{2}\] However, despite its effectiveness in generating informative responses, MLE loss encourages the model to imitate the training data blindly and leads to model hallucination [16]. Studies have found that models trained with standard MLE may over-rely on previously predicted tokens, exacerbating error propagation [41]. As a result, during the inference stage, as the generated sequence grows, the errors accumulate along the sequence, and the model tends to amplify errors and generate hallucinating contents. We propose a novel contrastive learning framework MixCL to address this problem.

5 MixCL↩︎

Next, we present the proposed MixCL framework for addressing the hallucination of LMs. MixCL explicitly samples negative knowledge (i.e., non-factual or irrelevant knowledge) and reduces the generation probability of negative tokens by LMs through contrastive learning. As illustrated in Fig. 3, MixCL consists of two steps: negative sampling and mixed contrastive learning. In this section, we first present the negative sampling methods, then the mixed contrastive learning, and finally our optimization strategies.

5.1 Negatives sampling↩︎

We sample negative knowledge for the dialogue context to construct training examples for contrastive learning. Formally, let \(z^+\) be positive knowledge, i.e., a factual and relevant knowledge snippet, and let \(\mathcal{Q}_{\text{Pos}}(x)\) be the collection of positive knowledge regarding \(x\), where the \(z^+ \sim \mathcal{Q}_{\text{Pos}}(x)\) is sampled from it. Here, \(\mathcal{Q}_{\text{Pos}}(x)\) can be obtained through human labeling [4] or heuristic methods [19]. We define \(z^-\) as negative knowledge, i.e., a non-factual or irrelevant knowledge snippet for \(x\). Then, negative sampling is applied to construct the snippets \(z^-\) where the model is most likely to get confused. We introduce two methods for negative sampling, i.e., retrieved negatives and model-generated negatives, as illustrated in Fig. 3.

5.1.0.1 Retrieved negatives.

For a given \(x\), a retrieval tool \(\text{Ret}(*)\) is employed to retrieve irrelevant but potentially confusing knowledge from knowledge corpus \(\mathcal{K}\): \[\label{eq:ret} \mathcal{Q}_{\text{Ret}}(x) = \{z^-|z^-\in\text{Ret}(x, \mathcal{K}), z^-\notin\mathcal{Q}_{\text{Pos}}(x)\},\tag{3}\] where \(\text{Ret}(\cdot,\cdot)\) is implemented as TF-IDF retriever [4], and \(z^-\notin\mathcal{Q}_{\text{Pos}}(x)\) imposes the constraint that negative knowledge snippets should not be included in the positive knowledge.

5.1.0.2 Model-generated negatives.

We also exploit a model bootstrapping approach, in which we generate knowledge by a model \(p_\theta(z|x)\) and retain the examples where hallucination exist. We define: \[\label{eq:model} \mathcal{Q}_{\text{Model}}(x) = \{z^-|z^-\sim p_\theta(z|x), z^-\cap\mathcal{Q}_{\text{Pos}}(x)=\varnothing\},\tag{4}\] where \(z^-\sim p_\theta(z|x)\) denotes a negative knowledge snippet sampled from the LM with \(\theta\), and \(z^-\cup\mathcal{Q}_{\text{Pos}}(x)=\varnothing\) imposes the constraint that negative knowledge snippets should not be included in the positive knowledge, which is implemented with a natural language inference (NLI) toolkit.³

On the basis of the above two methods, we define the constructed negative collection \(\mathcal{Q}_{\text{Neg}}(x)\) with a hyper-parameter \(\beta \in [0,1]\) to control the relative contribution of the methods: \[\label{eq:neg} \mathcal{Q}_{\text{Neg}}(x) = \beta \mathcal{Q}_{\text{Ret}}(x) + (1 - \beta) \mathcal{Q}_{\text{Model}}(x).\tag{5}\]

5.2 Mixed contrastive learning↩︎

Based on the positive knowledge \(z^+\) and the sampled negative knowledge \(z^-\), we introduce a contrastive learning framework to identify positive knowledge from negatives: \[\label{eq:cl} \mathcal{L}_{\text{CL}} = \mathbb{E}_{z^+\sim\mathcal{Q}_{\text{Pos}}(x),\{z^-_i\}_{i=1}^{M}\overset{\text{iid}}{\sim}\mathcal{Z}} l(x,z^+,\{z^-_i\}_{i=1}^{M},\theta),\tag{6}\] where \(l\) denotes a contrastive loss function that is typically defined as cross-entropy loss \(l_{\text{ce}}\)⁴ [30], [40], and \(M\) denotes the number of negatives.

However, \(l_{\text{ce}}\) only considers token-level or sentence-level contrast. It ignores fine-grained span-level contrast even though hallucinations typically exists at the span level. Therefore, inspired by work on mix-up data augmentation [18], [42]–[44], we propose a mixed contrast objective, which mixes the positive and negative examples into a sequence at the span level. As illustrated in Fig. 3, the proposed mixed contrastive learning method has three parts:

5.2.1 Extracting spans.↩︎

We extract the key components from the both positive and negative knowledge, \(z^+\) and \(z^-\). Regarding the two types of hallucinations, i.e., the intrinsic and extrinsic, we design two extraction strategies. As part of the pilot experiment reported in Section 1, we find that intrinsic hallucinations are typically associated with confused entities. Therefore, we use named entity recognition (NER)⁵ to extract entities of various types e.g. person and time. Moreover, we find that extrinsic hallucination is mainly triggered by the emergence of irrelevant sentence fragments in the text. Therefore, we use constituency parsing (CP)⁶ to extract sentence constituents, e.g., noun and particle. Through the two strategies, we extract sequence spans from \(z^+\) and \(z^-\), respectively.

Example. Consider knowledge snippets about the French soccer player Thierry Henry. A statement like He was born and raised in Paris would be in \(z^+\), while the span “Montreal, Quebec, Canada” could be extracted from a snippet such as in He was born in Montreal, Quebec, Canada in \(z^-\).

5.2.2 Mixing examples.↩︎

Based on the extracted spans, we mix the two examples \(z^+\) and \(z^-\) into a mixed sequence \(\tilde{z}\) via a mix-up function: \(\tilde{z} = \text{Mix}(z^+,z^-)\). The mix-up function randomly selects a span in \(z^+\), and then selects a span with the same type in \(z^-\) to substitute it. We define a sequence \(\phi\) with the same length of \(\tilde{z}\), which annotates the tokens in \(\tilde{z}\) as \(1\) if they come from \(z^+\) and \(0\) if they come from \(z^-\).

In the earlier Thierry Henry example, the span “Paris” in a snippet in \(z^+\) can be selected and substituted by the corresponding ones from a snippet in \(z^-\), such as “Montreal, Quebec, Canada.”

5.2.3 Mixed-contrast loss.↩︎

Based on the mixed sequence \(\tilde{z}\) and \(\phi\), we design a loss function \(l_{mix}\) as follows: \[\begin{align} l_{mix}(z^+,z^-)={}&\\ \textstyle- \sum_{j=1}^{|\tilde{z}_i|} \lbrack~&\phi_{i,j}\log p_\theta(\tilde{z}_{i,j}|\tilde{z}_{i,<j},x) +{}\\ &(1-\phi_{i,j}) \log (1 - p_\theta(\tilde{z}_{i,j}|\tilde{z}_{i,<j},x))\rbrack, \end{align} \label{eq:mix-loss}\tag{7}\] where \(\tilde{z}_i=\text{Mix}(z^+,z^-_i)\) is a mixed sequence of \(z^+\) and \(z^-_i\), and \(\phi_{i,j}\) denotes the sign of token \(\tilde{z}_{i,j}\), which equals \(1\) for positive tokens and \(0\) for negative tokens. Using the negative collection \(\mathcal{Q}_{\text{Neg}}(x)\) defined in Eq. 5 , we formalize the mixed contrast objective \(\mathcal{L}_{\text{MCL}}\) as: \[\textstyle {\sum}_{z^+\sim\mathcal{Q}_{\text{Pos}}(x)}{\sum}_{z^-_i\sim\mathcal{Q}_{\text{Neg}}(x)}^{i=1,...,M} l_{mix}(x,z^+,z^-_i,\theta). \label{eq:mcl}\tag{8}\]

5.3 Optimization↩︎

During finetuning, MixCL is optimized by minimizing \(\mathcal{L}_{\text{MCL}}\). Two additional loss are considered in training, i.e., \(\mathcal{L}_{\text{LM}}\), \(\mathcal{L}_{\text{MLE}}\). \(\mathcal{L}_{\text{LM}}\) is used to alleviate catastrophic knowledge forgetting [7], [45] and \(\mathcal{L}_{\text{MLE}}\) is used to optimize the response generation ability. Therefore, the final training objective is defined as: \[\label{eq:all} \mathcal{J}(\theta) = \alpha_1 \mathcal{L}_{\text{MLE}} + \alpha_2 \mathcal{L}_{\text{MCL}} + \alpha_3 \mathcal{L}_{\text{LM}},\tag{9}\] where three losses are optimized jointly and \(\alpha_1\), \(\alpha_2\), \(\alpha_3\) denote the weights of the three losses, respectively.

6 Experimental setup↩︎

6.1 Datasets and evaluation metrics↩︎

We conduct experiments on the Wizard of Wikipedia (WoW) dataset. WoW is built with crowd-sourcing and employs Wikipedia as the knowledge corpus. WoW consists of 22,311 conversations over 1,365 general topics that range from e-books to toga parties to showers. The ground-truth knowledge used in each turn is manually labeled. The WoW test set is split into test seen and test unseen based on whether the topic appears in the training set. We evaluate our methods on both test seen and test unseen.

We choose F1, ROUGE, BLEU, MT, Knowledge-F1 (KF1), Entity-F1 (EF1), and Accuracy (Acc) as metrics. F1 [4] calculates the unigram F1 between the generated text and the ground-truth text. For ROUGE [46] we use ROUGE-L (RL for short) following previous work. BLEU [47] we use BLEU-2 and BLEU-4 (or B2 and B4 for short) and use the implementation in the NLTK Toolkit. MT (Meteor) [48] is based on the harmonic mean of unigram precision and recall. Knowledge-F1 [4] (or KF1 for short) calculates the F1 between the generated response and the ground-truth knowledge sentence, which indicates the informativeness of a response. Acc measures the knowledge selection accuracy. As we skip the knowledge selection step, we select knowledge by matching the generated response with each knowledge candidate in WoW using the F1 score. Entity-F1 (or EF1 for short) identifies entities in text using Spacy, deletes the non-entity words, and calculates the F1 score between the modified generated text and the ground-truth response. EF1 eliminates the impact of the stop-word and focuses on the accuracy of entities.

In addition, we randomly sample 100 examples from the test seen and test unseen segments of the test set, respectively, and recruit three experts for human evaluation. Each annotator is presented with examples that come with dialogue context and model responses. Four metrics are considered in the human evaluation: Informativeness, which measures whether the response is knowledge-inclusive; Relevancy, which measures whether the response’s content is relevant to the dialogue; Factuality, which measures whether the information in the response is factually correct;⁷ and Humanlikeness, which measures whether the response is human-like in its fluency and naturalness. The annotators are asked to assign a score in \(\{0, 1\}\) (representing “non-factual” and “factual”) for factuality, and a score in \(\{0, 1, 2\}\) (representing “bad” “fair”, and “good”) for the others.

6.2 Baselines↩︎

We compare MixCL with baselines of two categories:

All models are re-evaluated with the same evaluation function using the official public checkpoints.

The KB-based methods we consider are: TMN [4] (50M), which combines a transformer with an external memory network to select knowledge and generate a response; DukeNet [21] (150M), which is the best performing KB-based method without using pre-trained LMs and which models knowledge shift with a dual learning scheme; KnowledGPT [19] (227M), which exploits pre-trained LMs in a KB-based approach, selects knowledge using BERT, generates responses using GPT-2, and optimizes the two modules jointly with reinforcement learning; it achieves state-of-the-art performance. We also introduce KnowBART (600M), a KB-based model that selects knowledge using RoBERTa and generates responses using BART-Large.

The KB-based methods listed above retrieve knowledge under oracle conditions, i.e., they are given a small subset of Wikipedia with roughly ten passages that definitely contain the ground-truth knowledge [4], [5]. We also consider KB-based methods under realistic experimental conditions, where passages from the full knowledge corpus (i.e., Wikipedia) are retrieved. We employ the state-of-the-art passage retrieval model GENRE [49] from the KILT leaderboard [50], which is reported to outperform competitors (e.g., DPR and BM25) by a substantial margin on WoW.

The LM-based methods that we consider are: GPT-2 [12] (345M),which finetunes GPT-2 on knowledge-grounded dialogue data; BlenderBot [26] (400M), which pre-trains a transformer with encoder-decoder architecture on reddit data, and then finetunes the model on KGD data; KnowExpert [6] (117M), which uses a topic-aware adapter that first clusters Wikipedia using a topic model and then employs a mix-of-adapter architecture to adapt a GPT-2 model to open-domain dialogues; MSDP [5] (357M), which uses a multi-stage prompting model, designs task-specific prompts with task instructions and in-context examples, and uses Megatron-LM [51] to produce knowledge and response in a two-stage process.

6.3 Implementation details↩︎

We implement MixCL using BART-Large (400M) [15] in HuggingFace’s Transformers library. We use Wikipedia as the knowledge corpus \(\mathcal{K}\), as it is used as knowledge corpus by WoW. We determine the hyperparameters through pilot experiments. We set the weight of the language model loss \(\alpha_3\) to \(0.3\) at initialization and linearly decay until \(0\). We set \(\alpha_1\) and \(\alpha_2\), i.e., the weight of the MLE loss and MCL loss, to \(0.4\) and \(0.3\), respectively, and linearly increase to \(0.5\) and \(0.5\). We use greedy decoding in testing. More details are available in Appendix 10 or at https://github.com/sunnweiwei/MixCL.

7 Experimental results↩︎

7.1 Results of automatic evaluation↩︎

Table ¿tbl:table:main? shows the results of automatic evaluation metrics. Overall, MixCL achieves the highest scores of the LM-based methods and competitive results compared to the KB-based methods under realistic conditions. Compared with previous LM-based methods (the third group in Table ¿tbl:table:main?), MixCL achieves the highest scores on almost all metrics. For example, MixCL gets F1\({}={}\)​21.6, B4\({}={}\)​2.7 on test seen and F1\({}={}\)​19.6, B4\({}={}\)​1.4 on test unseen, with about 5% to 15% relative improvements over previous-best LM-based baselines. Moreover, we find a dilemma with the previous LM-based methods in terms of response quality (e.g., F1, RL, B2) and knowledge relevance (e.g., KF1, EF1, Acc). For example, MSDP performs well on knowledge relevance at the expense of response quality, while GPT-2 and BlenderBot show the opposite. MixCL, on the other hand, performs well on both fronts.

Furthermore, compared with KB-based methods (the first block in Table ¿tbl:table:main?), we find that MixCL outperforms two non-LM methods (DukeNet and TMN) by a large margin. Compared to KnowledGPT and KnowBART, which combine LMs with the KB-based approach, MixCL outperforms them on test seen. On test unseen, MixCL lags behind the best performing KB-based baselines, probably due to knowledge forgetting issues.

Finally, under oracle conditions, the KB-based methods (the second group in Table ¿tbl:table:main?) show better results than MixCL. However, the manually selected knowledge candidates include the ground-truth, which is unavailable in realistic scenarios.

7.2 Results of human evaluation↩︎

Table 1 shows the human evaluation results. The Fleiss’ kappa value is above \(0.60\), indicating substantial agreement among the annotators. MixCL consistently outperforms LM-based baselines on all metrics, and also outperforms KB-based baselines in metrics. MixCL is capable of generating more informative responses compared to previous LM-based methods. Moreover, MixCL effectively increases relevance and factuality, demonstrating its effectiveness in reducing both types of hallucinations. In particular, we find that KnowledGPT is outperformed by MixCL in terms of knowledge relevance, probably due to the presence of retrieval errors. Finally, MixCL’s responses are considered more human-like by the annotators.

7.3 Ablation studies↩︎

In Table 2, we compare MixCL with several ablative variants. The variants and our findings are as follows:

No \(\mathcal{L}_{\text{MCL}}\) – We remove the mixed contrast objective. The performance of the model shows a notable degradation, especially for the knowledge relevance metric, i.e., KF1. This suggests that the proposed mixed contrast objective is effective in increasing the relevance of responses.

No \(\mathcal{Q}_{\text{Neg}}(*)\) – We remove the hard negative sampling process and use randomly sampled instances as negatives. The effectiveness of the hard negative sampling is evidenced by the decrease in the metric on KF1.

No \(\mathcal{Q}_{\text{Model}}\) – We remove the negatives generated by the model. The results suggest that model-generated negatives provides harder negative examples for the model, i.e., the knowledge that is more likely to be confounded by the LMs.

No \(\mathcal{L}_{\text{LM}}\) – We remove the LM loss. The effect of the model is a decline, especially on unseen topics. This results suggests that LM loss is instrumental in suppressing the catastrophic knowledge forgetting problem of the LMs in conversations [45].

Only \(\mathcal{L}_{\text{MLE}}\) – This variant optimizes the model only by MLE loss. We observe a substantial performance drop, especially on KF1, which demonstrates the effectiveness of MixCL in improving the knowledge relevancy and factuality of LMs.

7.4 Efficiency analysis↩︎

In Fig. 4, we compare MixCL against baselines in terms of efficiency and effectiveness. We adjust the inference efficiency of the models by evaluating the model with different numbers of parameters (e.g., 140M and 400M). Compared with KB-based methods, LM-based methods generally have an advantage in terms of speed as they get rid of the extra IR step. However, previous LM-based methods are outperformed by KB-based methods regarding response quality. By explicitly eliminating the hallucinations of LM in conversations, MixCL significantly improves the response quality of LM-based methods without compromising efficiency. Notably, MixCL is 5\(\times\) more efficient than state-of-the-art KB-based methods while achieving competitive response generation performance. Moreover, the improvements of MixCL along with the model size are more noticeable compared to KB-based methods (see the dashed lines), indicating its superior ability to utilize the knowledge of pre-trained model.

7.5 Case study↩︎

We conduct several case studies and find that MixCL is more effective at incorporating knowledge and generating more engaging and human-like responses than baselines. Details about our case studies are available in Appendix 11 or https://github.com/sunnweiwei/MixCL.

8 Conclusions↩︎

In this paper, we have proposed MixCL, a contrastive learning framework aimed at reducing the hallucination of language models in conversations. MixCL is enhanced by negative sampling and mixed contrastive objective. Experiments on the Wizard-of-Wikipedia dataset have shown that MixCL outperforms existing LM-based methods and achieves comparable performance as state-of-the-art KB-based methods. Human evaluation and ablative experiments also confirm MixCL’s effectiveness in eliminating hallucination of LMs. Moreover, MixCL demonstrates advantages in terms of efficiency and scalability. Hence, we believe that MixCL provides new insights on using knowledge inside large language models’ parameters for KGD tasks.

The limitations of this work include the problem of knowledge forgetting. In future work, we would like to explore practical approaches to avoiding catastrophic knowledge forgetting. We also plan to reproduce our findings for other, less resource-rich languages.

Acknowledgements↩︎

This work was supported by the National Key R&D Program of China with grant No. 2020YFB1406704, the Natural Science Foundation of China (62272274, 62202271, 61902219, 61972234, 62072279, 62102234), the Natural Science Foundation of Shandong Province (ZR2021QF129), the Key Scientific and Technological Innovation Program of Shandong Province (2019JZZY010129), and by the Hybrid Intelligence Center, a 10-year program funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research, https://hybrid-intelligence-centre.nl.

All content represents the opinion of the authors, which is not necessarily shared or endorsed by their respective employers and/or sponsors.

9 Results of pilot experiment↩︎

Our pilot human evaluation revealed that 51% of the responses of BART are hallucinatory on the datasets of Wizard-of-Wikipedia. We group the hallucinations into two types: intrinsic hallucinations and extrinsic hallucinations. The rest is categorized as other. Here, we provide more detailed results and examples from each type:

• Intrinsic hallucinations that account for 24% of the annotated data, are non-factual statements. Specifically, intrinsic hallucinations can be further grouped into three cases.

• Extrinsic hallucinations that account for 27% of the annotated data, are irrelevant responses, which can be further grouped into three cases.

• Others that account for 49%. First of all, 32.5% of the annotated examples were considered qualified responses. We find that of the remaining responses 8% used knowledge in a non-human-like way resulting in mechanical responses, 5.5% did not use knowledge, and 3% repeated the knowledge used in the previous dialogue turns.

We summarize these findings in Fig. 5.

10 Implementation details↩︎

We set the hyperparameters by pilot experiments following the common practice: First, we set the ratio of the three losses to {0.4,0.3,0.3} to make them balanced and not oversampled; Second, we use the objective shifting method (Chen et al., 2020) to vary the scale linearly to {0.5,0.5,0} to avoid knowledge forgetting and balance the upstream/downstream objectives. We set the ratio of language model loss \(\alpha_3\) to \(0.3\) at initialization and linearly decay it until \(0\). We set \(\alpha_1\) and \(\alpha_2\), i.e., the ratio of MLE loss and MCL loss, to \(0.4\) and \(0.3\), respectively, and linearly increase them to \(0.5\) and \(0.5\). As for \(\beta\), which controls the ratio of the two span extraction functions, we set it to \(0.5\). The number of negative example (\(M\)) is set to \(8\). We truncate the input sequence to a maximum of 128 tokens, and the output sequence to 64 tokens. We append task-specific prompts (i.e., “Response generation”, “Knowledge identification”, and “Wikipedia denosing”) to three loss (i.e., \(\mathcal{L}_{\text{MLE}}\), \(\mathcal{L}_{\text{MCL}}\), and \(\mathcal{L}_{\text{LM}}\)) to distinguish different training targets separately.

All models have been trained with the AdamW optimizer with a constant learning rate of \(2e{-}5\). We use gradient clipping with a maximum gradient norm of \(0.1\). We optimize the model for up to 5 epochs with a batch size of 16 on 4 V100 GPUs. During training, we first extracted spans in negatives offline, which takes about 1 hour using 20 threads. Then, we use 8 threads for getting the training samples (num_workers=8 in data_loader) and find that this setup does not increase the training time. We choose the model checkpoints by evaluating the metrics on the validation set for each 5,000 training steps. During inference, the responses are decoded using a greedy search. On a single GPU, the training of the model can be completed within 10h, and the average latency of the model inference for one example is 0.05s.

@rp15cm@ Topic &Yoga
& User1: Do you like yoga which is the practice not only physical but mental exercises or disciplines.
& User2: I love yoga! I actually practice hot yoga pretty recently, it combines my two favorite things: Sauna and Yoga.

DukeNet &i love hatha yoga too ! i like hatha yoga and raja yoga .
KnowledGPT &i love hatha yoga and raja yoga. i like both.
KnowExpert &i love hot yoga too, it is a type of yoga that emphasizes physical exercises over emotional ones
MSDP &Yes ! It ’s very intense , classes are usually an hour and a half and by the first half hour you are feeling it , the positions can be intense .
MixCL &I love hot yoga as well. It’s been around since the 5th and 6th centuries BCE.
Human &Oh wow that sounds interesting. I read that Hot Yoga is actually refered to as Bikram Yoga

@rp15cm@ Topic &Online shopping
& User2: I like to shop online, probably a bit too much.
& User1: Lol me too! What’s your favorite online store?
& User2: I love to shop on amazon. So you shop online a lot too? Where do you shop?

DukeNet &i shop in amazon . i shop at the amazon . they are an american multinational technology company based in seattle , washington .
KnowledGPT &i shop online at amazon. it’s a great online store. i like to shop at amazon.
KnowExpert &i shop online at amazon. i like to shop online because i can find the best deals online.
MSDP &Yes I do . I shop online a lot too . I know that Online shopping is a form of electronic commerce which allows consumers to directly buy goods or services from a seller over the Internet using a web browser .
MixCL &I do! I like to shop online at Amazon.com and Barnes & Noble. What’s your favorite store?
Human &Amazon as well. I also shop a lot online at Old Navy and Gap. They have great sales.

@rp15cm@ Topic &Archery
& User2: So I picked up archery recently and to my surprise, I really enjoy it. Have you ever tried?
& User1: Yes, actually! I enjoy archery very much. I actually found out it comes from (arcus) Latin!
& User2: What other Latin words do you know?
& User1: I don’t know many, but I do know that a significant portion of English comes from Latinate sources.
& User2: Huh. When did you start archery? Do you take part in any competitions?
& User1: I started when I was ten. I got a winning streak when I was 15, which means I won a consecutive sequence of games.
& User2: I think I want to enter one but I have no idea where to start.

DukeNet &the earliest evidence of use traces to the olmecs ( mexico ) , with evidence of chocolate beverages dating to 1900 bc
KnowledGPT &you should start by playing chess. it’s played by millions of people worldwide.
KnowExpert &i’m not sure, but i do know that archery is a sport that involves using a bow and arrow to shoot a long-range shot at targets on a moving target.
MSDP &I think I want to enter one but I have no idea where to start .
MixCL &You should start with the World Archery Championship. It’s the biggest one in the world.
Human &You could enter the Junior Grand Prix! It’s a series of international junior level competitions.

@rp15cm@ Topic &Paramedic
& User2: my friend wanted to be a paramedic
& User1: I wouldn’t want to be EMS on 911 calls. The scenes would haunt you for life.
& User2: yea thats why my friend quit
& User1: I imagine it is not as bad not that there is enchanced 911 so paramedics only have to go to the calls they are absolutely neccessary for
& User2: oh yea really?
& User1: In Winnipeg they have was is known as the winnipeg fire paramedic servic, or WFPS. 6They have 2 different paramedic divisions depending on the type of 911 call.
& User2: oh i didnt know that.
& User1: it started in 1882 with the Winnipeg Fire Department and grew from there.
& User2: dang thats so long ago

DukeNet &it was built in 1738 the first fort in 1738 .
KnowledGPT &the library has both adult and children’s books in over 30 languages. i think i have a lot of books in my library.
KnowExpert &it was founded in 1881 by a man named william c. durant.
MSDP &Wow that’s so long ago . I know that Paramedic is a children ’s book by Dr. Seuss , first published on August 12 , 1960 .
MixCL &Yes, the Winnipeg Fire Department is one of the largest fire departments in the world.
Human &It is from our view point but not in the grand scheme of things.

@rp15cm@ Topic &Valedictorian
& User2: I hope I am Valedictorian of my graduating class.
& User1: Valedictorian is an academic title of success used in the United States, Canada, Central America, and the Philippines for the student who delivers the closing or farewell statement at a graduation ceremony
& User2: I’m not sure what kind of grades I’ll need.

DukeNet &the chosen valedictorian is the highest ranking grade average
KnowledGPT &the chosen valedictorian is often the student with the highest ranking (highest grade point average, or gpa for short) among their graduating class
KnowExpert &valedictorian is also known as a bachelors of arts or bsc
MSDP &I don’t know what kind of grades I ’ll need . I know that: The valedictory address generally is considered a final farewell to classmates , before they disperse to pursue their individual paths after graduating .
MixCL &In the United States, the title is sometimes awarded to a member of a graduating university class on the basis of contribution to the school rather than academic success.
Human &valedictorian is often the student with the highest ranking (highest Grade Point Average, or GPA for short) among their graduating class.

11 Case study↩︎

We provide examples of model output in Table ¿tbl:table:outputs1?, ¿tbl:table:outputs2?, ¿tbl:table:outputs3?,¿tbl:table:outputs4?, and ¿tbl:table:outputs5?. In Table ¿tbl:table:outputs1?, we see that MixCL generates an informative response through the use of knowledge (i.e., the history of yoga). Compared to the baselines, MixCL’s response is more specific and relevant: DukeNet and KnowledGPT express subjective preferences rather than knowledge, KnowExpert repeats the knowledge from the previous round, and MSDP’s response is hallucinating.

In Table ¿tbl:table:outputs2?, we see that despite appropriate integration of knowledge, MixCL repeatedly asked “What’s your favorite store” as it ignores the dialogue context, leading the annotator to consider its response is non-human. Still, we find MixCL’s response to be competent compared to the baselines. Besides, we find that MSDP’s response, although very informative, is wordy and preachy with its rigid insertion of large paragraphs of knowledge.

In Table ¿tbl:table:outputs3?, we see that KB-based methods (DukeNet and KnowledGPT) suffer from the problem of retrieval error. Their knowledge retrieval module misinterpreted the dialogue context and retrieved irrelevant knowledge (e.g., chocolate beverages and chess). Meanwhile, we find that MSDP simply repeats the last utterance. In contrast, MixCL’s response is the best in terms of relevance, factuality, and humanlikeness, because it responds to the user’s question, uses the correct knowledge, and is fluent in expression.

In Table ¿tbl:table:outputs4?, we provide an example that MixCL produces a non-factual response. In Table ¿tbl:table:outputs5?, we provide an example that MixCL uses appropriate knowledge, albeit distinct from the knowledge in the human response.

References↩︎