1 Introduction↩︎

While NLP models have achieved great successes, results can depend on spurious correlations with protected attributes of the authors of a given text, such as gender, age, or race. Including protected attributes in models can lead to problems such as leakage of personally-identifying information of the author [1], and unfair models, i.e., models which do not perform equally well for different sub-classes of user. This kind of unfairness has been shown to exist in many different tasks, including part-of-speech tagging [2] and sentiment analysis [3].

One approach to diminishing the influence of protected attributes is to use adversarial methods, where an encoder attempts to prevent a discriminator from identifying the protected attributes in a given task [1]. Specifically, an adversarial network is made up of an attacker and encoder, where the attacker detects protected information in the representation of the encoder, and the optimization of the encoder incorporates two parts: (1) minimizing the main loss, and (2) maximizing the attacker loss (i.e., preventing protected attributes from being detected by the attacker). Preventing protected attributes from being detected tends to result in fairer models, as protected attributes will more likely be independent rather than confounding variables. Although this method leads to demonstrably less biased models, there are still limitations, most notably that significant protected information still remains in the model’s encodings and prediction outputs [4], [5].

Many different approaches have been proposed to strengthen the attacker, including: increasing the discriminator hidden dimensionality; assigning different weights to the adversarial component during training; using an ensemble of adversaries with different initializations; and reinitializing the adversarial weights every \(t\) epochs [5]. Of these, the ensemble method has been shown to perform best, but independently-trained attackers can generally still detect private information after adversarial removal.

In this paper, we adopt adversarial debiasing approaches and present a novel way of strengthening the adversarial component via orthogonality constraints [6]. Over a sentiment analysis dataset with racial labels of the document authors, we show our method to result in both more accurate and fairer models, with privacy leakage close to the lower-bound.¹

2 Methodology↩︎

Formally, given an input \(x_{i}\) annotated with main task label \(y_{i}\) and protected attribute label \(g_{i}\), a main task model \(M\) is trained to predict \(\hat{y}_{i}=M(x_{i})\), and an adversary, aka “discriminator”, \(A\) is trained to predict \(\hat{g}_{i}=A(h_{M,i})\) from \(M\)’s last hidden layer representation \(h_{M,i}\). In this paper, we treat a neural network classifier as a combination of two connected parts: (1) an encoder \(E\), and (2) a linear classifier \(C\). For example, in the main task model \(M\), the encoder \(E_{M}\) is used to compute the hidden representation \(h_{M,i}\) from an input \(x_{i}\), i.e., \(h_{M,i}=E_{M}(x_{i})\), and the decoder is used to make a prediction, \(\hat{y}_{i}=C_{M}(h_{M,i})\). Similarly, for a discriminator, \(\hat{g}_{i}=A(h_{M,i})=C_{A}(E_{A}(h_{M,i}))\).

2.1 Adversarial Learning↩︎

Following the setup of [1] and [5] the optimisation objective for our standard adversarial training is: \[\min_{M}\max_{A}\mathcal{X}(y, \hat{y}_M) - \lambda_{\text{adv}} \mathcal{X}(g, \hat{g}_A),\] where \(\mathcal{X}\) is cross entropy loss, and \(\lambda_{\text{adv}}\) is the trade-off hyperparameter. Solving this minimax optimization problem encourages the main task model hidden representation \(h_{M}\) to be informative to \(C_{M}\) and uninformative to \(A\). Following [7], the above can be trained using stochastic gradient optimization with a gradient reversal layer for \(\mathcal{X}(g, \hat{g}_A)\).

2.2 Differentiated Adversarial Ensemble↩︎

Inspired by the ensemble adversarial method [5] and domain separation networks [8], we present differentiated adversarial ensemble, a novel means of strengthening the adversarial component. Figure [fig:separation95adv] shows a typical ensemble architecture where \(k\) sub-discriminators are included in the adversarial component, leading to an averaged adversarial regularisation term: \[-\frac{\lambda_{\text{adv}}}{k}\sum_{j\in\{1,...,k\}} \mathcal{X}(g, \hat{g}_{A_{j}}).\]

One problem associated with this ensemble architecture is that it cannot ensure that different sub-discriminators focus on different aspects of the representation. Indeed, experiments have shown that sub-discriminator ensembles can weaken the adversarial component [5]. To address this problem, we further introduce a difference loss [8] to encourage the adversarial encoders to encode different aspects of the private information. As can be seen in Figure [fig:separation95adv], \(h_{A_{k},i}\) denotes the output from the \(k\)-th sub-discriminator encoder given a hidden representation \(h_{M,i}\), i.e., \(h_{A_{k},i}=E_{A_{k}}(h_{M,i})\).

The difference loss encourages orthogonality between the encoding representations of each pair of sub-discriminators: \[\mathcal{L}_{\text{diff}} = \lambda_{\text{diff}}\sum_{i,j\in\{1,...,k\}}\norm{{h_{A_{i}}}^{\intercal}h_{A_{j}}}_{F}^{2}\mathbb{1}(i\neq{j}),\] where \(\norm{\cdot}_{F}^{2}\) is the squared Frobenius norm.

Intuitively, sub-discriminator encoders must learn different ways of identifying protected information given the same input embeddings, resulting in less biased models than the standard ensemble-based adversarial method. According to [8], the difference loss has the additional advantage of also being minimized when hidden representations shrink to zero. Therefore, instead of minimizing the difference loss by learning rotated hidden representations (i.e., the same model), this method biases adversaries to have representations that are a) orthogonal, and b) low magnitude; the degree to which is given by weight decay of the optimization function.

[!t]
\centering
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{ll@{\smaller$\pm$}Vr@{\smaller$\pm$}Vr@{\smaller$\pm$}Vl@{\smaller$\pm$}Vl@{\smaller$\pm$}V}
\toprule
Model         & \multicolumn{2}{c}{Accuracy$\uparrow$}     &\multicolumn{2}{c}{TPR Gap$\downarrow$}&\multicolumn{2}{c}{TNR Gap$\downarrow$}&\multicolumn{2}{c}{Leakage@$\mathbf{h}$ $\downarrow$}& \multicolumn{2}{c}{Leakage@$\hat{y}$ $\downarrow$} \\ 
\midrule
Random        & 50.00    &     0.00              & 0.00   &  0.00            & 0.00   &  0.00            & \multicolumn{2}{c}{---\z\z}   & \multicolumn{2}{c}{---\z\z} \\
  Fixed Encoder & 61.44    &     0.00               & 0.52   &  0.00             & 17.97  &  0.00             & 92.07  &  0.00            & 86.93  &  0.00\\[1ex]
Standard      & 71.59    &     0.05              & 31.81  &  0.29            & 48.41  &  0.27            & 85.56  &  0.20            & 70.09  &  0.19 \\[1ex]
INLP        & 68.54     & 1.05      & 25.13     & 2.31      & 40.70     & 5.02  & \bf 66.64     & 0.87 & 66.19 & 0.79 \\
Adv Single Discriminator           & 74.25    &     0.39              & 13.01  &  3.83            & 28.55  &  3.60            & 84.33  &  0.98            & 61.48  &  2.17 \\
Adv Ensemble      & 74.08    &     0.99              & 12.04  &  3.50            & 31.76  &  3.19            & 85.31  &  0.51            & 63.23  &  3.62 \\[1ex]
Differentiated Adv Ensemble  & \bf 74.52&     0.28              &\bf 8.42&  1.84            & \bf 24.74  &  2.07            & 84.52  &  0.50            & \bf 61.09  &  2.32 \\
\bottomrule
\end{tabular}
\end{adjustbox}
\caption{
Evaluation results $\pm$ standard deviation ($\%$) on the test set, averaged over 10 runs with different random seeds. 
\textbf{Bold} = best performance. ``$\uparrow$'' and ''$\downarrow$'' indicate that higher and lower performance, resp., is better for the given metric. Leakage measures the accuracy of predicting the protected attribute, over the final hidden representation $\mathbf{h}$ or model output $\hat{y}$. Since the Fixed Encoder is not designed for binary sentiment classification, we merge the original 64 labels into two categories based on the results of hierarchical clustering.}
\label{table:results}

2.3 INLP↩︎

We include Iterative Null-space Projection (“INLP”: [9]) as a baseline method for mitigating bias in trained models, in addition to standard and ensemble adversarial methods. In INLP, a linear discriminator (\(A_{\text{linear}}\)) of the protected attribute is iteratively trained from pre-computed fixed hidden representations (i.e., \(h_{M}\)) to project them onto the linear discriminator’s null-space, \(h_{M}^{*} = P_{N(A_{\text{linear}})}h_{M}\), where \(P_{N(A_{\text{linear}})}\) is the null-space projection matrix of \(A_{\text{linear}}\). In doing so, it becomes difficult for the protected attribute to be linearly identified from the projected hidden representations (\(h_{M}^{*}\)), and any linear main-task classifier (\(C_{M}^{*}\)) trained on \(h_{M}^{*}\) can thus be expected to make fairer predictions.

3 Experiments↩︎

3.0.0.1 Fixed Encoder

Following [5] and [9], we use the DeepMoji model [10] as a fixed-parameter encoder (i.e.it is not updated during training). The DeepMoji model is trained over 1246 million tweets containing one of 64 common emojis. We merge the 64 emoji labels output by DeepMoji into two super-classes based on hierarchical clustering: ‘happy’ and ‘sad’.

3.0.0.2 Models

The encoder \(E_{M}\) consists of a fixed pretrained encoder (DeepMoji) and two trainable fully connected layers (“Standard” in Table [table:results]). Every linear classifier (\(C\)) is implemented as a dense layer.

For protected attribute prediction, a discriminator (\(A\)) is a 3-layer MLP where the first 2 layers are collectively denoted as \(E_{A}\), and the output layer is denoted as \(C_{A}\).

3.0.0.3 TPR-GAP and TNR-GAP

In classification problems, a common way of measuring bias is TPR-GAP and TNR-GAP, which evaluate the gap in the True Positive Rate (TPR) and True Negative Rate (TNR), respectively, across different protected attributes [11]. This measurement is related to the criterion that the prediction \(\hat{y}\) is conditionally independent of the protected attribute \(g\) given the main task label \(y\) (i.e., \(\hat{y} \bot g | y\)). Assuming a binary protected attribute, this conditional independence requires \(\mathbb{P}\{\hat{y}|y, g=0\} = \mathbb{P}\{\hat{y}|y, g=1\}\), which implies an objective that minimizes the difference (GAP) between the two sides of the equation.

3.0.0.4 Linear Leakage

We also measure the leakage of protected attributes. A model is said to leak information if the protected attribute can be predicted at a higher accuracy than chance, in our case, from the hidden representations the fixed encoder generates. We empirically quantify leakage with a linear support vector classifier at two different levels:

3.0.0.5 Data

We experiment with the dataset of [12], which contains tweets that are either African American English (AAE)-like or Standard American English (SAE)-like (following [5] and [9]). Each tweet is annotated with a binary “race” label (on the basis of AAE or SAE) and a binary sentiment score, which is determined by the (redacted) emoji within it.

In total, the dataset contains 200k instances, perfectly balanced across the four race–sentiment combinations. To create bias in the dataset, we follow previous work in skewing the training data to generate race–sentiment combinations (AAE–happy, SAE–happy, AAE–sad, and SAE–sad) of \(40\%\), \(10\%\), \(10\%\), and \(40\%\), respectively. Note that we keep the test data unbiased.

3.0.0.6 Training Details

All models are trained and evaluated on the same training/test split. The Adam optimizer [13] is used with learning rates of \(3\times10^{-5}\) for the main model and \(3\times10^{-6}\) for the sub-discriminators. The minibatch size is set to 1024. Sentence representations (2304d) are extracted from the DeepMoji encoder. The hidden size of each dense layer is 300 in the main model, and 256 in the sub-discriminators. We train \(M\) for 60 epochs and each \(A\) for 100 epochs, keeping the checkpoint model that performs best on the dev set. Similar to [5], hyperparameters (\(\lambda_{\text{adv}}\) and \(\lambda_{\text{diff}}\)) are tuned separately rather than jointly. \(\lambda_{\text{adv}}\) is tuned to \(0.8\) based on the standard (single-discriminator) adversarial learning method, and this setting is used for all other adversarial methods. When tuning \(\lambda_{\text{adv}}\), we considered both overall performance and bias gap (both over the dev data). Since adversarial training can increase overall performance while decreasing the bias gap (see Figure 2), we select the adversarial model that achieves the best task performance. For adversarial ensemble and differentiated models, we tune the hyperparameters (number of sub attackers and \(\lambda_{\text{diff}}\)) to achieve a similar bias level while getting the best overall performance. To compare with a baseline ensemble method with a similar number of parameters, we also report results for an adversarial ensemble model with 3 sub-discriminators. The scalar hyperparameter of the difference loss (\(\lambda_{\text{diff}}\)) is tuned through grid search from \(10^{-4}\) to \(10^{4}\), and set to \(10^{3.7}\). For the INLP experiments, fixed sentence representations are extracted from the same data split. Following [9], in the INLP experiments, both the discriminator and the classifier are implemented in scikit-learn as linear SVM classifiers [14]. We report Leakage@\(\hat{y}\) for INLP based on the predicted confidence scores, which could be interpreted as logits, of the linear SVM classifiers.

3.0.0.7 Results and Analysis

Table [table:results] shows the results over the test set. Training on a biased dataset without any fairness restrictions leads to a biased model, as seen in the Gap and Leakage results for the Standard model. Consistent with the findings of [9], INLP can only reduce bias at the expense of overall performance. On the other hand, the Single Discriminator and Adv(ersarial) Ensemble baselines both enhance accuracy and reduce bias, consistent with the findings of [1].

Compared to the Adv Ensemble baseline, incorporating the difference loss in our method has two main benefits: training is more stable (results have smaller standard deviation), and there is less bias (the TPR and TNR Gap are smaller). Without the orthogonality factor, \(\mathcal{L}_{\text{diff}}\), the sub-discriminators tend to learn similar representations, and the ensemble degenerates to a standard adversarial model. Simply relying on random initialization to ensure sub-discriminator diversity, as is done in the Adv Ensemble method, is insufficient. The orthogonality regularization in our method leads to more stable and overall better results in terms of both accuracy and TPR/TNR Gap.

As shown in Table [table:results], even the Fixed Encoder model leaks protected information, as a result of implicit biases during pre-training. INLP achieves significant improvement in terms of reducing linear hidden representation leakage. The reason is that Leakage@\(\mathbf{h}\) is directly correlated with the objective of INLP, in minimizing the linear predictability of the protected attribute from the \(\mathbf{h}\). Adversarial methods do little to mitigate Leakage@\(\mathbf{h}\), but substantially decrease Leakage@\(\hat{y}\) in the model output. However, both types of leakage are well above the ideal value of 50%, and therefore none of these methods can be considered as providing meaningful privacy, in part because of the fixed encoder. This finding implies that when applying adversarial learning, the pretrained model needs to be fine-tuned with the adversarial loss to have any chance of generating a truly unbiased hidden representation. Despite this, adversarial training does reduce the TPR and TNR Gap, and improves overall accuracy, which illustrates the utility of the method for both bias mitigation and as a form of regularisation. Overall, our proposed method empirically outperforms the baseline models in terms of debiasing, with a better performance–fairness trade-off.

3.0.0.8 Robustness to \(\lambda_{\text{adv}}\)

We first evaluate the influence of the trade-off hyperparameter \(\lambda_{\text{adv}}\) in adversarial learning. As can be seen from Figure 2, \(\lambda_{\text{adv}}\) controls the performance–fairness trade-off. Increasing \(\lambda_{\text{adv}}\) from \(10^{-2}\) to around \(10^{-0}\), TPR Gap and TNR Gap consistently decrease, while the accuracy of each group rises. To balance up accuracy and fairness, we set \(\lambda_{\text{adv}}\) to \(10^{-0.1}\). We also observe that an overly large \(\lambda_{\text{adv}}\) can lead to a more biased model (starting from about \(10^{1.2}\)).

3.0.0.9 Robustness to \(\lambda_{\text{diff}}\)

Figure 3 presents the results of our model with different \(\lambda_{\text{diff}}\) values, for \(N \in \{3,5,8\}\) sub-discriminators.

First, note that when \(\lambda_{\text{diff}}\) is small (i.e., the left side of Figure 3), our Differentiated Adv Ensemble model generalizes to the standard Adv Ensemble model. For differing numbers of sub-discriminators, performance is similar, i.e., increasing the number of sub-discriminators beyond 3 does not improve results substantially, but does come with a computational cost. This implies that an Adv Ensemble model learns approximately the same thing as larger ensembles (but more efficiently), where the sub-discriminators can only be explicitly differentiated by their weight initializations (with different random seeds), noting that all sub-discriminators are otherwise identical in architecture, input, and optimizer.

Increasing the weight of the difference loss through \(\lambda_{\text{diff}}\) has a positive influence on results, but an overly large value makes the sub-discriminators underfit, and both reduces accuracy and increases TPR/TNR Gap. We observe a negative correlation between \(N\) and \(\lambda_{\text{diff}}\), the main reason being that \(\mathcal{L}_{\text{diff}}\) is not averaged over \(N\) and as a result, a large \(N\) and \(\lambda_{\text{diff}}\) force the sub-discriminators to pay too much attention to orthogonality, impeding their ability to bleach out the protected attributes.

Overall, we empirically show that \(\lambda_{\text{diff}}\) only needs to be tuned for Adv Ensemble, since the results for different Differentiated Adv models for a given setting achieve similar results. I.e., \(\lambda_{\text{diff}}\) can safely be tuned separately with all other hyperparameters fixed.

4 Conclusion and Future Work↩︎

We have proposed an approach to enhance sub-discriminators in adversarial ensembles by introducing a difference loss. Over a tweet sentiment classification task, we showed that our method substantially improves over standard adversarial methods, including ensemble-based methods.

In future work, we intend to perform experimentation over other tasks. Theoretically, our approach is general-purpose, and can be used not only for adversarial debiasing but also any other application where adversarial training is used, such as domain adaptation [15].

Acknowledgments↩︎

We thank Lea Frermann, Shivashankar Subramanian, and the anonymous reviewers for their helpful feedback and suggestions.

References↩︎