1 Introduction↩︎

As machine learning algorithms are deployed in societal computer vision applications such as facial recognition [@wang2019racial] and job interview [@nguyen2016hirability], concerns have grown regarding their potential to discriminate against certain individuals and groups. For instance, a face recognition system might exhibit disparate accuracies across different demographic groups [@buolamwini2018gender], while a job interview algorithm could be biased based on protective attributes even for the same interviewee [@objective_biased]. Consequently, algorithmic fairness in image classifiers has gained significant attention in academic and industrial research communities.

While conceptually apparent, determining a concrete notion of fairness is challenging, leading to the proposal of several different fairness notions. One prevalent notion is counterfactual fairness (CF) [@kusner2017counterfactual] which seeks consistent predictions when only a sensitive attribute is intervened. Another important notion is group fairness (GF) [@zafar2017fairness] that aims to treat different demographic groups equally to prevent one group unfairly disadvantaged compared to another. Many researchers have focused on developing separate algorithms to achieve each notion, while understanding the exact relationship between CF and GF is yet under-explored; e.g., some recent work [@anthis2023causal; @rosenblatt2023counterfactual] showed that a model achieving CF can meet several GF notions only under specific conditions of Structural Causal Models.

Furthermore, previous studies on the relationship between CF and GF have not considered the setting of image classification due to the absence of evaluation datasets with counterfactual images, in which only the sensitive attribute is altered from the original images while other attributes not caused by the sensitive attribute remain unchanged — a data nearly impossible to collect in the real world. There have been several works generating counterfactual images using generative models [@dash2022evaluating; @kim2021counterfactual; @louizos2017causal; @pfohl2019counterfactual; @zhang2022fairness; @ramaswamy2021fair; @d2024improving], but they have only focused on utilizing generated counterfactual samples for training rather than evaluation. Moreover, these methods often suffer from low-quality counterfactual images generated based on VAE [@kingma2013auto] or GAN [@goodfellow2014generative]. One notable exception is [@liang2023benchmarking], which offers an evaluation dataset including counterfactual images. However, their images are all synthetic; thus, it is still insufficient to evaluate CF due to distribution shifts from real-world images.

In this paper, we construct CF benchmarks for image classification tasks using high-performing diffusion model-based generative models. Our datasets build upon popular facial benchmark datasets used for evaluating GF, CelebA and LFW, by altering the sensitive attribute with pre-trained InstructPix2Pix (IP2P) [@brooks2023instructpix2pix]. We then carefully curate the edited samples by human annotators and verify the reliability of our datasets as counterfactual samples from additional annotators. Note that our datasets, CelebA-Counterfactual Face(CelebA-CF) and LFW-Counterfactual Face(LFW-CF), share the same test samples as the original GF benchmarks, enabling the evaluation of both GF and CF.

Using our datasets, we conduct a primitive study on the relationship between CF and GF in image classification, e.g., test whether CF implies GF for image classifiers. To that end, we train CF-aware methods [@russell2017worlds; @garg2019counterfactual] and evaluate them with our datasets using both CF and GF metrics. From the result, we observe that they achieve CF but fail to satisfy GF, contrary to previous findings that CF can imply GF [@anthis2023causal; @rosenblatt2023counterfactual]. We attribute this failure to Structural Causal Models (SCMs) of image generation. Specifically, for an image SCM, a latent attribute \(G\) is more likely to exist, which could be correlated with, but not caused by, the sensitive attributes. For example, in the case where the sensitive attribute is the sex of a person in an image, secondary sex characteristics such as beard and hairline are highly correlated with hair length, but it does not mean that such characteristics cause the length of hair. In this scenario, if a model achieving CF relies on the attribute \(G\) (e.g., hair length) on its prediction, it could more severely violate GF in the worst case. Therefore, if we can reduce the dependency on \(G\) of a CF-aware model, we may achieve both CF and GF. Empirically, we find that a model trained with vanilla cross-entropy loss is more robust to \(G\) than a model trained with a CF-aware method. Motivated by this, we propose a simple baseline, named Counterfactual Knowledge Distillation(CKD), which distills the robustness to \(G\) during the original CF-aware optimization. Finally, our extensive experiments using CelebA-CF and LFW-CF demonstrate that CF-achieving models satisfy GF when reducing the reliance on \(G\) (e.g., using CKD).

In summary, our contributions are three-fold. Firstly, we construct two new image classification benchmarks for measuring CF, CelebA-CF and LFW-CF. Secondly, using these datasets, we observe the disparity between CF and GF in image classifiers and provide a theoretical rationale; a counterfactually fair classifier may not necessarily achieve GF when an additional latent attribute that is correlated with the sensitive attribute exists. Finally, we propose a simple baseline, CKD, to reduce the sensitivity to such latent attributes of a model, resulting in achieving CF and GF simultaneously.

2 Constructing high-quality counterfactual images↩︎

The degree of counterfactual fairness (CF) can be measured by the prediction consistency between an original sample and its corresponding counterfactual (CTF) sample. For a given sample and a sensitive attribute, a CTF sample is defined as the one of which the sensitive attribute is altered while all the other attributes not caused by the sensitive attribute remain the same. However, acquiring a CTF sample for an image is challenging. For example, if the sex of a person in an image is the sensitive attribute, obtaining a CTF sample requires changing the secondary sex characteristics of the person such as beard or hairline, while preserving their identity and the other attributes, which is impossible in practice. One possible alternative is to generate a virtual face by altering such secondary sex characteristics of the given identity using a high-quality image editing method.

Several previous approaches [@kim2021counterfactual; @ramaswamy2021fair; @xu2018fairgan; @zhang2022fairness; @kocaoglu2017causalgan] have attempted to generate CTF images by VAE or GAN-based editing methods. However, they have struggled with low image quality or unintended modifications to non-sensitive attributes, rendering them unreliable for evaluating CF. To address such issues, we employ IP2P [@brooks2023instructpix2pix], an advanced diffusion model-based image editing method. Notably, IP2P can generate high-quality CTF samples by simply adjusting the text instructions without any model retraining.

As the first step, we edit the test images of two popular facial image datasets, CelebA [@celeba] and LFW [@Huang2007a]. We choose the “sex” of a person in an image as the sensitive attribute³ and edit the sex-related visual characteristics of facial images using text prompts. We generated 720 CelebA CTF image pairs and 632 LFW CTF image pairs, where the images are selected to be balanced across groups for both target and sensitive labels. Here, we treat “blond hair” and “smiling” as the target labels for CelebA and LFW, respectively. Namely, for example, the CelebA CTF image pairs have a balanced group of <female, non-blond hair>, …, and so on. 1 and 4 show examples of generated CTF images together with the originals. Hyperparameter settings are reported in 10.1. Note that while we adopt the “sex” attribute, our generation process is attribute-agnostic (e.g., age or skin color can be also used in place of sex) as illustrated in 5.

Image filtering. Despite the high quality of IP2P, low-quality CTF images can still be generated. To address this, we employed five human annotators to filter the images, i.e., each image pair was annotated as either “correct” or “incorrect”. To ensure objective and precise annotation criteria, we created guidelines as follows. Initially, we compiled a list of 20 masculine and feminine visual features using GPT-4o and with guidance from experts specialized in fairness, selected nine key facial attributes representing sex-related visual characteristics: facial hair, Adam’s apple, skin texture, jawline, chin shape, brow ridge, cheekbone prominence, lip fullness, and hairline. These attributes were used to establish the criteria for evaluating correct CTF samples. One notable issue is that most of the feminine-like images in CelebA and LFW datasets include makeups (for instance, many female celebrities in the CelebA dataset appear to be wearing makeup) and the IP2P model is biased towards removing makeup when altering feminine features. To prevent images from being filtered out solely due to changes in makeup, we additionally included makeup in the set of key attributes, even though it is not a sex characteristic. Finally, the guidelines were created based on these ten attributes, providing some criteria for correct CTF samples, such as whether the change of some of the ten attributes was accurate and whether other facial characteristics remained consistent with the original image. Using these guidelines, we filtered out pairs receiving two or fewer “correct” votes, resulting in 230 and 144 images for CelebA and LFW, respectively. More details about the human annotating interface are in 10.2, and additional information on the newly created dataset can be found in 10.4.

\begin{table}{r}{0.4}

	Sensitive	Non-Sensitive
	96.52	95.98
	98.61	93.75

\vskip

-0.1in \end{table}

Reliability check. We further verify the quality of our datasets by additional five human annotators, distinct from those participated in the filtering process. Those annotators evaluate only the images that remained after the filtering, based on two criteria: (1) whether the sensitive attribute was correctly changed and (2) whether the other non-sensitive attributes were preserved. The annotators evaluated the images for the sensitive attribute, “sex” and three non-sensitive attributes, “blond hair”, “gray hair”, and “smiling”; we chose these three because other attributes can be subjective (e.g., “big nose”) [@wu2023consistency] or had already been filtered (e.g., “wearing hat”). Details of the annotating interface provided to the five annotators are in 10.2. Based on the majority vote, we compute the percentage of CTF samples which met each of the two criteria, i.e., the accuracies for whether the sensitive and non-sensitive attributes are correctly altered and preserved. ¿tbl:table:double-check? displays the values for CelebA-CF and LFW-CF. The non-sensitive accuracy is averaged across three non-sensitive attributes. The results demonstrate that our CTF samples almost meet the two CTF criteria, suggesting that our datasets can be reliably utilized to evaluate CF.

Ethical considerations. In our study, we use the term “sex”, not “gender”, to represent the sensitive attribute with biological traits, because terms such as “gender” might imply associations with social identities, potentially raising some ethical issues. We also specifically choose ten perceived facial attributes as the visual features representing the biological sex in facial images. We believe that these considerations help alleviate various normative harms that arise from dichotomizing gender, which refers to social identity. However, despite our efforts, the sex-related visual characteristics are complex and intertwined, making it challenging to fully represent with a binary label. Thus, we urge practitioners to use our datasets with these considerations in mind.

3 Primitive study on the relationship between CF and GF↩︎

3.1 Experimental setup↩︎

We consider the image classification task where each data sample consists of an input image \(X\), a class attribute \(Y \in \mathcal{Y}=\{0, \cdots, |\mathcal{Y}|-1\}\) and a sensitive attribute \(A \in \{0, 1\}\), e.g., sex.

3.1.0.1 Metrics.

We measure three metrics for CF, GF, and classification accuracy. Firstly, we describe the metric for CF. A classifier satisfies CF when the predictions for the original sample and its counterfactual (CTF) sample are the same for every sample \(x\) and sensitive attribute \(a\), i.e., \(P(\widehat{Y}\text{\,\texttt{=}\,}y|X\text{\,\texttt{=}\,}x, A\text{\,\texttt{=}\,}a) =P(\widehat{Y}_{A\leftarrow a'}\text{\,\texttt{=}\,}y|X\text{\,\texttt{=}\,}x, A\text{\,\texttt{=}\,}a)\), where \(\widehat{Y}_{A\leftarrow a'}\) represents the prediction for a counterfactual sample intervened on \(A\) with \(a'\) (e.g., changing female to male). We quantify the degree of violence with respect to CF using counterfactual disparity (CD):

\[\begin{align} \text{ \boldsymbol{Counterfactual Fairness:} Counterfactual Disparity \boldsymbol{(CD)}}\triangleq\mathbb{E}_{x,a} \big[P\big(\mathbb{1} \{\widehat{Y}_{A\leftarrow a'} \neq \widehat{Y}\}|x,a\big)\big]. \label{eq:cd} \end{align}\tag{1}\]

Secondly, we adopt equalized odds (EO) as our notion for GF. If a predictor \(\widehat{Y}\) and the sensitive attribute \(A\) are conditionally independent given the true class attribute \(Y\), the predictor satisfies EO; namely, EO holds when \(P(\widehat{Y}\text{\,\texttt{=}\,}y'|A\text{\,\texttt{=}\,}0,Y \text{\,\texttt{=}\,}y) =P(\widehat{Y}\text{\,\texttt{=}\,}y'|A\text{\,\texttt{=}\,}1,Y\text{\,\texttt{=}\,}y)\). From the definition, we can capture the degree of violence with respect to GF with the disparity of EO (DEO):

\[\begin{align} \text{Disparity of EO \boldsymbol{(DEO)}}\triangleq \max_{y,y'\in\mathcal{Y}} \big |P&(\widehat{Y}\text{\,\texttt{=}\,}y' |A\text{\,\texttt{=}\,}0, Y\text{\,\texttt{=}\,}y)\text{\texttt{-}}P(\widehat{Y}\text{\,\texttt{=}\,}y' |A\text{\,\texttt{=}\,}1,Y\text{\,\texttt{=}\,}y)\big|. \label{eq:deo} \end{align}\tag{2}\]

We note that we empirically compute CD and DEO, defined in 1 and (2 ), using our benchmark datasets and the original test datasets of CelebA and LFW, respectively. Additionally, [@pinto2024matrix] propose several other metrics to evaluate CF, and accordingly, we conducted an additional evaluation based on these metrics, with results provided in 14.5.

3.1.0.2 Baseline methods.

We evaluate a model trained with the vanilla cross-entropy loss (denoted as “Scratch”) and two CF-aware training methods, Scratch+aug and counterfactual pairing (CP). Scratch+aug is a Scratch method using an augmented training dataset with counterfactual samples [@garg2019counterfactual], and CP [@russell2017worlds] adopts a regularization term that promotes pairs of original and its CTF sample to obtain the same prediction (see 4 for the formal definition). Note both methods need counterfactual samples for training, and hence, we use the samples generated via IP2P with the same prompts used in 2 without any filtering process to obtain results for them. For a comprehensive study, we additionally evaluate two individual fairness-aware methods, SenSeI [@yurochkin2020sensei] and LASSI [@peychev2022latent], of which goals are analogous to CF in aiming to make a model robust to perturbation of the sensitive attribute. More details are described in 11.3.

3.1.0.3 Model selection.

Due to the accuracy-fairness trade-off [@dutta2020there], appropriate model selection is important for fair evaluation. We explore varying hyperparameters and select the best model that shows the lowest CD (1 ) for the held-out validation set while achieving a lower bound of the accuracy⁴.

8pt

3.2 Performance comparison↩︎

1 shows accuracy, CD, and DEO for Scratch and four baseline methods. Note that we omit the result of LASSI on LFW because the number of samples in LFW is not enough to train the Glow model [@kingma2018glow], which is the main component of LASSI. From the table, CF-aware and individual fairness-aware methods are mostly effective in mitigating CD, when compared to Scratch. However, it does not necessarily lead to improvements in DEO. Especially, while CP significantly improves CD for both datasets, it exacerbates DEO compared to Scratch. Namely, contrary to the previous studies [@anthis2023causal; @rosenblatt2023counterfactual] showing that CF implies GF on tabular datasets, our observation shows that CF does not always imply GF on image datasets. In the following section, we theoretically investigate why the previous observations may not hold on images.

4 Theoritical analysis on the relationship between CF and EO for images↩︎

4.1 Structural Causal Model (SCM) for images↩︎

Structural Causal Models (SCMs) are represented as directed acyclic graphs satisfying the conditions specified in [@pearl2000models]. In these models, nodes and edges indicate variables and their causal relationships within the data-generating process. As studied in previous works [@dash2022evaluating; @kocaoglu2017causalgan], the nodes of an SCM for image can be categorized into three parts. As shown in 2, the blue, gray and yellow nodes indicate latent attributes, e.g., \(Y\) or \(A\), components of the image influenced by these attributes, e.g., \(X_Y\) or \(X_A\), and the whole image \(X\), respectively. Taking an SCM for facial images as an example, we can interpret these nodes as follows: latent attributes such as hair color or sex, facial components like the hair or an Adam’s apple in a facial image, and an entire face. Note that the blue region in the figure describes that various causal relationships among latent attributes can exist ⁵. Furthermore, although an image SCM may contain additional latent attributes, we simplify our focus to only include the class and sensitive attribute, \(Y\) and \(A\), and a third-party attribute, \(G\), which may correlate with the sensitive attribute \(A\).

4.2 Theoretical analysis↩︎

According to the Markov assumption of SCM [@pearl2000models], if there are no unblocked paths between two variables in an SCM (i.e., they are d-separated), the variables are statistically independent. Utilizing this property, [@anthis2023causal] demonstrated that CF implies several GF notions, including Equalized Odds (EO) [@hardt2016equality], under the specific condition on SCMs such as no backdoor path from the sensitive attribute \(A\) to the image \(X\) exists (Theorem 2 of [@anthis2023causal]). Moreover, the authors empirically show that these conditions would hold on some tabular datasets.

However, we argue that these conditions would not hold for image datasets due to a fundamental difference in what sensitive attributes represent in an image. Specifically, tabular datasets typically consist of recorded information by subjects, where sensitive attributes such as sex or race usually represent immutable genetic information; hence they are not caused by other attributes and cause all attributes correlated with the sensitive attributes. In contrast, sensitive attributes in image datasets indicate visual characteristics that can change and be influenced by some other attributes, such as the attribute \(G\). For example, in a facial image dataset, attributes like hair length or accessories might be highly correlated with, but not caused by, secondary sex characteristics such as beard. Namely, a backdoor path from the sensitive attribute \(X\) through the attribute \(G\) could exist, thereby breaking the connection between CF and GF discovered in previous studies.

Our theoretical result specifies the relationship between CF and GF (especially for EO) with \(G\):

The proof of the theorem is in 8. Note that when we take the maximum over \((y, y')\) on both sides of the inequality in 1, the left-hand side of the inequality becomes identical to DEO (2 ). Therefore, the theorem implies that DEO is upper bounded by the maximum of \(d_{\boldsymbol{\theta},X_Y}(X_G,X_G')\) (in which the maximum is over \(X_G\), \(X_G'\), \(y\), \(y'\)), which measures the sensitivity of the model with respect to \(G\). In other words, the theorem shows that when a counterfactually fair model is sensitive to \(X_G\) (i.e., when \(\max d_{\boldsymbol{\theta}, X_Y}(X_G, X'_G)\) is large), the model may result in having high DEO in the worst-case.

1 elucidates why CF-aware methods in 1 often fail to mitigate DEO despite significant improvements in CD. Namely, if the attribute \(G\) assumed in 1 exists on CelebA and LFW, DEO for the classifiers trained by CF-aware methods can worsen depending on their robustness to \(G\). This will be empirically demonstrated using “hair length” as \(G\) in 5.2, together with the results using a controllable synthetic dataset. Furthermore, 1 suggests that we can re-establish the relationship between two notions by making counterfactually fair classifiers non-sensitive to \(G\). In the following section, we introduce a method to promote a classifier not to depend on \(G\) while achieving CF.

5 Empirical analyses on the effect of \(G\) to CF and GF↩︎

5.1 Counterfactual Knowledge Distillation(CKD)↩︎

Motivated by 1, we propose a baseline fair-training method to achieve both CF and GF. Conceptually, if we can reduce the dependency between the latent attribute \(G\) described in Theorem 1 and the prediction of a CF-aware trained model, we can expect that the model will achieve CF and GF simultaneously. Therefore, we improve the CF-aware method, CP [@russell2017worlds] (best-performing in 1), such that the dependency to the attribute \(G\) is reduced. We first describe the CP regularization (which is used along with the cross-entropy loss) for given counterfactual samples \(\mathcal{D'} = \{x_{i,A\leftarrow a_i'}\}_{i=1}^N\) corresponding to the original training dataset \(\mathcal{D}\): \[\begin{align} \label{eq:cp} \mathcal{L}_{\text{CP}}&(\theta,\mathcal{D} \cup\mathcal{D'}) := \frac{1}{N} \sum_{i=1}^N \lVert f(\boldsymbol{\theta}, x_i) - f(\boldsymbol{\theta}, x_{i,A\leftarrow a_i'})\rVert_2^2, \end{align}\tag{4}\]

in which \(f(\boldsymbol{\theta}, x)\) is a representation vector of input \(x\) produced by a classifier \(\boldsymbol{\theta}\), such as logit or feature vector. Note that the images \(x\) and \(x_{A\leftarrow a'}\) differ only in their components corresponding to the sensitive attribute \(A\) and the attributes caused by the sensitive attribute \(A\). Hence, although the CP regularization works well for achieving CF, it does not ensure the model does not rely on the attribute \(G\), potentially leading to worse DEO as argued in the previous section.

Recent studies [@jung2021mfd; @song2020denoising; @zi2021revisiting] have shown that the robustness of a teacher model can be transferred into a student model through knowledge distillation (KD) [@hinton2015distilling]. To that end, we first assume a teacher model that is robust to the attribute \(G\) is available. Then, our idea is to apply both KD and CP regularization to train our student model, which leads to a simple yet effective approach, dubbed as Counterfactual Knowledge Distillation(CKD). Specifically, CKD employs averaged representation vectors of original and counterfactual samples extracted by the teacher model \(\boldsymbol{\theta}^T\) as target vectors. Then, representation vectors of both samples from the student model \(\boldsymbol{\theta}\) are enforced to follow the target vectors. Namely, the distillation term of CKD is defined as follows: \[\begin{align} \label{eq:ckd} \mathcal{L}&_{CKD}(\boldsymbol{\theta},\mathcal{D}\cup\mathcal{D}') := \nonumber \frac{1}{2N}\sum_i^N \bigg( \lVert f(\boldsymbol{\theta},x_i)-f^T_i \rVert_2^2 + \lVert f(\boldsymbol{\theta}, x_{i, A\leftarrow a_i'})-f^T_i \rVert_2^2\bigg), \nonumber\\ &\text{in which} \;\;f^T_i = \frac{1}{2} \big(f(\boldsymbol{\theta}^T, x_i) + f(\boldsymbol{\theta}^T, x_{i, A\leftarrow a_i'})\big)\;\; \text{is the target vector for the i-th pair.} \end{align}\tag{5}\] Note that our distillation terms have both effects of KD and CP by promoting both representations of original and counterfactual samples to be aligned with the target vectors \(f_i^T\) produced by the teacher model. Therefore, based on 1, we can deduce that the CKD regularization encourages the model to achieve both CF (by the CP effect) and EO-based GF (by the KD effect that distills the robustness of the teacher with respect to the attribute \(G\)). In addition, we optionally incorporate CP regularization into our objective to further mitigate CD. The final objective of our method (which we again dub as CKD for brevity) is as follows: \[\begin{align} \label{eq:ckd95full} \underset{\boldsymbol{\theta}}{\min}\;&\mathcal{L}_{\text{CE}}(\boldsymbol{\theta},\mathcal{D}) +\mu\mathcal{L}_{\text{CKD}} (\boldsymbol{\theta}, \mathcal{D}\cup\mathcal{D}')+ \lambda\mathcal{L}_{\text{CP}} (\boldsymbol{\theta}, \mathcal{D}\cup \mathcal{D}'), \end{align}\tag{6}\] in which \(\mu\) and \(\lambda\) are controllable hyperparameters for the CKD and CP regularization, respectively.

While we assumed above the availability of a teacher model that is robust to the attribute \(G\), obtaining such a model could be challenging in practice. Empirically, we observe that vanilla-trained models (referred to as “Scratch” models) less depend on the attribute \(G\) than CP-trained ones (see 3 and ¿tbl:table:CD95G? for more details). We presume that this is because the attributes \(A\) and \(G\) behave as “shortcut” features for classifying the class attribute \(Y\), i.e., they are easy-to-learn discriminatory features. As observed by [@scimeca2022wcst-ml], making a model blind to a certain shortcut feature causes it to rely more heavily on the other shortcut features. In our case, CP-trained models are trained to be invariant to the sensitive attribute \(A\), resulting in a greater dependence on the attribute \(G\) compared to the Scratch models. Thus, unless otherwise specified, we will assume the vanilla-trained model is relatively robust to the attribute \(G\) since it would mostly rely on the sensitive attribute \(A\), hence, we use it as the teacher model.

5.2 Impact of robustness to \(G\) on CF and GF↩︎

We empirically validate our theoretical result and CKD on both a newly introduced synthetic dataset (CIFAR-10B) and a real-world dataset (CelebA) by analyzing CF, GF, and the robustness with respect to the attribute \(G\) described in Theorem 1. We thus introduce a new metric for the robustness to the attribute \(G\), the rate of flipped predictions (RFP) :

\[\begin{align} \text{RFP}\triangleq\mathbb{E}_{x,x'} \big[P\big(\mathbb{1} \{\widehat{Y} \neq \widehat{Y}'\}|x,x'\big)\big]. \label{eq:CD-G} \end{align}\tag{7}\] in which \(x\) is an original image, \(x'\) is its corresponding image with the attribute \(G\) flipped. \(\widehat{Y}\) and \(\widehat{Y}'\) refer to the predicted label by the trained model given \(x\) and \(x'\), respectively. RFP quantifies the amount of flipped predictions when the attribute \(G\) is altered. For example, if a model shows the same prediction after changing the attribute \(G\), its RFP becomes 0%.

5.2.0.1 CIFAR-10B, a controllable synthetic dataset.

We construct the CIFAR-10B dataset, where we can perfectly control the degree of bias with respect to the attribute \(G\) while the target label is biased towards the sensitive attribute \(A\). We make binary class labels from the 10 classes of CIFAR-10 (0-4 and 5-9 classes). We set the attributes \(A\) and \(G\) in 1 with the presence of Gaussian and Contrast noise, respectively. We also set a fixed ratio of 0.8 and a controllable ratio \(\alpha\), which represent skewnesses among (\(Y\), \(A\)) and (\(A\), \(G\)), respectively; the former ratio is the spurious correlation between \(Y\) and \(A\), and the latter one is the correlation between \(A\) and \(G\). We then construct the CIFAR-10B dataset by randomly injecting Gaussian or Contrast noise to each CIFAR-10 image at given ratios, as illustrated in 10. Unless otherwise noted, we set \(\alpha\) as 0.8.

We train models with Scratch, CP, and CKD on CIFAR-10B by adjusting \(\alpha\) from 0.5 (i.e., \(A\) and \(G\) are decorrelated) to 0.9 at intervals of 0.1. 3 shows CD, DEO, and RFP metric values for each method. The figures indicate that while CP and our CKD consistently achieve CF, CP fails to meet GF as \(\alpha\) increases, potentially due to higher RFP. Furthermore, RFP of Scratch is lower than that of CP when \(\alpha\) is greater than 0.7. This empirically justifies the use of vanilla-trained models as teacher models robust to \(G\). By using these teacher models, CKD significantly improves DEO regardless of the value of \(\alpha\) by maintaining the robustness to \(G\), i.e., low RFP, supporting the result of 1.

\begin{table}{r}{0.4}

	CelebA
Method	CD \(\downarrow\)	DEO \(\downarrow\)	RFP \(\downarrow\)
Scratch	10.26	47.10	15.27
CP	2.53	51.01	20.37
	4.44	13.23	10.85

\vskip

-0.2in \end{table}

5.2.0.2 Impact of \(G\) manipulation on CelebA.

We assume “hair length” as \(G\) for facial image datasets, e.g., CelebA because the hair length \(G\) can be highly correlated with, but not caused by, the sex \(A\). To compute RFP for the hair length attribute, we manipulate the hair length of CelebA test images using SDEdit [@meng2021sdedit]. More details and generated examples can be found in 12.1. Using the hair length-edited images, we report RFP in ¿tbl:table:CD95G?, together with DEO and CD. The results share the same trend as the CIFAR-10B results, i.e., CP shows worse DEO and RFP than Scratch but better CD, whereas CKD shows the best DEO and RFP, despite a slight increase of CD.

5.3 Impact of the robustness to \(G\) of the teacher model on CKD↩︎

r0.45

3pt

Our CKD requires a robust teacher model with respect to the attribute \(G\) to distill the robustness to the target model. To analyze the impact of the robustness of the teacher model, we compare various teacher models with different dependencies on the attribute \(G\) using CIFAR-10B. We consider four teacher models, ordered by robustness to the attribute \(G\): CP (\(\boldsymbol{\theta}^T_\text{CP}\)), Scratch (\(\boldsymbol{\theta}^T_\text{Scratch}\)), and CKD model with a Scratch teacher (\(\boldsymbol{\theta}^T_\text{CKD}\)), and a de-biased model trained on CIFAR-10B balanced for \(G\), i.e., \(\alpha=0.5\), (\(\boldsymbol{\theta}^T_\text{De-biased}\)). Using these teacher models, we report DEO, CD, and RFP of CKDs on the CIFAR-10B dataset in 2. We observe that the degree of robustness to the attribute \(G\) of the teacher model (i.e., RFP\(^T\)) highly correlates to DEO. It is because as the teacher model becomes more robust to \(G\), RFP of the target model gets lower, finally leading to a lower DEO while maintaining fair CD. Namely, these results support our theoretical result again.

6 Full comparisons of fair-training methods on image classification↩︎

Finally, we evaluate the existing fair-training methods focusing on group fairness (GF) and counterfactual fairness (CF) on CelebA and LFW, together with CIFAR-10B for image classification tasks. We emphasize that only CelebA-CF and LFW-CF have counterfactual images of the real-world images; hence, we measure a CF metric, i.e., Counterfactual Disparity (CD) (1 ), using our datasets. Along with the CF-aware methods, such as CP [@russell2017worlds] and CKD, we report the GF-aware methods including SS [@idrissi2022simple], RW [@kamiran2012data], COV [@zafar2017fairness], MFD [@jung2021mfd], and LBC [@jiang2020identifying]. In addition, we report the naive combinations of GF-aware and CF-aware methods, e.g., training GF-aware method with the augmented training dataset with counterfactual images generated by IP2P [@brooks2023instructpix2pix] (denoted as “+aug”) and combinations of the GF-aware methods and the CP regularization (4 ) (denoted as “+CP”). The hyperparameters for all methods besides the GF-aware methods are selected using the same protocol in 3, and ones for the GF-aware methods are chosen based on DEO using the same lower bound of the accuracy. Implementation details are provided in 11.3.

4.5pt

3 shows the holistic evaluation of CF and GF for all the methods mentioned above on the three image classification tasks. The table shows four important observations. First, although CP (a CF-aware method) mostly performs the best on CD, it even shows worse DEO than Scratch. We theoretically and empirically discussed the reason in 4 and 5. Second, the GF-aware methods are effective in improving DEO but have a minimal impact on CD. This suggests that the faithfulness assumption for SCM may not hold, which will be discussed in more details in 13. Third, the naive combinations of GF-aware and CF-aware methods exhibit much better CD than using the GF-aware methods alone. Additionally, DEOs achieved by the naively combined methods tend to be improved since their training datasets are balanced over the sensitive attributes by incorporating generated samples into the original training datasets. Lastly, we found that CKD shows the best DEO for every evaluation dataset. It shows that if we can train a CF-aware model by reducing the dependency on \(G\), we can achieve both CF and GF even on the image classification task. We additionally conduct an ablation study on CKD by removing CP (i.e., CKD(\(\lambda=0\))) from 6 . We observe that this ablated version achieves suboptimal performances than CKD. This suggests that adding the CP regularization term to the CKD objective function can be helpful to improve both CD and DEO.

7 Concluding remarks↩︎

This paper offers carefully crafted benchmark datasets for evaluating the counterfactual fairness (CF) of image classification methods. Since obtaining true counterfactual images is impossible in practice, we employ a high-quality image editing technique to generate counterfactual images of the given images. We construct two facial image benchmarks, CelebA-CF and LFW-CF, by carefully filtering out and verifying the generated counterfactual images by human annotators. Our datasets relax the constraints of the impossibility of evaluating CF in image classification. Using our datasets, we also provide theoretical and empirical results showing that CF may not imply GF, contradictory to the studies conducted on tabular datasets. We elucidate this phenomenon by the presence of the third-party attribute highly correlated with, but not caused by, the sensitive attribute. From this finding, we propose a simple baseline method, CKD, to achieve CF and GF simultaneously. Our extensive experimental results on both GF and CF metrics show that when reducing the reliance on the attribute (e.g., by using CKD), improving the CF metric leads to a significant improvement in the GF metric. By providing our benchmarks and various analyses, we believe that our findings bridge CF and GF in image classification, contributing to the development of fair and robust image recognition systems.

Acknowledgments↩︎

This work was supported in part by the National Research Foundation of Korea (NRF) grant [No.2021R1A2C2007884] and by Institute of Information & communications Technology Planning & Evaluation (IITP) grants [RS-2021-II211343, RS-2021-II212068, RS-2022-II220113, RS-2022-II220959] funded by the Korean government (MSIT). It was also supported by AOARD Grant No. FA2386-23-1-4079, SNU-Naver Hyperscale AI Center, and Hyundai Motor Chung Mong-Koo Foundation.

Checklist↩︎

The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default to , , or . You are strongly encouraged to include a justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:

• Did you include the license to the code and datasets?

• Did you include the license to the code and datasets?

• Did you include the license to the code and datasets?

Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

1. For all authors...

   1. Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

   2. Did you describe the limitations of your work?

   3. Did you discuss any potential negative societal impacts of your work?

   4. Have you read the ethics review guidelines and ensured that your paper conforms to them?

2. If you are including theoretical results...

   1. Did you state the full set of assumptions of all theoretical results?

   2. Did you include complete proofs of all theoretical results?

3. If you ran experiments (e.g. for benchmarks)...

   1. Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

   2. Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

   3. Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

   4. Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

   1. If your work uses existing assets, did you cite the creators?

   2. Did you mention the license of the assets?

   3. Did you include any new assets either in the supplemental material or as a URL?

   4. Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

   5. Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

5. If you used crowdsourcing or conducted research with human subjects...

   1. Did you include the full text of instructions given to participants and screenshots, if applicable?

   2. Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

   3. Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

8 Proof of 1↩︎

We start from LHS in equation 3 : \[\begin{align} \big|P(\widehat{Y}=y'|A=0,&Y=y)-P(\widehat{Y}=y'|A=1,Y=y)\big| \nonumber \\ =\bigg|\sum_{X_A,X_Y,X_G}P(\widehat{Y}=&y'|X_A,X_Y,X_G,A=0,Y=y)P(X_A,X_Y,X_G|A=0,Y=y) \nonumber \\ &-P(\widehat{Y}=y'|X_A,X_Y,X_G,A=1,Y=y)P(X_A,X_Y,X_G|A=1,Y=y)\bigg| \end{align}\] \[\begin{align} =\bigg|\sum_{X_A,X_Y,X_G}P(\widehat{Y}=y'|X_Y,X_G)P(X_A,&X_Y,X_G|A=0,Y=y)\nonumber\\ &-P(\widehat{Y}=y'|X_Y,X_G)P(X_A,X_Y,X_G|A=1,Y=y)\bigg| \\ =\bigg|\sum_{X_Y,X_G}P(\widehat{Y}=y'|X_Y,X_G)\big(P(X_Y|X_{G}&,A=0,Y=y)P(X_G|A=0,Y=y) \nonumber \\ &-P(X_Y|X_G,A=1,Y=y)P(X_G|A=1,Y=y)\big)\bigg| \end{align}\] \[\begin{align} =\bigg|\sum_{X_Y,X_G}P(\widehat{Y}=y'|X_Y,X_G)\bigg(&P(X_Y|Y=y)P(X_G|A=0,Y=y)-P(X_Y|Y=y)P(X_G|A=1,Y=y)\bigg)\bigg| \\ =\bigg|\sum_{X_Y}P(X_Y|Y=y)\Big(\sum_{X_G}P(\widehat{Y}&=y'|X_Y,X_G)P(X_G|A=0,Y=y)\\ &-\sum_{X_G'}P(\widehat{Y}=y'|X_Y,X_G')P(X_G'|A=1,Y=y)\Big)\bigg|. \end{align}\] Note the first and third equalities are driven by Bayes’ theorem, the second one is from the independence between \(\widehat{Y}\) and \(X_G,A\) conditioned on \(X_Y, X_G\) based on the Markov properties of SCM, and the fourth one is due to the independence between \(X_Y\) and \(X_G, A\) conditioned on \(Y\). We denote a coupling between the two distributions \(P(X_G|A=0,Y)\) and \(P(X_G'|A=1,Y)\) as \(\Pi(X_G,X_G')\), then we have: \[\begin{align} & \big|P(\widehat{Y}=y'|A=0,Y=y)-P(\widehat{Y}=y'|A=1,Y=y)\big| \nonumber \\ & =\bigg|\sum_{X_Y}P(X_Y|Y=y)\Big(\sum_{X_G,X_G'}\Pi(X_G,X_G') \big( P(\widehat{Y}=y'|X_Y,X_G)-P(\widehat{Y}=y'|X_Y,X_G')\big)\Big)\bigg|. \\ & \leq\sum_{X_Y}P(X_Y|Y=y)\bigg(\sum_{X_G,X_G'}\Pi(X_G,X_G') \Big|P(\widehat{Y}=y'|X_Y,X_G)-P(\widehat{Y}=y'|X_Y,X_G')\Big|\bigg) \label{eq:jensen}\\ & =\sum_{X_Y}P(X_Y|Y=y)\sum_{X_G,X_G'}\Pi(X_G,X_G') d_{\boldsymbol{\theta},X_Y}(X_G,X_G') \end{align}\tag{8}\] where the sample distance is denoted as \(d_{\boldsymbol{\theta},X_Y}(X_G,X_G') =\big|P(\widehat{Y}=y'|X_Y,X_G)-P(\widehat{Y}=y'|X_Y,X_G')\big|\). The inequality in Equation 8 is driven by Jensen’s inequality.

9 Limitations and societal impacts↩︎

While our datasets and analyses reveal the relationship between CF and GF in image classification, we clarify our study’s limitations. First of all, our study uses sex as a sensitive attribute based on visually perceived biological traits. However, as mentioned in 2, this simplification does not capture the full spectrum of sexual traits, which is more complex and nuanced. Therefore, we emphasize again that practitioners should use our data with these considerations in mind; they should not utilize our datasets for gender categorization but rather for investigating the unfairness in terms of CF and GF and enhancing fairness in AI systems. Second, our data generation process relies on IP2P to create CTF samples. We tried to mitigate the potential bias problem during the data generation process through the sophisticated human filtering process, but our data samples could be affected by the unintended bias of IP2P. Third, while we assume the structural causal model (SCM) for images as 2, specifying an SCM in the real world is often infeasible. This difficulty also makes it challenging to apply some of our experiments, such as analyzing the attribute \(G\) or using a robust teacher model to \(G\). However, we addressed these challenges to some extent by proposing a systematic method to investigate \(G\) (12.2) and analyzing the robustness of vanilla teacher models (5.3).

Despite the limitations, we believe that our work makes significant contributions through our curated image datasets and extensive analyses to the evolving field that addresses relationships among different fairness notions.

10 Datasets: CelebA-CF and LFW-CF↩︎

We provide access to our newly created dataset, i.e., CelebA-CF and LFW-CF, through the following link: CelebA-CF (https://figshare.com/s/62b6f7f69d0eab9c3c71) , LFW-CF (https://figshare.com/s/39f2daac58148e10e5fe)

10.1 Hyperparameters for IP2P image editing↩︎

We set the resolution of generated images to 256\(\times\)​256 and the denoising step to 50. Furthermore, we set the Image-CFG weight to 1.8 and the Text-CFG weight to 7.5. These two hyperparameters are the guidance scales to control how the generated images closely resemble the input image or are intensely edited. To alter the sensitive attribute of facial images, we use the prompts of “turn the woman into a man” for female images and “turn the man into a woman” for male images.

10.2 Human annotations↩︎

After generating CTF images using IP2P, we filtered them through five annotators to construct high-quality CTF samples, namely CelebA-CF and LFW-CF. Before evaluating the created CTF samples, the guidelines are given to the annotators, as presented in 7. To establish the guidelines, we extracted 20 facial attributes of secondary sex characteristics using Chat-GPT and then, with guidance from experts specialized in fairness, selected 9 key facial attributes (facial hair, Adam’s apple, skin texture, jawline, chin shape, brow ridge, cheekbone prominence, lip fullness, and hairline). The guidelines instruct human annotators to filter out counterfactual samples based on these attributes (including considerations for the presence of makeup). Subsequently, given the original image and generated CTF image pair, annotators assess whether the generated image is correctly created based on the instructions. 6 shows the interfaces of the annotation task for image filtering.

We further verify the reliability of our datasets with another five human annotators, different from those who participated in the previous filtering process, and report the result in ¿tbl:table:double-check?. For more objective annotation, we show 8 example images to the annotators before the labeling task, which are randomly sampled the same number of times for each attribute value from test image datasets. Then, the annotators label CelebA-CF and LFW-CF for 4 attributes, i.e., “sex”, “blond hair”, “gray hair”, and “smiling” in order. Specifically, the annotators evaluate whether the sensitive attribute was correctly altered and the non-sensitive attributes were maintained for a generated image. Similar to the image filtering task, we provide the annotators with the set of 10 sex-related facial attributes for objective and accurate labeling. 8 shows the interfaces of the annotation task for this reliability check. We provide the attribute values originally annotated for the original image datasets on the screen together for annotators to refer to as a guide for their annotating tasks. Although we focus on visually perceived sex traits, we use the terms Male and Female for convenience in the annotation interface.

We note that the wage paid to each participant is 18 USD per hour, resulting in a total expenditure of 360 USD on participant compensation.

10.3 License information of assets employed in this study↩︎

• CelebA [@celeba] was made available for academic research purposes without a formal license. The dataset can be downloaded at https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html.

• LFW [@Huang2007a] is publicly available for research purposes. There is also no formal license and further information is reported at https://vis-www.cs.umass.edu/lfw/.

• InstructPix2Pix (IP2P) [@brooks2023instructpix2pix] is licensed under the MIT license and is available at https://github.com/timothybrooks/instruct-pix2pix.

• IP2P further builds upon stable-diffusion-v1-5 that is released under CreativeML- Open-RAIL-M License.

10.4 Further information of the new dataset.↩︎

CelebA-CF and LFW-CF are based on real facial image datasets, such as CelebA and LFW, which include 40 attribute annotations. We note that because the original datasets have an imbalance between some pairs of two attributes, our datasets also possess a different skewness between the sensitive attribute and other attributes. For example, in CelebA and CelebA-CF, most images with the blond hair attribute are female, and most males do not have blond hair. We present the skewness values between the sensitive attribute and other non-sensitive attributes in CelebA-CF in ¿tbl:table:attriabute-imbalance?. Additionally, we displayed the group-specific failure rate identified through the filtering process in ¿tbl:table:failure-rate?.

\begin{table}[t!]

2.0pt

Attribute	Hair Length	Bangs	Wearing Hat	Brown Hair	Pale Skin	Big Lips	Mouth Slightly Open	Smiling	Wavy Hair
Skewness	0.798	0.907	0.974	0.820	0.980	0.815	0.265	0.335	0.833

\vskip

-0.1in \end{table}

\begin{table}[t!]

	CelebA-CF	LFW-CF
Female	0.57	0.70
Male	0.80	0.69

\vskip

-0.1in \end{table}

11 Implementation details↩︎

11.1 Details on training datasets↩︎

For CelebA, we utilize the official train-validation-test split [@celeba]. For LFW, we also use the official train-test split [@Huang2007a] and then divide the training data into a training and a validation set, with a ratio of 80:20.

CIFAR-10B is a modified dataset from CIFAR-10, as described in 5.2. We modify CIFAR-10 into a binary classification task by dividing the original 10 classes into two classes (classes 0-4 and 5-9). To introduce a fairness issue, we set the sensitive attribute \(A\) as the presence of Gaussian noise and skew the dataset by randomly injecting the noise into 20% and 80% of the data in class 0 and class 1, respectively. Additionally, we introduce Contrast noise for the attribute \(G\). Using the skew-ratio \(\alpha\), we create a statistical correlation between \(A\) and \(G\) by adding the noise into \(100\times\alpha\)% of the data samples with \(A=1\) and \(100\times(1-\alpha)\)% of the data samples with \(A=0\). Unless otherwise noted, we set \(\alpha\) to 0.8. We partition the dataset into train-validation-test sets with a ratio of 64:16:20, respectively, maintaining consistent values for the two skewness ratios (i.e., skewness between \(A\) and \(Y\), \(G\) and \(A\)) across all sets during our experiments.

11.2 Compute Infrastructure and optimization↩︎

Our all experiments including the dataset construction and performance comparison of existing methods and CKD were conducted using AMD Ryzen Threadripper PRO 3975WX CPUs and NVIDIA RTX A5000 GPUs. Our dataset generation was parallelized using 8 GPUs and took 2 days to complete. Training time for models used in the experiments for performance comparison ranges between 12 to 24 hours depending on the dataset and method used.

For CIFAR-10B, we use ResNet56 models with the Adam optimizer for 50 epochs. We set the mini-batch size and learning rate as 128 and 0.001, respectively. Because the skewness between \(G\) and \(A\) in CIFAR-10B test datasets varies, we compute the balanced CD over both the target class and the sensitive attribute for the consistent metric. For CelebA and LFW, we train ResNet18 models with the AdamW optimizer. We use the epoch size of 70 and 50 for each dataset, and set the mini-batch size, learning rate, and weight decay as 128, 0.001, and 1e-4, respectively. We use identical hyperparameters regarding the optimization for all methods. All results are averaged over results from four different random seeds.

11.3 Implementation details of baselines and CKD↩︎

CF-aware methods. Scratch (+Aug) [@garg2019counterfactual] minimizes the empirical cross-entropy loss computed using both original and counterfactual images. CP [@russell2017worlds] has a regularization term that promotes the image pairs to be the same prediction. We use logits of a neural network model as representation vectors for the CP regularization term. Since Scratch (+aug) and CP utilize CTF samples, we generated these samples using IP2P with the same prompt in 10.1 using the image-CFG of 7.5 and the Text-CFG of 2.0. We note that we do not apply any filtering process for their generated training datasets. SenSeI [@yurochkin2020sensei] uses two metrics for training: one for a pre-defined fair regularizer distance metric and the other obtained by fair metric learning. We use the same metrics as presented in their code. By generating the worst-case samples based on these metrics, we apply a fair regularization term to promote their predictions to be the same, as originally implemented. LASSI [@peychev2022latent] minimizes an objective function which is composed of the classification loss, the reconstruction loss, and the adversarial loss to learn individually fair representation. We use the official code of LASSI as it is.

GF-aware methods. LBC [@jiang2020identifying] necessitates multiple full-training iterations, alternately re-weighting each group based on the given group fairness metric and re-training. Due to its high computation budget for iterative full-training, we limit the number of epochs for each training to 5 and repeat this process 14 times. COV [@zafar2017fairness] utilizes a fairness constraint based on the covariance between the group label and the signed distance of feature vectors from the decision boundary of a classifier. We minimize the constraint-regularized objective function through gradient descent optimization, instead of directly solving its optimization problem. MFD [@jung2021mfd] employs an additional fairness-promoting regularization term based on Maximum Mean Discrepancy (MMD). For the MMD distance of the regularization term, we use the Gaussian RBF kernel with the variance parameter set as the mean of squared distance between all data points. We implemented SS [@idrissi2022simple] and RW [@kamiran2012data] identically to the original algorithm.

CF- and GF-aware methods. The combinations of GF method and the augmentation were implemented so that GF methods train a model on their own objective function using training datasets augmented by generated CTF samples. The combinations of GF methods and CP optimize the objective functions of GF methods combined by the CP regularization. The CKD regularization term (5 ) builds upon representation vectors \(f(\boldsymbol{\theta}, x)\). For this vector, we use the logits of a neural network model on LFW and CIFAR-10B. For CelebA, we utilize feature vectors from the penultimate layer of models as a representation vector since its training dataset is relatively much larger and more complex than others, leading to more fine-grained feature vectors.

Our code is available at https://github.com/sumin-yu/CKD.git.

11.4 Hyperparameter search↩︎

The range of hyperparameter search used for 1 and 3 are shown in ¿tbl:table:hyperparameters?. We utilize grid search to select hyperparameter values within a certain range. Note CKD and the combinations of GF-aware methods and CP have additional parameters \(\lambda\) for CP loss. We use the same range as CP for \(\lambda\). We also note that for LASSI, we only search the hyperparameter for an adversarial loss while maintaining other parameters as the same as used in their experiments on CelebA.

\begin{table}[th]

Method	Hyperparameter	Search range
CP [@russell2017worlds]	CP strength \(\lambda\)	\([10^{-2}, 10^2]\)
SenSeI [@yurochkin2020sensei]	Fair regularization strength \(\rho\)	\([10^{-2}, 10^2]\)
LASSI [@peychev2022latent]	Adversarial loss weight \(\lambda_2\)	\([10^{-3}, 10^{-1}]\)
COV [@zafar2017fairness]	Covariance strength \(\lambda\)	\([10^{-2}, 10^2]\)
MFD [@jung2021mfd]	MMD strength \(\lambda\)	\([10^{-1}, 10^6]\)
LBC [@jiang2020identifying]	LR for re-weights \(\eta\)	\([10^{-1}, 10^3]\)
	strength \(\mu\)	\([10^{-1}, 10^3]\)

\vskip

-0.1in \end{table}

12 Rate of Flipped Predictions (RFP)↩︎

12.1 RFP measurement on CelebA↩︎

To measure RFP on CelebA, we assume that the hair length of facial images is \(G\), i.e., it is correlated with, but not caused by, the sensitive attribute. Then, we use Stochastic Differential Editing (SDEdit) [@meng2021sdedit], an image editing method based on a diffusion model, to modify the hair length in each image. SDEdit selectively edits specific regions of a given image based on the colored stroke. Namely, SDEdit depicts the image region indicated by the stroke with the given color in the most plausible manner. By doing so, SDEdit generates realistic and faithful edited images, while preventing changes in the region not indicated by the strokes. To utilize SDEdit, we randomly select 40 samples for each group of the same target label and sensitive attribute from original samples of CelebA-CTF pairs and then we manually apply strokes on the hair of facial images for a total of 160 samples. Specifically, To extend the hair length, we applied strokes with the hair color to the areas where the hair should grow. Conversely, to shorten the hair length, we applied strokes with the background color to the areas where the hair should be removed. After this process, we utilize the official PyTorch implementation of [@meng2021sdedit] to edit images with the applied strokes. 9 shows some examples of images edited by SDEdit.

12.2 Discussion about selecting \(G\) on CelebA↩︎

As mentioned in 5.2, we intuitively chose “hair length” as \(G\) on CelebA since \(G\) is highly correlated with, but not caused by, the sensitive attribute \(A\). However, we introduce a more generalized approach for choosing \(G\) by leveraging a CP-trained model that exhibits low CD but high DEO. Specifically, using a pre-defined set of attributes, we first train a linear classifier on the top of the feature extractor from the CP-trained model for each attribute. In cases where annotations are not available, CLIP-based pseudo labels can be utilized. Based on the accuracy of each linear classifier, we can then identify which attributes the CP-trained model learns more, indicating potential heavy reliance on these attributes. Finally, we can select \(G\) attributes based on two criteria: (1) high accuracy of a linear classifier and (2) high correlation (not causation) with the sensitive attribute.

To validate this approach, we conducted an experiment on CelebA dataset using a subset of 40 pre-annotated attributes and the “hair length” (“hair length” labels are predicted by CLIP as it is not originally labeled in CelebA). ¿tbl:table:G-selection? displays the accuracy for each attribute after training a linear classifier on the top of the CP-trained model, alongside the skewness between the sensitive attribute and other attributes. Attributes like “Pale Skin” show a high correlation with the sensitive attribute but low accuracies, suggesting CP might not rely on them (not satisfying the second condition). “Mouth Sightly Open” exhibits low correlation and accuracy, thus not being considered as \(G\) (failing the first condition). In contrast, attributes such as “bangs”, “wearing hat”, and “hair length” exhibit both high correlation values and accuracies, indicating that they are promising candidates for the attribute \(G\).

\begin{table}[t!]

	Hair Length	Bangs	Wearing Hat	Pale Skin	Mouth Slightly Open
Acc (%)	64.93	68.07	72.63	50.79	56.79
Skewness	0.8	0.91	0.97	0.98	0.27

\vskip

-0.1in \end{table}

13 Further discussion about the faithfulness assumption↩︎

The faithful assumption states that if some two variables are statistically independent, they are d-separated, i.e., there is no connected path between them. Thus, under this assumption, GF-aware methods, which enforce independence between the sensitive attribute and the target label, can achieve CF simultaneously when they successfully achieve GF. The previous works [@anthis2023causal; @rosenblatt2023counterfactual] provide the same argument that GF implies CF under the faithfulness assumption. However, some other previous works [@kennaway2020causation] have demonstrated that while the faithfulness assumption is crucial for causal inference literature, it may not always hold true, especially in complex real-world scenarios. Moreover, the result for GF-aware methods in 3 reveals that the GF-aware methods have a minimal impact on improving CD, implying that the faithfulness assumption does not hold in CelebA and LFW datasets.

14 Additional results↩︎

\begin{table}[t]

4pt

	CelebA						LFW
Method	RFP\(^T\) \(\downarrow\)	Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)	RFP \(\downarrow\)		Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)
w/ \(\bm\theta^T_\text{Scratch}\)	15.27	93.08	13.23	4.44	10.85		89.26	1.88	7.94
w/ \(\bm\theta^T_\text{CKD}\)	10.85	93.98	14.37	4.05	11.64		89.17	1.48	8.07
w/ \(\bm\theta^T_\text{CP}\)	20.37	94.25	34.49	3.28	16.61		89.85	9.26	8.32

\vskip

-0.1in \end{table}

14.1 Result tables with standard deviation↩︎

We report the standard deviation values of the performance comparison results in ¿tbl:table:method-comparison-std? for CIFAR-10B, CelebA, and LFW respectively. The standard deviation values are calculated over four different seeds.

14.2 Impact of the robustness to \(G\) of the teacher model on CKD with CelebA and LFW↩︎

We analyze the effectiveness of the teacher model on real image datasets. We report the performance of CKD using variants of teacher models that are more or less robust to \(G\) on CelebA and LFW. We consider three teacher models, ordered by robustness to \(G\): CP (\(\boldsymbol{\theta}^T_\text{CP}\)), Scratch (\(\boldsymbol{\theta}^T_\text{Scratch}\)), and CKD model with a Scratch teacher (\(\boldsymbol{\theta}^T_\text{CKD}\)). ¿tbl:table:change-teacher-real? displays ACC, DEO, CD (on CelebA and LFW), and RFP (on CelebA) depending on the teacher models. Since we generate a dataset for RFP measurements on CelebA with “hair length” as \(G\), we report RFP values only for CelebA. Through the result, we observe that compared to vanilla-trained teachers \(\boldsymbol{\theta}^T_\text{Scratch}\), using more robust teachers (e.g., \(\boldsymbol{\theta}^T_\text{CKD}\)) achieves slightly better or competitive DEO and CD, while employing less robust teachers (e.g., \(\boldsymbol{\theta}^T_\text{CP}\)) significantly degrades DEO, which are consistent with the results in 5.3 on CIFAR-10B.

14.3 Analaysis on CKD↩︎

Ablation study. To study the effectiveness of our CKD regularization term, we additionally consider a method that is a naive combination of a typical KD method proposed by [@hinton2015distilling] and CP [@russell2017worlds] (i.e., HKD+CP). Note that this combination can be considered as a baseline method that considers both CF and GF if it uses a robust teacher model to \(G\), because the method robustifies the model with respect to \(G\) while achieving CF. 4 compares the method with our CKD. As we expected, the results show that HKD+CP improves both DEO and CD simultaneously. However, its performance is still suboptimal compared to CKD, showing CKD is more effective than the naive combination of KD and CP.

CKD on feature vectors vs logits. CKD can utilize either feature or logit vectors as the target vectors, i.e., \(f(\theta, x)\). For CIFAR-10B and LFW, where we use logits as the target vectors in our experiments, we displayed the performance of CKD using feature vectors as the target vectors in ¿tbl:table:CKD-vectors?. The results demonstrate that CKD using feature vectors exhibits comparable performance to those with logits. Moreover, we can get even better performance on CIFAR-10B using feature vectors, demonstrating that CF and GF can be achieved simultaneously regardless of which type of target vectors is used.

Sensitiveness of \(\mu\). \(\mu\) is the regularization strength for the CKD loss term. Specifically, as \(\mu\) increases, we expect improvements in both DEO and CD. ¿tbl:table:sensitivity-mu? shows the performance of CKD across different values of \(\mu\), aligning with our expectations. Additionally, we note that CKD performance is insensitive to \(\mu\).

The implication of using non-curated IP2P. Uncurated CTF datasets are imperfect. Specifically, some samples generated from the original images in our test datasets were filtered out because the images either showed minimal changes or had alterations that affected non-sensitive attributes including the target attributes. Consequently, the more such incomplete samples exist, the more they will negatively impact the performance of our method. To assess how sensitive CKD is to incomplete CTF training samples, we conducted additional experiments on CIFAR-10B by varying the ratio of incomplete CTF samples in the training set. For a given ratio \(\alpha\) , we assumed that half of the incomplete samples are nearly unchanged, while the other half are samples where both the target and sensitive attributes are altered. We varied \(\alpha\) from 20% to 60% in 10% increments and reported the accuracy, CD, and DEO of CKD in the table below. The results in ¿tbl:table:non-curated-IP2P? indicate that CKD significantly improves both CD and DEO compared to Scratch, even for high \(\alpha\) s. Although this phenomenon has not been fully explained, we hypothesize that the robustness can be attributed to the distillation process, as empirically demonstrated in [@goldblum2020adversarially].

14.4 Additional experimental results on counterfactual samples.↩︎

We additionally report the accuracy (acc-CTF) and DEO (DEO-CTF) for CKD and several baseline methods on counterfactual samples in CelebA-CF in ¿tbl:table:acc-deo-on-CTF?.

14.5 Additional experimental results for other metrics proposed by [@pinto2024matrix].↩︎

We first note that the Counterfactual Disparity (CD) we used is the same metric as the Switch Rate (SR) proposed by [@pinto2024matrix]. We computed P2NR (another metric proposed by [@pinto2024matrix]) on CelebA and obtained values of \(0.036\), \(0.165\), and \(0.339\) for Scratch, CP, and CKD, respectively. These results indicate that CKD achieves low CD with a balanced rate of misclassification across the labels. Additionally, we would like to emphasize that [@pinto2024matrix] focused on scenarios where GF does not imply CF in their experiments—highlighting cases where the faithfulness assumption, which can be overly stringent, does not hold (see line 323 in their paper). However, our work primarily explores the converse: whether CF can imply GF depending on the presence of \(G\), independent of the faithfulness assumption.

\begin{table}[t!]

	Acc \(\uparrow\)	CD \(\downarrow\)	DEO \(\downarrow\)
Scratch	78.01	17.90	27.46
CKD	78.49	2.85	7.30
CKD (20%)	79.82	2.77	11.41
CKD (30%)	79.76	2.88	12.78
CKD (40%)	79.70	2.94	12.88
CKD (50%)	79.72	2.85	14.10
CKD (60%)	79.61	3.01	14.94

\vskip

-0.1in \end{table}

\begin{table}[t!]

	Acc-CTF \(\uparrow\)	DEO-CTF \(\downarrow\)
Scratch	77.22	15.92
SS	81.43	28.74
RW	79.75	23.61
LBC	78.06	33.95
CP	75.21	62.82
CKD	90.08	21.86

\vskip

-0.1in \end{table}

4.5pt

\begin{table}[t!]

	CIFAR-10B (\(\alpha\)=0.8)			LFW
Method	Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)	Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)
w/ logit	78.46	7.11	2.86	89.26	1.88	7.94
w/ feature	78.24	2.64	1.23	88.37	3.98	6.22

\vskip

-0.1in \end{table}

\begin{table}[t!]

	CelebA				LFW
\(\mu\)	Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)		\(\mu\)	Acc \(\uparrow\)	DEO \(\downarrow\)	CD \(\downarrow\)
0.01	94.13	14.83	4.4		0.01	90.05	2.82	15.01
0.1	94.13	14.45	3.03		0.1	90.09	2.39	14.19
1.0	93.63	13.84	4.24		1.0	89.78	2.75	13.66
7.0	93.08	13.23	4.44		10.0	89.23	1.85	13.82
10.0	92.93	13.67	4.10		50.0	88.83	1.8	7.80

\vskip

-0.1in \end{table}

15 Datasheet for dataset↩︎

15.1 Motivation↩︎

For what purpose was the dataset created? These datasets were created for evaluating counterfactual fairness in image classifiers. Furthermore, since our datasets contain counterfactual images generated from real-world images, our datasets can be also used for analyzing the relationship between counterfactual and group fairness on image datasets. For more discussion of the motivation behind our datasets, see 1.

Who created the dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, institution, organization)? The datasets were created by the authors of this paper who were affiliated with Seoul National University and NAVER AI LAB.

Who funded the creation of the dataset? Funding was provided by the National Research Foundation of Korea (NRF); Institute of Information & Communications Technology Planning & Evaluation (IITP); and the SNU-Naver Hyperscale AI Center.

15.2 Composition↩︎

What do the instances that comprise the dataset represent (e.g., documents, photos, people, countries)? The instances represent synthetically generated images and corresponding real-world original images from two popular benchmark facial image datasets, CelebA [@celeba] and LFW [@Huang2007a].

How many instances are there in total (of each type, if appropriate)? CelebA-CF and LFW-CF contain a total of 230 and 144 image pairs of original and counterfactual images, respectively.

Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set? We uniformly sampled a subset of test images in CelebA and LFW to balance the target and group labels (see more details in 2). Then, we made our datasets including all possible samples according to our filtering process.

What data does each instance consist of? Each instance contains a pair of original and counterfactual images.

Is there a label or target associated with each instance? Yes, there are 40 binary annotations that originated from CelebA and LFW.

Is any information missing from individual instances? No

Are relationships between individual instances made explicit (e.g., users’ movie ratings, social network links)? Yes, instances that correspond to a counterfactual pair are explicitly annotated as such in our dataset. Otherwise, there are no relationships between individual instances.

Are there recommended data splits (e.g., training, development/validation, testing)? No, the dataset is created for the purpose of testing.

Are there any errors, sources of noise, or redundancies in the dataset? Using IP2P as the initial step in constructing our datasets might introduce some noise or errors in the datasets. Refer to 9 for further details.

Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)? Yes, it is self-contained.

Does the dataset contain data that might be considered confidential (e.g., data that is protected by legal privilege or by doctor patient confidentiality, data that includes the content of individuals’ non-public communications)? No

Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety? Yes, the dataset may cause some anxiety about sex labels. See 2 and 9.

Does the dataset identify any subpopulations (e.g., by age, gender)? Yes, our datasets were created after evaluating whether counterfactual samples regarding visually perceived sexual traits were generated correctly or not. This evaluation was conducted by five human annotators. Thus, our datasets contain the identification of visually perceived sexual traits which represent some statistically representative features for each sex. See more discussion in 2.

Is it possible to identify individuals (i.e., one or more natural persons), either directly or indirectly (i.e., in combination with other data) from the dataset? Yes, our datasets are generated from CelebA and LFW, which are facial datasets collected on the internet.

Does the dataset contain data that might be considered sensitive in any way (e.g., data that reveals race or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers; criminal history)? Yes, we set sex as the sensitive attribute and created our CTF samples with the sensitive attribute flipped.

15.3 Collection Process↩︎

How was the data associated with each instance acquired? Our datasets are generated through image editing using IP2P (See 2).

What mechanisms or procedures were used to collect the data (e.g., hardware apparatuses or sensors, manual human curation, software programs, software APIs)? Refer to 2 for a complete description of our data generation process.

If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)? Original test samples in our datasets were uniformly sampled from the test datasets of CelebA and LFW.

Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)? The filtering process for our datasets involved five student annotators who received about 18 USD per hour for their wage.

Over what timeframe was the data collected? Our datasets were generated and evaluated over one month.

Were any ethical review processes conducted (e.g., by an institutional review board)? No

Did you collect the data from the individuals in question directly, or obtain it via third parties or other sources (e.g., websites)? No, we initially obtained the data from publicly available sources. Subsequently, we edited the data and filtered the edited one through human annotators.

Were the individuals in question notified about the data collection? Not applicable

Did the individuals in question consent to the collection and use of their data? Not applicable

If consent was obtained, were the consenting individuals provided with a mechanism to revoke their consent in the future or for certain uses? Not applicable

Has an analysis of the potential impact of the dataset and its use on data subjects (e.g., a data protection impact analysis) been conducted? Not applicable

15.4 Preprocessing/cleaning/labeling↩︎

Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)? Yes, we filtered our generated datasets with human annotators. Refer to 2 for a complete description of our filtering process.

Was the “raw” data saved in addition to the preprocessed/cleaned/labeled data (e.g., to support unanticipated future uses)? No, however, raw data can be reproduced by applying IP2P as described in 2.

Is the software that was used to preprocess/clean/label the data available? Yes, refer to the 2.

15.5 Uses↩︎

Has the dataset been used for any tasks already? Yes, we applied our datasets to evaluate CF in image classifiers in 3 and 6 and analyze the relationship between CF and GF in 5.

Is there a repository that links to any or all papers or systems that use the dataset? We will provide a link to a repository on GitHub that includes references to all papers and systems utilizing the dataset.

What (other) tasks could the dataset be used for? There is no other task where our dataset can be used. The dataset is exclusively designed for evaluating counterfactual fairness in real-world image datasets.

Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses? Because our datasets were generated through the image editing technique, IP2P [@brooks2023instructpix2pix], they may contain implicit biases or errors, which are present in the IP2P model [@luccioni2023stable; @Bianchi_2023]. While we have conducted a thorough human filtering and validation process to minimize these issues in our dataset, future users should still be aware of these limitations.

Are there tasks for which the dataset should not be used? The dataset should not be employed for tasks where the limitations discussed in 9 could pose critical issues, or for tasks that are not for research purposes.

15.6 Distribution↩︎

Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created? Yes, the datasets will be made publicly available.

How will the dataset will be distributed (e.g., tarball on website, API, GitHub)? The dataset will be distributed using tarball on the website. Refer to 10.

When will the dataset be distributed? The datasets will be made publicly available upon acceptance.

Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)? The datasets will be distributed under the CC BY 4.0 license.

Have any third parties imposed IP-based or other restrictions on the data associated with the instances? No

Do any export controls or other regulatory restrictions apply to the dataset or to individual instances? No

15.7 Maintenance↩︎

Who will be supporting/hosting/maintaining the dataset? The datasets are hosted, supported, and maintained by the authors.

How can the owner/curator/manager of the dataset be contacted (e.g., email address)? The corresponding author can be contacted by the e-mail address which will be listed on the first page of this paper after camera-ready.

Is there an erratum? No

Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete instances)? No future updates are currently planned. However, we will monitor the GitHub repository for related issues and address any problems that arise.

If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were the individuals in question told that their data would be retained for a fixed period of time and then deleted)? Not applicable

Will older versions of the dataset continue to be supported/hosted/maintained? Yes, if the datasets are updated, we will maintain the older versions.

If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so? Yes, we make our code and datasets public, and hence others can contribute or extend to our work and datasets.

8pt

---

1. Equal contribution.↩︎

2. Co-corresponding author.↩︎

3. The two datasets use the terms “gender” for indicating their sensitive attributes. However, using such terminology can present some ethical concerns because they can suggest meanings linked to social identities. Thus, we have decided to use the term “sex” instead, which more accurately refers to biological characteristics.↩︎

4. Considering the accuracy degradation of fair-training methods, we set the bound as 98% of the accuracy of Scratch, i.e., if Scratch achieves 95.0% accuracy, then we only consider models with more than 93.1% accuracy.↩︎

5. We assume no edge or unblocked path from \(A\) to \(Y\); otherwise, all counterfactually fair models based on that SCM would produce random predictions with respect to \(X_Y\).↩︎