Diversity Recommendation via Causal Deconfounding of Co-purchase Relations and Counterfactual Exposure


1 EXPERIMENT SUPPLEMENT↩︎

1.1 Source Code↩︎

The source code has been made anonymously available via https://anonymous.4open.science/status/AAAI-BF40.

1.2 Implementation details↩︎

We implemented Cadence using PyTorch and trained it with the Adam optimizer, setting the learning rate to 0.001. The batch size was fixed at 2048, and the embedding dimension was set to 32. For the UACR-driven item-item aggregation layer, we set \(L_{II} = 2\). The number of propagation layers in LightGCN was set to \(L = 3\). For the two-stage candidate selection in the CSCE module, the global selection number \(K_g\) was set to 4 for Beauty, 12 for TaoBao, and 6 for Toy, while the category-specific selection number \(K_c\) was set to 1 across all three datasets. The scaling factor \(\alpha\) was set to 1.15 for Beauty, 1.05 for TaoBao, and 1.15 for Toy.

We employ the Bayesian Personalized Ranking (BPR) [1] loss with negative sampling for training. The BPR loss is defined as: \[\mathcal{L}_{BPR} = \sum_{(u,i,j) \in D_S} -\ln \sigma(r_{u,i} - r_{u,j}) + \lambda ||\boldsymbol{\theta}||^2\] where \(D_S\) is the set of training triplets, \(\sigma\) is the sigmoid function, \(r_{u,i}\) and \(r_{u,j}\) are predicted scores for positive item \(i\) and negative item \(j\), \(\lambda\) is the regularization coefficient, and \(\boldsymbol{\theta}\) represents the model parameters.

1.3 Computing Infrastructure↩︎

All experiments were conducted using NVIDIA A100 80GB PCIe GPUs on an Ubuntu 22.04 system, with Python 3.9 and PyTorch version 2.4.

1.4 Parameter analysis↩︎

a
b
c
d
e
f

Figure 1: Parameter sensitivity on \(\alpha\) and \(K_g\). a — Beauty-\(\alpha\), b — Beauty-\(K_g\), c — TaoBao-\(\alpha\), d — TaoBao-\(K_g\), e — Toy-\(\alpha\), f — Toy-\(K_g\)

In this subsection, we conduct experiments to study the impact of the scaling factor \(\alpha\) and the global filtering coefficient \(K_g\), as illustrated in Figure 1. Under the recommendation list length setting of \(\{100, 300\}\), the category-specific selection number \(K_c\) achieves the best performance when set to 1 for all datasets. To simultaneously consider accuracy and diversity performance, we adopt the \(F_\beta\)-score as the evaluation metric, where the \(\beta\) value can be adjusted according to specific requirements and application scenarios. The \(F_\beta\)-score is defined as:

\[F_\beta = (1 + \beta^2) \cdot \frac{\text{Coverage@100} \cdot \text{Recall@100}}{\beta^2 \cdot \text{Recall@100} + \text{Coverage@100}}\]

Here, the \(\beta\) values for the Beauty, TaoBao, and Toy datasets are set to 4, 20, and 15, respectively.

1.4.0.1 Effect of \(\alpha\)

\(\alpha\) represents the intensity of counterfactual popularity enhancement. As \(\alpha\) increases, potential interest items selected based on UACR are more likely to be successfully recommended, leading to a continuous rise in diversity metrics. This is because the two-stage filtering and aggregation strategy introduces more diversified candidate items. The accuracy metrics show a trend of first increasing and then decreasing, indicating that moderate \(\alpha\) values can discover relevant potential interest items. However, when \(\alpha\) becomes too large, it leads to recommending too many items that users are not interested in, thereby compromising recommendation quality. This phenomenon demonstrates the need to find an optimal balance between discovering potential diversified interests and maintaining recommendation relevance.

1.4.0.2 Effect of \(K_g\)

\(K_g\) represents the number of candidate items in the global filtering process of the two-stage screening. As \(K_g\) increases, the accuracy of the Beauty dataset shows a trend of first increasing and then decreasing, indicating that in small-scale dense datasets, moderately increasing the number of candidates helps discover relevant items, but too many candidates introduce noise and reduce accuracy. In contrast, the accuracy of TaoBao and Toy datasets continues to rise, showing that in large-scale sparse data environments, more candidate items can effectively alleviate data sparsity issues and significantly improve recommendation hit rates. However, the diversity metrics of all three datasets show a declining trend, because as the number of global candidates increases, mainstream items with high UACR scores occupy more recommendation positions, squeezing out the space for diverse candidates, thus sacrificing the diversity of recommendation results to some extent.

1.5 Complexity Analysis.↩︎

Table 1: Total Training Time Comparison Across Different Datasets and Methods
Dataset EDUA DGRec DGCN CPGRec KG-Diverse DivGCL Ours LightGCN
Beauty 4958.0593 201.0455 127.8954 130.6326 2018.9638 2331.0843 97.5654 87.1020
TaoBao 83792.6725 6018.0012 3443.7021 5168.6267 49538.6218 54355.1921 2308.6280 2217.9973
Toy 117801.3664 8617.4631 5594.5200 7989.3924 8430.4206 53799.6355 2756.3432 2357.4224

Our framework is built on the lightweight LightGCN backbone, with the insertion of two modules: UACR-Guided Causal Inference and Graph Refinement (CIGR) and UACR-Guided Candidate Selection and Counterfactual Exposure (CSCE). In LightGCN, user-item interactions are modeled through \(L\)-layer graph convolutions. Let \(E\) denote the number of user-item edges, \(d\) the embedding dimension, and \(b\) the batch size. The time complexity of one forward pass is \(O(L \times E \times d)\).

In the CIGR module, we compute UACR values only for item pairs with co-purchase relations. Let \(\gamma\) denote the number of such co-purchase edges; the computation complexity is \(O(\gamma)\), as these edges are typically sparse. As this process is executed once offline, the overhead is negligible. Afterward, we apply geometric truncation to prune the co-purchase graph to \(\beta\) edges and perform \(L_{II}\)-layer graph convolution over this sparse graph. The time complexity is \(O(L_{II} \times \beta \times d)\). The CSCE module is executed only once after training for offline inference, with a complexity of \(O(U)\), which is negligible. The overall training complexity is \(O(L \times E \times d + L_{II} \times \beta \times d)\). Therefore, the overall training complexity remains at the same level as LightGCN, since \(\beta \ll \gamma\) and \(\beta \ll E\).

To validate the training efficiency, we record the total time to convergence (measured in seconds) for each model. To ensure a fair comparison, we adopt an early stopping strategy based on Recall@100, where training is terminated if no improvement is observed for 10 consecutive epochs. Experimental results show that our method consistently outperforms all baselines, while achieving training efficiency comparable to LightGCN, as shown in Table 1, demonstrating both its effectiveness and practicality.

1.6 Performance Stability under Random Initialization↩︎

a

Figure 2: Performance comparison with error bars across three datasets. The error bars represent the standard deviation over five independent runs with different random seeds..

To validate the robustness of Cadence, we conduct five independent runs for each experiment using different random seeds [2021, 2022, 2023, 2024, 2025], reporting the mean and standard deviation for each evaluation metric. As illustrated in Figure 2, Cadence consistently demonstrates superior performance with low variance across multiple independent executions, confirming its stability under various random initialization conditions.

To further establish the statistical significance of the performance improvements, we employ the Wilcoxon signed-rank test to compare Cadence against the top-performing baseline methods on each dataset: CPGRec on the Beauty dataset, DGRec on the TaoBao dataset, and DivGCL on the Toy dataset. The statistical analysis focuses on two key metrics: Recall@100 and Coverage@100. The test yields a statistic of 15.0 with a corresponding p-value of 0.0312, confirming that Cadence’s improvements over the existing state-of-the-art methods are statistically significant at the 5% significance level.

2 Related Work↩︎

2.1 Counterfactual Recommendation↩︎

In recent years, counterfactual recommendation has emerged as a key direction in causal recommender systems, aiming to correct selection and exposure biases introduced by historical logging policies. Early approaches mainly relied on propensity score modeling, such as inverse propensity scoring (IPS) [2], to achieve unbiased estimation from logged user feedback. With the rise of deep learning, recent work has explored integrating counterfactual reasoning with neural models. These approaches include generating counterfactual sequences in sequential recommendation [3], [4], synthesizing alternative interactions in knowledge graphs [5], and modeling confounders through structural causal models [6]. These methods not only improve the accuracy and robustness of recommendation under biased data, but also provide theoretical foundations and practical solutions across various tasks, including causal disentanglement of multi-feedback signals in multi-behavior recommendation [7], correction of contextual interference in session-based scenarios [8], counterfactual explanation generation for interpretable recommendation [9], and mitigation of unfair exposure caused by user attributes or system biases in fair recommendation [10].

Discussion. UCRS [11] uses user-issued controls and counterfactual inference to adjust recommendations by simulating changes in user features, aiming at controllable recommendation rather than explicit diversity optimization or causal item modeling. In contrast, our method enhances diversity from a causal perspective by constructing a deconfounded causal item graph and applying counterfactual exposure interventions to uncover under-exposed yet relevant items.

2.2 Diversified Recommendation↩︎

Diversity-aware recommendation aims to alleviate the issue of homogeneity in recommendation results and enhance overall user satisfaction [12]. Early approaches primarily relied on re-ranking strategies, such as Maximum Marginal Relevance (MMR) [13][16] and Determinantal Point Process (DPP) [17][19], which optimize the recommendation list to promote item dissimilarity. With the advancement of graph neural networks [20], more recent methods leverage high-order graph structures to capture complex user–item relationships for improved diversity, including category-aware sampling [21], submodular neighbor selection [22], clustering-based candidate generation [23], and dynamic graph updates [24]. Moreover, the emergence of large language models (LLMs) has introduced new opportunities for diversity modeling, where controllable generation frameworks [25] are developed to regulate category coverage and further enrich the diversity of recommendations.

Discussion. Most existing approaches lack principled theoretical grounding and a causal perspective, limiting their capacity to generate genuinely diverse recommendations. Traditional co-occurrence-based methods are prone to confounding from item popularity and user attributes, leading systems to favor popular but homogeneous items while neglecting underexposed yet causally relevant ones. In our work, we employ causal inference to remove these spurious associations, recover true item dependencies, uncover latent user interests, and enhance diversity without compromising relevance.

3 Detailed Proof of Theorem: Norm-Popularity Correlation↩︎

This section provides the complete mathematical proof of the norm–popularity correlation theorem.

A.1 Notation and Sampling Assumptions↩︎

  • \(\mathcal{U},\mathcal{I}\): user set and item set;

  • \(A\): the random-walk normalised adjacency matrix with \(\|A\|_2 \le 1\);

  • \(M = \sum_{k=0}^{K} \alpha_k A^k\): LightGCN propagation operator with \(\alpha_k \ge 0\) and \(\sum_k \alpha_k = 1\);

  • \(n_i\): number of positive interactions for item \(i\) (“popularity”), and \(|D| = \sum_i n_i\);

  • \(\eta,\lambda\): SGD step size \(\eta\) and \(\ell_2\) regularisation coefficient \(\lambda\);

  • \(\mathbf{e}_x^{(\ell)}\): embedding of node \(x\) at layer \(\ell\); the final embedding is defined as \(\displaystyle \mathbf{e}_x = \sum_{k=0}^{K} \alpha_k \,\mathbf{e}_x^{(k)}\);

  • \(\Delta = \langle \mathbf{e}_u,\mathbf{e}_i \rangle - \langle \mathbf{e}_u,\mathbf{e}_j \rangle\);

  • \(\sigma(\cdot)\): sigmoid function.

Default sampling strategy. At each training step we randomly draw one positive interaction \((u,i)\) and then uniformly sample a negative item \(j\) from the set of items not interacted by \(u\); hence \[\Pr[i] = \frac{n_i}{|D|}. \label{eqA:prob}\tag{1}\]

A.2 Full Chain-Rule Derivation of \(\partial\ell/\partial\mathbf{e}_i^{(0)}\)↩︎

The BPR loss is \[\ell = -\log \sigma(\Delta), \qquad \frac{\partial \ell}{\partial \Delta} = 1-\sigma(\Delta).\]

3.0.0.1 Gradient with respect to the final embeddings.

\[\frac{\partial \ell}{\partial \mathbf{e}_i} = (1-\sigma(\Delta))\,\mathbf{e}_u,\] \[\frac{\partial \ell}{\partial \mathbf{e}_j} = -(1-\sigma(\Delta))\,\mathbf{e}_u,\] \[\frac{\partial \ell}{\partial \mathbf{e}_u} = (1-\sigma(\Delta))(\mathbf{e}_j-\mathbf{e}_i).\]

3.0.0.2 Backpropagation to \(\mathbf{e}_i^{(0)}\)

Since \(\mathbf{e} = M\mathbf{e}^{(0)}\) (LightGCN only performs linear propagation), we have \[\frac{\partial\mathbf{e}_i}{\partial\mathbf{e}_i^{(0)}} = M_{ii},\] \[\frac{\partial\mathbf{e}_u}{\partial\mathbf{e}_i^{(0)}} = M_{ui},\] \[\frac{\partial\mathbf{e}_j}{\partial\mathbf{e}_i^{(0)}} = 0.\] Therefore \[\frac{\partial\ell}{\partial\mathbf{e}_i^{(0)}} =(1-\sigma(\Delta)) \bigl[M_{ii}\mathbf{e}_u+M_{ui}(\mathbf{e}_j-\mathbf{e}_i)\bigr]. \label{eqA:grad95full}\tag{2}\]

3.0.0.3 Component along \(\hat{\mathbf{e}}_i^{(0)}\) direction

Let \(\theta_{ui}\) be the angle between \(\mathbf{e}_u\) and \(\mathbf{e}_i^{(0)}\), \(\rho_i:=\|\mathbf{e}_i\|_2/\|\mathbf{e}_u\|_2\), then \[g_{ui}^{\parallel} =(1-\sigma(\Delta)) \bigl[M_{ii}\cos\theta_{ui}+M_{ui}\cos\theta_{ui}-M_{ui}\rho_i\bigr] \|\mathbf{e}_u\|_2. \label{eqA:g95parallel}\tag{3}\]

A.3 Single-step Norm Increment↩︎

SGD update with small learning rate for stability, \(T\eta\!\ll\!1\), ignoring \(O(\eta^2)\) terms \[\mathbf{e}_i^{(0)}\leftarrow \mathbf{e}_i^{(0)}-\eta[\mathbf{g}+2\lambda\mathbf{e}_i^{(0)}], \quad \mathbf{g}=\frac{\partial\ell}{\partial\mathbf{e}_i^{(0)}}.\] Let \(g_\parallel=\hat{\mathbf{e}}_i^{(0)\top}\mathbf{g}\), then \[\Delta\|\mathbf{e}_i^{(0)}\|_2 =-\eta g_\parallel-2\eta\lambda\|\mathbf{e}_i^{(0)}\|_2+O(\eta^2). \label{eqA:delta95norm}\tag{4}\]

Substituting 3 and taking the conditional expectation over \((u,i,j)\), in large-scale datasets \(\bar\kappa\) and \(\beta\) vary minimally across items, so we approximate them as constants \[\bar\kappa := \mathbb{E}_{u|i}[(1-\sigma(\Delta))(M_{ii}\cos\theta_{ui}+M_{ui}\cos\theta_{ui})],\] \[\beta := \mathbb{E}_{u|i}[(1-\sigma(\Delta))M_{ui}/\|\mathbf{e}_u\|_2].\] Since user embedding norms have no significant correlation with individual item popularity, in the statistical sense of large datasets we can treat \(\mathbb{E}_{u|i}[\|\mathbf{e}_u\|_2]\) as a constant \[\mathbb{E}[\Delta\|\mathbf{e}_i^{(0)}\|_2] =\eta\!\Bigl[(\bar\kappa-\beta\|\mathbf{e}_i\|_2)\frac{n_i}{|D|} -2\lambda\|\mathbf{e}_i^{(0)}\|_2\Bigr]. \label{eqA:exp95delta}\tag{5}\]

A.4 Fixed-Point Analysis and Linear Relationship↩︎

Define \(\mu_i := \mathbb{E} \|\mathbf{e}_i\|_2\). At steady state \(\mathbb{E}[\Delta \|\mathbf{e}_i^{(0)}\|_2] = 0\), i.e. \[(\bar\kappa - \beta \mu_i) \frac{n_i}{|D|} = 2\lambda \mu_i.\] Solving yields \[\boxed{\mu_i = \frac{\bar\kappa}{2\lambda + \beta}\,\frac{n_i}{|D|}}\quad (\beta \ge 0). \label{eqA:fixedpoint}\tag{6}\] Because the dependence of \(\bar\kappa\) and \(\beta\) on \(i\) is extremely weak, \(\mu_i\) grows linearly with \(n_i\) and the proportionality constant is essentially item-independent.

A.5 High-Probability Norm Upper Bound↩︎

3.0.0.4 Hard Bound.

Let \(\eta \le 1/(4\lambda)\) and set \(C = \bar\kappa/(2\lambda)\) and \(c = 2\eta(\bar\kappa + \lambda C)\). Then for any \(\epsilon>0\) and \(T \in \mathbb{N}\), \[\Pr\!\Bigl( \sup_{t \le T} \|\mathbf{e}_{i,t}^{(0)}\|_2 > C + \epsilon \Bigr) \le \exp\!\bigl(-\epsilon^{2}/(4Tc^{2})\bigr).\]

3.0.0.5 Proof.

Let \(Z_t = \|\mathbf{e}_{i,t}^{(0)}\|_2 - C\). From 4 and the bound \(|g_{\parallel}| \le \bar\kappa \|\mathbf{e}_u\|_2 + |M_{ui}| \|\mathbf{e}_j\|_2 \le \bar\kappa C\), we have \(|Z_{t+1} - Z_t| \le c\). Moreover, \(\mathbb{E}[Z_{t+1}-Z_t \mid \mathcal{F}_t] \le 0\), so \(\{Z_t\}\) is a supermartingale. The maximal-difference form of the Azuma–Hoeffding inequality then gives the claimed bound.

The parameters \(\bar\kappa\) and \(C\) vary little across items, so the bounded-difference condition holds uniformly for all \(i\).

A.6 Theorem Statement (Uniform Negative Sampling)↩︎

Combining 1 , 6 and the above bound, we obtain \[\|\mathbf{e}_i\|_2 = \Theta(n_i) \quad\text{and}\quad \Pr\bigl( \|\mathbf{e}_i\|_2 > \|\mathbf{e}_j\|_2 \bigr) \ge 1 - e^{-\Omega(1)}.\]

This completes the proof.

References↩︎

[1]
S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, “BPR: Bayesian personalized ranking from implicit feedback,” arXiv preprint arXiv:1205.2618, 2012.
[2]
A. Agarwal, K. Takatsu, I. Zaitsev, and T. Joachims, “A general framework for counterfactual learning-to-rank,” in Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 2019, pp. 5–14.
[3]
Z. Wang et al., “Counterfactual data-augmented sequential recommendation,” in Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, 2021, pp. 347–356.
[4]
S. Zhang, D. Yao, Z. Zhao, T.-S. Chua, and F. Wu, “Causerec: Counterfactual user sequence synthesis for sequential recommendation,” in Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, 2021, pp. 367–377.
[5]
S. Mu, Y. Li, W. X. Zhao, J. Wang, B. Ding, and J.-R. Wen, “Alleviating spurious correlations in knowledge-aware recommendations through counterfactual generator,” in Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, 2022, pp. 1401–1411.
[6]
Y. Chen, J. Cao, Y. Wang, J. Wu, H. Chen, and G. Xu, “Causal variational inference for deconfounded multi-behavior recommendation,” ACM Transactions on Information Systems, 2025.
[7]
W. Wang, X. Lin, L. Wang, F. Feng, Y. Ma, and T.-S. Chua, “Causal disentangled recommendation against user preference shifts,” ACM Transactions on Information Systems, vol. 42, no. 1, pp. 1–27, 2023.
[8]
S. Tang, S. Lin, J. Ma, and X. Zhang, “CoDeR: Counterfactual demand reasoning for sequential recommendation,” in Proceedings of the AAAI conference on artificial intelligence, 2025, vol. 39, pp. 12649–12657.
[9]
Y. Li, X. Sun, H. Chen, S. Zhang, Y. Yang, and G. Xu, “Attention is not the only choice: Counterfactual reasoning for path-based explainable recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 9, pp. 4458–4471, 2024.
[10]
P. Shao et al., “Average user-side counterfactual fairness for collaborative filtering,” ACM Transactions on Information Systems, vol. 42, no. 5, pp. 1–26, 2024.
[11]
W. Wang, F. Feng, L. Nie, and T.-S. Chua, “User-controllable recommendation against filter bubbles,” in Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, 2022, pp. 1251–1261.
[12]
C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen, “Improving recommendation lists through topic diversification,” in Proceedings of the 14th international conference on world wide web, 2005, pp. 22–32.
[13]
M. Abdool et al., “Managing diversity in airbnb search,” in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 2952–2960.
[14]
J. Carbonell and J. Goldstein, “The use of MMR, diversity-based reranking for reordering documents and producing summaries,” in Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval, 1998, pp. 335–336.
[15]
Z. Lin et al., “Feature-aware diversified re-ranking with disentangled representations for relevant recommendation,” in Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 2022, pp. 3327–3335.
[16]
L. Peska and P. Dokoupil, “Towards results-level proportionality for multi-objective recommender systems,” in Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, 2022, pp. 1963–1968.
[17]
L. Chen, G. Zhang, and E. Zhou, “Fast greedy map inference for determinantal point process to improve recommendation diversity,” Advances in neural information processing systems, vol. 31, 2018.
[18]
L. Gan, D. Nurbakova, L. Laporte, and S. Calabretto, “Enhancing recommendation diversity using determinantal point processes on knowledge graphs,” in Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, 2020, pp. 2001–2004.
[19]
Y. Huang, W. Wang, L. Zhang, and R. Xu, “Sliding spectrum decomposition for diversified recommendation,” in Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2021, pp. 3041–3049.
[20]
Y. Tian, C. Zhang, Z. Guo, C. Huang, R. Metoyer, and N. V. Chawla, “RecipeRec: A heterogeneous graph learning model for recipe recommendation,” arXiv preprint arXiv:2205.14005, 2022.
[21]
Y. Zheng, C. Gao, L. Chen, D. Jin, and Y. Li, “DGCN: Diversified recommendation with graph convolutional networks,” in Proceedings of the web conference 2021, 2021, pp. 401–412.
[22]
L. Yang et al., “Dgrec: Graph neural network for recommendation with diversified embedding generation,” in Proceedings of the sixteenth ACM international conference on web search and data mining, 2023, pp. 661–669.
[23]
K. Liu, J. Ma, S. Feng, H. Zhang, and Z. Zhang, “DRGame: Diversified recommendation for multi-category video games with balanced implicit preferences,” in International conference on database systems for advanced applications, 2024, pp. 254–263.
[24]
R. Ye et al., “Dynamic graph construction for improving diversity of recommendation,” in Proceedings of the 15th ACM conference on recommender systems, 2021, pp. 651–655.
[25]
J. Chen, C. Gao, S. Yuan, S. Liu, Q. Cai, and P. Jiang, “Dlcrec: A novel approach for managing diversity in llm-based recommender systems,” in Proceedings of the eighteenth ACM international conference on web search and data mining, 2025, pp. 857–865.

  1. Corresponding author.↩︎

  2. Corresponding author.↩︎