1 Introduction↩︎

Graph learning has emerged as a pivotal field at the intersection of machine learning and graph theory, offering a versatile framework for modeling and analyzing complex relationships in various domains [1], [2]. With the increasing prevalence of graph-structured data, ranging from social networks to biological interactions [3], the demand for effective methods to extract meaningful insights from graphs has grown substantially.

With the rising popularity of graph neural networks, particularly notable for their effective message-passing mechanisms [4], a myriad of graph-related challenges have witnessed outstanding performance. However, despite the promising advancements in graph learning, the current landscape predominantly relies on extensive labeled data, introducing challenges associated with high annotation costs, time-intensive processes, and resource demands [5], [6]. This limitation becomes particularly evident in practical applications where obtaining substantial labeled data is impractical. To illustrate this imperative, in molecular domain, annotating intricate molecular structures or interactions could involve sophisticated experiments and the engagement of specialized scientists, leading to elevated annotation costs [7]; in the genomics research, acquiring precisely annotated gene functions or interactions might demand the expertise and time of professional biologists for understanding complex relationships [8]. These instances highlight the urgent need for graph learning methodologies tailored to low-resource settings, emphasizing the critical necessity of developing approaches that can operate effectively with limited labeled data.

=[draw=black, rounded corners,minimum height=1em, text width=24.50em, edge=black!10, text opacity=1, align=center, fill opacity=.3, text=black,font=, inner xsep=3pt, inner ysep=1pt, ] =[draw=black, rounded corners,minimum height=1em, text width=6.28em, edge=black!10, text opacity=1, align=center, fill opacity=.5, text=black,font=, inner xsep=3pt, inner ysep=1pt, ] =[draw=black, rounded corners,minimum height=1em, text width=6.28em, edge=black!10, text opacity=1, align=center, fill opacity=.8, text=black,font=, inner xsep=3pt, inner ysep=1pt, ]

For effective graph learning, researchers have conducted extensive and specialized studies focusing on the exploration of graph machine learning under low-resource settings, aiming at mitigating the costs and time associated with annotation. Nevertheless, although there have been increasingly applied in effective graph learning, this rapidly expanding field still lacks a systematic review. To fill this gap, in this paper, we develop a novel concept of Data-Efficient Graph Learning (DEGL) to summarize the existing works of DEGL and provide a promising research frontier to aid researchers in reviewing, summarizing, and strategizing for the future.

2 Taxonomy↩︎

To enhance our understanding of the dynamic evolution in DEGL, we identify pivotal research endeavors, analyze their motivations, and succinctly encapsulate their primary technical contributions. As illustrated in Figure 1, this survey establishes a new taxonomy, which divides these works into three different categories, i.e., self-supervised graph learning, semi-supervised graph learning, and few-shot graph learning. These groups can be further summarized in different scenarios. Then, we briefly introduce these three categories as follows:

• Self-supervised Graph Learning is a paradigm that leverages the inherent structure and relationships within graph data to train models without relying on external labeled information. At its core, the key idea is to design tasks that encourage the model to learn meaningful representations from the graph data itself. Based on the presence of a specific downstream task, self-supervised graph learning can be further categorized into non-end-to-end graph representation learning [1] and end-to-end graph clustering [9], [10].

• Semi-supervised Graph Learning involves using both labeled and unlabeled samples to train models, capitalizing on the available information while handling scenarios with limited labeled samples. The fundamental idea is to leverage the relationships within the labeled and unlabeled instances to guide the model in learning representations for unlabeled nodes or graphs. Depending on whether there is a distribution shift during inference, we further categorize semi-supervised graph learning into classical semi-supervised graph learning [11], [12] and semi-supervised graph learning under domain shift [13], [14].

• Few-shot Graph Learning is a specialized area designed to enable models to effectively generalize and make accurate predictions when presented with only a limited number of labeled examples. The core idea is to equip the model with the ability to learn from a few annotated instances and then apply this acquired knowledge to make predictions on new, unseen data. Thus researchers either adopt metric learning to encourage each query node to approach its prototypes [15] or parameter optimization to generate node representations using meta-learning [7].

The three research directions are interconnected rather than mutually exclusive, contributing collectively to the rapid advancement of the data-efficient graph learning field. Table 1 further analyzes the representative DEGL works according to the different properties. In the following sections, we delve deeper into these three research directions, exploring key challenges, representative solutions, and emerging trends.

3 Self-supervised Graph Learning↩︎

3.1 Graph Representation Learning↩︎

Graph Representation Learning (GRL) [1] allows the model to capture nuanced patterns and dependencies among nodes without relying on manual labels, making it particularly valuable in scenarios where labels are expensive or hard to acquire. In general, the framework of GRL can be summarized as: \[\min_{\theta,\psi}\mathcal{L}\left (\mathcal{D}, f_\theta, g_\psi\right ), \label{eq:grl}\tag{1}\] where \(\mathcal{D}\) denotes the data distribution of the unlabeled graph, \(f_\theta\) denotes the node or graph encoder, aiming at learning a low-dimensional representation \(h_i\in\mathcal{H}\) for node \(v_i\). The decoder \(g_\theta\) takes the node representation \(\mathcal{H}\) as its input and is generally well-designed on specific downstream tasks.

Existing GRL methods can be roughly divided into three categories, i.e., generation-based, contrastive-based, and auxiliary property-based methods.

3.1.1 Generation-based GRL↩︎

Generation-based GRL [16], [17] aims at reconstructing the input full graph or sampled subgraphs. These methods contribute to understanding the underlying structure and node dependencies within the graph, which encourages the model to encode representations that preserve the inherent information. The self-supervised loss is generally designed to quantify the difference between the reconstructed and the original graph. In such case, Eq. 1 is derived as: \[\min_{\theta,\psi}\mathcal{L}\left ( g_\psi (f_\theta(\hat{\mathcal{G}})), \mathcal{G}\right ),\] where \(\mathcal{G}\) denotes the perturbed graph data, \(f_\theta(\cdot)\) and \(g_\psi (\cdot)\) denote graph encoder and decoder respectively. MGAE [18], estimates raw features from the noisy input node features in each GNN layer, i.e., \(\hat{\mathcal{G}}=(A,\hat{X})\), where \(\hat{X}\) denotes the corrupted random noise. EdgeMask [1] and SuperGAT [1] are representative works that recover the graph structure. Specifically, EdgeMask recovers the similarity between the embeddings of two connected nodes, and SuperGAT adopts a hierarchical variational inference strategy to reconstruct the global topological structure.

3.1.2 Contrastive-based GRL↩︎

Contrastive-based GRL [19]–[21] mainly incorporates contrastive learning on learning node representations by contrasting similar instances (positive pairs) against dissimilar instances (negative pairs). Contrastive-based models provide a powerful mechanism for exploiting the inherent structure and relationships within the graph. For this objective, Eq. 1 can be derived as: \[\min_{\theta,\psi}\mathcal{L}\left ( g_\psi (f_\theta(\hat{\mathcal{G}}_{\text{view1}}), f_\theta(\hat{\mathcal{G}}_{\text{view2}}))\right ),\] where \(\hat{\mathcal{G}}_{\text{view1}}\) and \(\hat{\mathcal{G}}_{\text{view2}}\) denote the augmented instances, which can be node pairs, graph pairs, and task pairs. \(g_{\psi}\) denotes the similarity discriminator, which estimates the distances of contrastive instances. GraphCL [19] contrasts two generated views of node feature masking and edge reconstruction, and then the Mutal Information is maximized between two target nodes from different views. JOAO [22] proposes a bi-level optimization problem, which optimizes the augmentation selection module and contrastive objectives jointly. DGI [23] contrasts node-level representations with graph-level ones for better local-global interactions. For heterogeneous graphs, HeCo [24] is designed to contrast two generated views of network schema and meta-paths.

3.1.3 Auxiliary Property-based GRL↩︎

Auxiliary property-based GRL [25], [26] generates pseudo labels by leveraging classical attributive and graph algorithms, to enrich the self-supervision signals. Compared with the human-annotated labels in supervised learning, the pseudo labels in auxiliary property-based GRL do not need additional cost. The utilization of auxiliary properties in GRL enhances the robustness and generalization capability of the representations. Under this setting, Eq. 1 can be derived as: \[\min_{\theta,\psi}\mathcal{L}\left ( g_\psi (f_\theta(\mathcal{G})), c(\mathcal{G})\right ),\] where \(c(\mathcal{G})\) denotes the annotated auxiliary properties, such as the shortest path and graph centrality. S\(^2\)GRL [27] is designed to estimate the shortest paths among different nodes for better structural representations. CentralityScoreRanking [28] predicts the ranking of centrality scores among node pairs, and compares the ranking with pseudo labels. NodeProperty [1] adopts a node-level property construction task aiming at exploring the properties of node features. SimP-GCN [29] designs a \(k\)-nearest-neighbor graph based on the node features to enhance the neighbor aggregation performance.

3.1.4 Discussion↩︎

Generation-based GRL is easier to implement as the reconstruction task is easy to build. But it is memory-consuming in certain cases such as dealing with large-scale graphs. Auxiliary property-based methods benefit from the uncomplicated decoders, while the selection of effective auxiliary properties is challenging. Compared with the above categories, contrastive-based ones are more flexible.

3.2 Graph Clustering↩︎

In self-supervised graph learning, an end-to-end learning paradigm that possesses a specific downstream task mainly boils down to graph clustering. Graph clustering aims to divide the nodes into \(C\) disjoint clusters without label signals as reliable guidance. Recently, the graph clustering task has attracted widespread attention and extensive approaches have focused on this task with promising performance. In general, given an attribute graph \(\mathcal{G}\), the framework of graph clustering includes a self-supervised neural network \(F\) that outputs the node representations and a clustering network/traditional clustering \(\Omega\) that exports the clustering result \(\hat{\mathbf{Y}}\), i.e., \[\hat{\mathbf{Y}} = \Omega(F(\mathcal{G}), C).\] Existing methods can be roughly partitioned into three main groups: reconstructive-based, adversarial-based, and contrastive-based approaches. In the following, we present a comprehensive overview of these algorithms.

3.2.1 Reconstructive-based Methods↩︎

Reconstructive-based methods typically encode either the attribute information or the structural information of the graph, aiming to reconstruct the input information to supervise the network training and achieve meaningful node representations. DAEGC [30] performs representation learning through an attention network based on node features and structural information under the GAT paradigm. The network training is supervised by reconstructing the graph structure and a self-supervised clustering module. SDCN [9] leverages auto-encoder (AE) to assist GCN, effectively alleviating the over-smoothing issue. It obtains semantically rich node representations by optimizing the reconstruction loss of the AE and a self-supervised clustering loss based on GCN. DFCN [31] introduces a dynamic information fusion technique based on AE and graph auto-encoder (GAE) to explore attributive and structural information finely. It also designs a triplet self-supervision mechanism for self-supervised clustering. Based on [31], DCRN [10] reduces information correlation through a dual approach to prevent representation collapse and obtain discriminative node representations. R\(^2\)FGC [32] builds upon the relational learning at the attributive and structural levels from both global and local views based on AE and GAE. It preserves relationships among positive samples and reduces redundant relationships among negative samples, thereby acquiring effective and discriminative node representations.

3.2.2 Adversarial-based Methods↩︎

Adversarial-based methods engage in a game between the generator and the discriminator to achieve robust node representations. AGAE [33] combines adversarial learning and auto-encoder to perform representation learning, which introduces an adversarial regularization term and adaptive partition-dependent prior to guide the network training. ARGA [34] incorporates an adversarial training scheme into the graph auto-encoder architecture to regularize the latent codes for learning a robust graph representation. The adversarial training module is designed to discern whether the latent codes originate from a real prior distribution or the graph encoder. Further, JANE [35] designs this prior distribution by incorporating the node embeddings to capture the semantic variations.

3.2.3 Contrastive-based Methods↩︎

Contrastive-based methods enhance the discriminative power of learned representations by pulling positive samples closer and pushing negative samples farther apart to explore the semantic information. Under the graph contrastive learning framework, GDCL [36] utilizes pseudo-labels to randomly select samples from classes different from positive samples to form negative samples, which corrects the sampling bias and thus decreases the false-negative samples in graph clustering. SCAGC [37] further constructs the self-consistent contrastive loss by treating all samples from the same class in both graph views as positive samples and all non-matching samples as negative samples. CGC [38] utilizes a multi-level scheme for the selection of positive and negative samples, ensuring their ability to reflect hierarchical community structures and network homophily. Moreover, CGC extends its applicability to temporal graph clustering, which is capable of detecting change points. Additionally, CONGREGATE [39] reexamines graph clustering through a geometric lens. It constructs a novel heterogeneous curvature space for generating representations and introduces an augmentation-free reweighted contrastive method to focus more on both hard negatives and hard positives in the curvature space. SCGC [40] incorporates a low-pass denoising operation in pre-processing, employs un-shared siamese encoders to eliminate the need for graph augmentation in contrastive learning, and introduces a cross-view structural consistency objective function to boost the discriminative capability of the learned network and avert direct clustering-guided loss.

3.2.4 Discussion↩︎

In addition to attribute graphs, numerous studies have explored clustering tasks on other graph types, such as heterogeneous graphs and dynamic graphs. Furthermore, graph-level clustering is another research-worthy issue [41], but it has been relatively underexplored. Besides, graph clustering can be applied to practical applications, such as guiding recommendation services, analyzing protein-protein interaction networks, and uncovering cellular heterogeneity in single-cell RNA-Seq analyses, e.g. [8].

4 Semi-supervised Graph Learning↩︎

4.1 Classical Semi-supervised Graph Learning↩︎

Semi-supervised learning is one of the most important tasks in machine learning, which manages to leverage an extensive corpus of unlabeled data to enhance the learning models trained using comparatively limited labeled examples. Compared with other semi-supervised learning methods, classical semi-supervised graph learning emphasizes the structural information of the graph. This focus enables a detailed exploration of the intrinsic relationships and dependencies among data points within the graph.

Based on the different data types within the graphs, classical semi-supervised graph learning can be categorized into node-level semi-supervised graph learning and graph-level semi-supervised graph learning. For the node-level task, the most representative methods are based on label propagation on the graph. For the graph-level task, the methods can be divided into consistency regularization and pseudo-labeling.

4.1.1 Label Propagation Methods↩︎

Label propagation (LP) is the most representative method for label inference on semi-supervised graph learning. The framework can be formulated as a propagation process in which some of the nodes with labels, also referred to as seeds, propagate their labels to unlabeled nodes based on the similarity of the connected nodes, which can be formulated as: \[\mathcal{L}=\mathcal{L}_{cls}(f(X), \mathcal{V}_L)+\lambda f(X)^{\top}\Delta f(X),\] where the first term \(\mathcal{L}_{cls}\) is the classification loss which trains the model \(f:\mathbb{R}^{N\times m}\) to predict the known labels \(\mathcal{V}_L\), \(m\) is the feature dimension. The second term is graph-based regularization, ensuring the connected nodes have a similar model output, \(\Delta\) is the graph Laplacian, \(\lambda\in\mathbb{R}\) is the regularization coefficient. For example, classical LP methods iteratively propagate the label of each data to its neighbors based on the constructed graph [42]. GCN [4] utilizes the power of graph neural networks and performs message passing on the graph to constrain neighborhood nodes with similar representations. GCN-LPA [11] further proposes to combine GCN and LP with learnable edge weights. The model views LP as regularization to assist the GCN in learning proper edge weights.

4.1.2 Consistency Regularization Methods↩︎

There are also some endeavors using consistency regularization for semi-supervised graph learning. These methods are based on the manifold or the smoothness assumption which posits that realistic perturbations of the graph data should not change the output of the model, and can be formulated as: \[\mathcal{L}=\mathcal{L}_s(f(X),\mathcal{G}_L)+\alpha\mathcal{L}_u(f(X))+\beta\mathcal{R}(f(X)),\] where \(\mathcal{L}_s\) and \(\mathcal{L}_u\) denote the supervised and unsupervised loss, \(R\) denotes the consistency regularization loss. Typically, InfoGraph [3] learns a supervised and an unsupervised model respectively and maximizes the mutual information between the two models. GraphCL [19] and GLA [43] leverage contrastive learning between a graph and its augmented views to learn graph representation for semi-supervised graph learning. TGNN [12] incorporates both graph neural network and graph kernels with consistency regularization loss to implicitly and explicitly explore graph structural knowledge from unlabeled data for semi-supervised graph classification.

4.1.3 Pseudo-Labeling Methods↩︎

Pseudo-labeling is another type of popular method for semi-supervised graph learning, which predicts the label distribution of unlabeled data and selects confident samples to the training data set as labeled data, which can be defined as: \[\mathcal{L}=\mathcal{L}_s(f(X),\mathcal{G}_L)+\alpha\mathcal{L}_s(f(X'),\mathcal{G}'_U),\] where \(X\) and \(X'\) denote the supervised and selected unsupervised graph data feature, \(\mathcal{G}_L\) and \(\mathcal{G}'_U\) denote the label of the original labeled graph data and pseudo label of the unlabeled graph data respectively. For graph-structured data, SEAL-AI [44] and ASGN [5] leverage active learning techniques to select the most representative graph samples from the unlabeled data. KGNN [45] adopts the posterior regularization to incorporate graph kernels as structured constraints, generating pseudo labels and guiding the training process of graph neural networks under the EM-style framework. DualGraph [46] further jointly learn a graph prediction and a graph retrieval module via posterior regularization during the pseudo-labeling process for semi-supervised graph learning.

4.1.4 Discussion↩︎

Label propagation methods represent nodes as training samples, and each edge denotes some similarity measurement of the node pair. In contrast, consistency regularization methods usually rely on the consistency constraint of rich data transformations and pseudo-labeling methods rely on the high confidence of pseudo-labels, which can be added to the training data set as labeled data. Besides, some hybrid works integrating different types of methods into one unified framework [47] can be adapted to graph data.

4.2 Semi-supervised Graph Learning under Domain Shift↩︎

In real-world scenarios, graph applications often face out-of-distribution (OOD) challenges, which arise when the data distribution during inference differs from the data on which the model was trained. Furthermore, the discrepancy between training and inference is complicated by the lack of labeled data in new domains, making supervised adaptation impractical. To address these issues, the concept of Graph Domain Adaptation (GDA) has been introduced to account for these distribution shifts and facilitate effective knowledge transfer. To achieve data efficiency, GDA must operate with minimal reliance on large amounts of labeled data, which are often scarce or expensive to obtain. It emphasizes the value of leveraging prior knowledge and discriminative features from the available data. Efficient GDA methods work to extend model adaptability with limited target domain data while maintaining robust performance despite rapid changes in data distribution.

Based on the data dependency, GDA can be categorized into unsupervised GDA, which does not require labeled data from the target domain, and source-free GDA, which performs adaptation without accessing the source domain data.

4.2.1 Unsupervised Graph Domain Adaptation↩︎

Unsupervised Graph Domain Adaptation (UGDA) uses both source and target graphs for training, and tests primarily on the target graphs. UGDA methods can be broadly categorized into the following four types.

Discrepancy-based methods. These methods usually apply discrepancy measurement, including MMD, Jensen–Shannon divergence, and Wasserstein distance, to measure the domain distribution shift. Formally, give source graphs \(\mathcal{G}^{s}\), target graphs \(\mathcal{G}^{t}\), and discrepancy measurement \(d\), we have \[\mathcal{L}_{dis} = d(\mathcal{G}^{s}, \mathcal{G}^{t})\,,\] When the domain discrepancy loss \(\mathcal{L}_{dis}\) is minimized, the knowledge can be transferred from the source domain to the target domain. Among them, UDAGCN [48] exploits local and global consistency within graphs and integrates an attention mechanism to fuse these consistencies into node representations. GRADE [49] introduces the graph subtree discrepancy measure to capture the distribution shift. SpecReg [50] proposes a tighter generalization bounded by spectral regularization. More recently, CoCo [13] has achieved superior performance by synergizing two branches, including the graph message passing branch and the graph kernel branch.

Adversarial adaptation methods. These methods typically utilize adversarial components to minimize domain discrepancies, where the graph encoder and domain classifier compete. Among them, AdaGCN [51] employs adversarial domain adaptation techniques to acquire node representations that are invariant across varying domains, enabling effective knowledge transfer. ACDNE [52] introduces adversarial learning across networks, in order to learn invariant node representations while preserving network structural information. DANE [53] is an adaptation method that attains domain-adaptive embedding through the utilization of a shared weight message passing network, complemented by adversarial learning regularization.

Discrimination learning methods. Researchers propose to utilize self-training to improve the performance in the target domain, where pseudo-labeling is one of the popular techniques. These methods obtain the pseudo-labels by the network itself on unlabeled target data to expand the training set. Among them, DEAL [54] uses augmented views to distill reliable pseudo-labels for better graph-level classification in the target domain. StruRW [55] introduces a structural reweighting approach that uses estimated edge probabilities based on the pseudo-labels to adjust the neighborhood bootstrapping of the graph neural network, counteracting conditional shifts between domains.

Disentangle-based methods. These methods generally disentangle graph representation into domain-invariant and domain-relevant parts, and conduct domain adaptation with domain-invariant embeddings. Among them, ASN [56] utilize both graph model embedding and adversarial adaptation to generate network-invariant node representations. DGDA [57] employs a variational graph auto-encoder approach to separate semantic, domain, and random latent variables for graph domain adaptation.

4.2.2 Source-free Graph Domain Adaptation↩︎

Source-free Graph Domain Adaptation (SFGDA) eliminates the dependence on source data. The assumption that source and target data can be used for adaptation is not always true in real-world scenarios. On the one hand, privacy, confidentiality, and copyright issues may prevent access to the source data. On the other hand, the requirement to store the complete source dataset on devices is often impractical. Among SFGDA methods, GT3 [58] utilizes a self-supervised test-time training framework, with a unique balancing constraint to prevent distribution bias. GTrans [14] introduces a data-centric method for target data transformation to improve generalization and robustness.

4.2.3 Discussion↩︎

Graph domain adaptation aims at achieving data-efficient learning under domain shift. UGDA methods learn by modeling between domains, using labeled source domain data and unlabeled target domain data. SFGDA focuses on an even more data-efficient and more challenging problem, while the research in SFGDA is not comprehensive, and ongoing research continues to advance the field.

5 Few-shot Graph Learning↩︎

Few-shot graph learning aims to learn graph models to make accurate predictions with a small amount of labeled data, which usually adopts a meta-learning paradigm. Compared with semi-supervised graph learning concerning incomplete supervision, few-shot graph learning focuses on transferring prior knowledge across different tasks with high generalization capacity. Specifically, we aim to build a graph model that is readily tuneable to fit future meta-test tasks given a variety of meta-training tasks. In this section, we primarily concentrate on the most popular few-shot node classification and cover numerous few-shot graph classification works as well. Current few-shot graph learning approaches can be roughly divided into metric learning approaches and parameter optimization approaches, which we will elaborate on as below.

5.1 Metric Learning Methods↩︎

Metric learning methods typically incorporate ProNet [59] into graph few-shot learning, which generates prototypes by averaging the node representations of GNNs in the support set for each class. Then, they encourage each query node to approach its corresponding prototypes. Formally, given each graph \(\mathcal{G}\) with \(N\) classes and a query set \(\mathcal{V}^q\) of \(K\) query nodes for each class, we have \[\mathcal{L}_{ML}=-\sum_{(v_i,y_i) \in \mathcal{V}^q} \log \frac{\exp \left(-d\left(\mathbf{z}_{i}, \mathbf{p}_{y_i}\right)\right)}{\sum_{c} \exp \left(-d\left(\mathbf{z}_{i}, \mathbf{p}_{c}\right)\right)},\] where \(\mathbf{p}_{c}\) denotes the representations for the \(c\)-th class, \(\mathbf{z}_{i}\) is the node representation of \(v_i\). \(d(\cdot,\cdot)\) is a distance metric, e.g., cosine distance. The episodic training would then be executed iteratively until convergence is achieved. In particular, GFL [60] integrates hierarchical graph representation gates to enhance node representations with structure-specific knowledge and employs a reconstruction loss to ensure stability during training. To improve the generalization capacity, GPN [61] estimates the importance of nodes for prototype reconstruction using a second GNN with additional adjustments based on centrality. HAG-Meta [15] additionally takes into account the scenarios involving newly encountered classes and introduces a regularization term based on task-level and novel-level attention to tackle the potential class imbalance issue. HGNN [62] combines prototype GNN with an instance GNN, which consists of all support nodes and the query, and encourages the consistency between these complementary GNNs. SuperClass [63] further computes prototype graphs for few-shot graph classification by minimizing the average distance to other similar graphs. Subsequently, it constructs a graph-of-graph to represent inter-class correlations for generalized graph learning.

5.2 Parameter Optimization Methods↩︎

Parameter optimization methods usually adhere to the paradigm of model-agnostic meta-learning (MAML) [64], [65], which generates node representations using GNNs for meta-learning. They construct meta-tasks using random nodes and then conduct multiple gradient descent for each meta-task. Formally, for each sampled meta-task \(\mathcal{T}_{i}\), given a GNN \(f_\theta(\cdot)\) and learning rate \(\lambda\), we have: \[\theta_{i}^{\prime}=\theta-\lambda \frac{\partial \mathcal{L}_{\mathcal{T}_{i}}\left(f_{\theta}\right)}{\partial \theta}.\] Finally, these tasks would be aggregated to optimize the task-agnostic parameters as a whole: \[\theta \leftarrow \theta-\beta \nabla_{\theta} \sum_{\mathcal{T}_{i} \sim p(\mathcal{T})} \mathcal{L}_{\mathcal{T}_{i}}\left(f_{\theta_{i}^{\prime}}\right),\] where \(\beta\) is the learning rate for meta-optimization. On the basis of the above optimization framework, numerous GNN approaches are developed. In particular, Meta-GNN [66] is the first work to combine meta-learning with GNNs and validate the effectiveness of GNNs in few-shot learning scenarios. AMM-GNN [67] further incorporates the attention mechanism to capture the relationships among meta-tasks for more generalized knowledge. GLITTER [68] extracts relevant nodes according to their influence to get rid of potential interference and then constructs a task-specific graph by leveraging mutual information and nodal influence, which makes the best of meta-tasks for effective meta-learning. In regard to few-shot graph classification, AS-MAML [69] analyzes the difficulty of applying MAML in graph domains and thus introduces an adaptive step controller that takes into account both training loss and the quality of graph embeddings to regulate the optimization procedure. HSL-RG [7] investigates the correlation between graph samples using graph kernels from a global perspective and introduces various augmentation strategies for self-supervised learning from a local perspective. The attention mechanism further integrates both complementary views into a task-adaptive meta-learning framework.

5.3 Discussion↩︎

In addition to the above works, several advanced techniques including prompt learning and contrastive learning have also been applied in few-shot graph learning [70]. Besides, these few-shot learning works have also been extended to tackle zero-shot graph learning [71], which focuses on nodes from unseen classes.

6 Conclusion and Future Work↩︎

In summary, this paper provides a comprehensive overview of Data-Efficient Graph Learning. We initiate by discussing the current challenges in graph machine learning. Subsequently, we categorize existing works into three parts: self-supervised graph learning, semi-supervised graph learning, and few-shot graph learning. In each section, we present representative strategies and introduce their key technical contributions. Despite the progress made, there still remain several challenges in the field that warrant further research in the future.

6.0.0.1 Inherent Challenges.

Though being efficient in general, the data-efficient graph learning models are inherently facing challenges such as robustness and generalizability.We might also face extra challenges when running data-efficient models on out-of-distribution data. Some real-world applications prefer explainable results, so that the model can be trusted. Drug discovery is a good example. All these considerations not only enhance the capabilities of existing data-efficient graph learning models but also steer the trajectory of research towards more versatile and reliable solutions for real-world scenarios.

6.0.0.2 Combined with Large Models.

Recently, combining LLM (i.e., Large Language Models) with graph learning approaches is one of the trending topics [72]. For example, one might use GNN components to learn the underlying knowledge graphs to improve LLMs for question-answering [73] or use LLMs to generate node embeddings in social network graphs [74]. For data-efficient graph learning, we care about all components. Both the LLM-based components and GNN-based components are crucial for achieving effectiveness and efficiency. It is theoretically possible to achieve high performance with only few-shot or zero-shot learning on these relatively-complicated models.

6.0.0.3 Towards Different Convolutional Algorithms.

There are some GNN models that are fundamentally different from most of the others. Therefore, data-efficient learning approaches that work elsewhere might no longer be applicable. For example, some GNNs are designed in non-Euclidean space. The non-Euclidean-space models (e.g., spherical or hyperbolic) are well-known for being good at handling certain substructures, such as trees or cycles. Researchers are still on their way to extend GNN models to non-Euclidean spaces, while believing it to be a promising direction. The data-efficient design in Euclidean space might no longer work for non-Euclidean space, and there might be data-efficient non-Euclidean GNN models that can not work in Euclidean space, highlighting the continuous evolution and diversification of GNN research.

6.0.0.4 Proof of Efficiency.

In theory, data-efficient learning has the potential to be rigorously justified by mathematical proofs. So far, with the help of learnability theory, researchers have made strides in proving some theoretical bound under certain problem settings [75]. However, so far, the majority of works in this field focus on fully-connected neural networks, together with fully-labeled data. If mathematical analysis could be provided in the data-efficient graph learning setting, it would undoubtedly provide valuable insights for future researchers, paving the way for more robust and informed advancements in this evolving domain.

Acknowledgments↩︎

This paper is partially supported by the National Natural Science Foundation of China (NSFC Grant Numbers 62306014 and 62276002) as well as the China Postdoctoral Science Foundation with Grant No. 2023M730057.

References↩︎