1 Introduction↩︎

Prompt tuning [1], [2] has driven many recent developments for Large Language Models (LLMs). These methods optimize external prompting parameters to guide a model’s responses. The goal is to avoid fine-tuning the model’s vast number of internal parameters for adaptation to new downstream tasks, keeping the core pre-trained knowledge intact [3]–[5]. However, prompting becomes more challenging for graphs because: First, there is a wide range of graph tasks; thus, unlike general language tasks [6], [7], task unification is difficult in graphs. This hinders the collection of large-scale data, e.g., from the internet [8], creating bottlenecks in pre-training large Graph Neural Networks () for general-purpose use. Second, because of the limited reasoning capabilities of [9], [10], it is non-trivial to constitute instructive graph prompts in human (natural) language [11]. Thus, prompting should be understood as a parameter-efficient adaptation paradigm inspired by LLM prompting, rather than as evidence that current possess LLM-like zero-/few-shot reasoning abilities.

A few recent studies have adopted prompting for to align the objectives of pre-training on source data and fine-tuning on target data, mostly following the “pre-train, prompt, fine-tune” pipeline [12]. These works design unified tasks that allow optimizing a with semantically similar objectives on the pretext and downstream tasks [13]–[18]. However, the current prompting paradigm suffers from two key limitations that hinder its efficiency relative to LLMs. First, existing methods rely heavily on labeled data — costly to obtain — to achieve competitive performance. Second, they require training new projection heads for each downstream task, a form of lightweight fine-tuning [1]. These dependencies on additional parameters and labeled data (especially in scenarios where the original source data is inaccessible, e.g., due to privacy) prevent from being used as truly frozen models. This gap motivates our work to establish a more practical and efficient prompting paradigm for .

To directly address these limitations, we first introduce the Unsupervised Graph Prompting Problem (), a rigorous novel problem formulation. The setup evaluates a method under four key conditions: the ’s parameters are frozen, there is a covariate shift in the target data distribution, no target labels are available for adaptation, and the source data is inaccessible. While this setup shares similarities with Unsupervised Source-Free Domain Adaptation (SFDA) [19], it differs in a crucial aspect: requires the entire source-trained to be frozen, whereas SFDA methods [20], [21] rely on fine-tuning the model’s parameters. This setting places our work within the paradigm of parameter-efficient prompting, rather than full model adaptation.

Within this challenging setup, we propose , the first fully unsupervised prompting framework. trains a prompting function using consistency regularization and confident pseudo-labeling, enabling the frozen to adapt its knowledge to the new target distribution. To ensure robustness, we introduce two additional regularization techniques: one to counteract prediction bias from class imbalance and another to make the prompted graphs close to the original data distribution. Our extensive experiments show that , despite being fully unsupervised, consistently outperforms state-of-the-art (SOTA) prompting methods that have the advantage of full access to labeled data. Our major contributions are summarized as follows.

• Problem formulation. We propose , a novel problem setup that isolates the true effectiveness of a prompting function by disallowing any updates to the source-trained ’s parameters.

• Novel unsupervised methodology. We propose , the first fully unsupervised prompting method that leverages consistency regularization and pseudo-labeling to adapt a frozen to new data distributions.

• Empirical analysis. We demonstrate that substantially outperforms supervised SOTA methods on node and graph classification tasks, validating the effectiveness of unsupervised adaptation in this novel and more practical setting.

2 Background & Problem Formulation↩︎

Previous studies on adapting prompt tuning for use lightweight fine-tuning [1], [2] with supervision, which has been addressed widely in different domains such as computer vision [22]–[24] and NLP [6], [25]. Specifically, recent works have followed the “pre-train, prompt, fine-tune” setup [12], [14]–[16]. In this section, we first introduce the setting and discuss its limitations, and then present our proposed problem setting, which addresses these limitations.

2.1 Pre-train, Prompt, Fine-tuning↩︎

This pipeline aims to bridge the generalization gap between pre-trained and downstream tasks that differ semantically from the pretext tasks. Unlike the traditional supervised and “pre-training, fine-tuning” methods, this pipeline employs a task unification step before pre-training and fine-tuning. This is essential because it aligns the pretext and downstream objectives to optimize a pre-trained model on new datasets. The steps are as follows.

(1) Pre-train. Formally, given a set of tuples \(\mathcal{S} = \{(\task_i, \data_i)\}^{N_s}_{i=1}\), with \(N_s\) samples, from pretext task \(\task_i\) and dataset \(\data_i\), first all the tasks are unified to task \(\task_u\) and the corresponding changes apply for their associated datasets to make a new set \(\mathcal{S}_u = \{(\task_u^i, \data_u^i)\}^{N_s}_{i=1}\). Then a encoder \(g(.;\theta_g)\) is pre-trained on \(\mathcal{S}_u\) using an unsupervised approach such as contrastive learning [26], [27]. For downstream tasks, by adding a projection head \(h(.; \theta_h)\) after the encoder, a model \(\psi = h \circ g\) is formed, of which the encoder parameters \(\theta_g\) are frozen, and only the head parameters \(\theta_h\) will be trained. (2) Prompting. At this stage, a prompting function \(f(., \theta_f)\) is employed to construct a prediction model \(\varphi\). The prompting function is either a prefix module, i.e., \(\varphi = h \circ g \circ f\), or a postfix one, i.e., \(\varphi = f \circ h \circ g\). (3) Fine-tuning. In the final step, the set of parameters \(\{\theta_h, \theta_f\}\) of \(\varphi\) are optimized for every unified downstream task \(\task_u\) with the new labeled samples.

Limitations. A deeper look at the “pre-train, prompt, fine-tune” pipeline reveals that although the current methods impose less trainable parameters compared to full fine-tuning (fine-tuning both pre-trained feature encoder and decoder), they involve partial (lightweight) fine-tuning [1], [2], [4], as they also train the ’s decoder (projection head) along with the new parameters of the prompting function. Subsequently, labels become essential for this partial fine-tuning, and they are unable to leverage unlabeled data from large datasets when collecting labeled data is challenging [8]. Also, fine-tuning a large pre-trained model on large datasets may introduce noisy information when the labeled downstream datasets are small [28], [29]. Therefore, it reduces model generalization across diverse applications [30].

2.2 Our Problem Setting↩︎

To address the above limitations, we first need a suitable problem setting that offers insights into how well a prompting method performs when there is lack of labeled data. It is also crucial to evaluate if the method generalizes across tasks without fine-tuning the base model parameters.

2.2.0.1 Unsupervised Graph Prompting Problem (UGPP).

Suppose a model \(\varphi(.; \theta_g, \theta_h) = h(.; \theta_h) \circ g(.; \theta_g)\) is given, where \(g\) and \(h\) are its encoder and decoder. Also, \(\varphi\) is trained for task \(\task\) on a labeled source dataset \(\data_s = \{(x_s^i, y_s^i) : x_s^i \sim \prob^s_X,\;y_s^i \sim \prob^s_{Y|X}\}_{i=1}^{Ns}\), where \(x_s^i\) is a sample (e.g., a graph or node), and \(y_s^i\) is its associated label. The problem is to train a prompting module \(f(.; \theta_f)\) on an unlabeled target dataset \(\data_t = \{x_t^j: x_t^j \sim \prob^t_X\}_{j=1}^{N_t}\), s.t. \(\prob^s_X \neq \prob^t_X\), to enhance the performance of \(\varphi\) for task \(\task\) on \(\data_t\), assuming that \(\theta_g\) and \(\theta_h\) are fixed, \(\data_s\) is unobservable, and \(\prob^t_{Y|X}\) remains invariant across domains, i.e., \(\prob^t_{Y|X} = \prob^s_{Y|X}\).

This problem, Ugpp, focuses on prompting without labeled data. A prompting method that performs well in this setting has three advantages: First, it is model- and task-agnostic, i.e., it works for conventional (e.g., GCN, GAT) and tasks (e.g., node/graph classification). Second, it is unsupervised, allowing the use of many datasets to improve generalization. Third, this setting does not depend on the source data. This is particularly beneficial when the source data is inaccessible, e.g., due to privacy issues. Throughout the paper, we use “source-trained” to refer to the model trained on the labeled source domain for task \(\task\), and reserve “pre-trained” for the general prompting literature.

It is noteworthy that can be viewed as a restrictive prompt-based instance of Unsupervised SFDA. Conventional SFDA methods typically fine-tune the source-trained model, whereas keeps the source-trained , including both the encoder and the prediction head, entirely frozen. Therefore, all adaptation is isolated in the external prompting function. This restriction allows us to evaluate the effect of prompting without entangling it with the base model’s latent fine-tuning, and it also makes the setting model-agnostic. Please find additional discussions on the definition in Appendix 6.3 We also compare our proposed method against recent graph SFDA methods in Appendix 6.5.4

3 Our Method:↩︎

Here, we introduce , an unsupervised prompting method, to address .
Motivation. We aim to design an unsupervised prompting framework that helps a source-trained make robust predictions while its parameters are frozen. To achieve this, we take advantage of pseudo-labeling [31], [32] and consistency regularization [33], [34]. To train a prompting function in a fully unsupervised manner, first obtains randomly augmented graphs from the target dataset. Then we employ consistency regularization [35], [36], where certain predictions of the base are filtered by a confidence threshold and exploited as pseudo-labels for optimization. Indeed, knowledge from the source dataset is used to reduce prediction entropy and handle distribution shifts by leveraging ’s confident predictions for unlabeled target data. The prompting module enhances target input samples by adding key information, making them more similar to the source samples. Empirical evidence to support this claim is in the Appendix 6.6.1. We describe our framework (Figure 1) for graph-level tasks. Note that it also generalizes to node- and edge-level tasks.

Overview of .Our framework to address has a training and an inference step. The training step involves two components: consistency-based prompting and prompt regularization. Consistency-based prompting starts by augmenting each input graph twice, with one of the augmentations modified by a prompting function. Next, the scores both samples. The objective is to train the prompting function such that the produces “consistent prediction scores” for both samples of this pair with a certain confidence. In prompt regularization we introduce two regularization techniques: one to tackle biased predictions caused by a class imbalance in the data and another to prevent generating out-of-distribution (OOD) prompted graphs. During inference, a test graph is fed to the prompting function without augmentation, and its output goes to the . We discuss a task unification step that generalizes to other graph tasks, e.g., node classification, in Appendix 6.4.3.

3.1 The Consistency-based Prompting↩︎

Our prompting method is designed to reduce the discrepancy of the base predictions over random augmentations of the same input graphs. We achieve this without labels by first augmenting the data with an algorithmic step, then generating pseudo-labels from the unlabeled augmentations when their assigned scores by the meet a certain confidence threshold. We provide the details of these two stages below.

Algorithmic augmentation. Consistency regularization techniques [35] train strong augmentations of samples using the pseudo-labels derived from their weak augmentations. Since our focus is on optimizing the prompting module rather than fine-tuning the base , we adopt this technique as follows. We use a random non-parameterized augmentation algorithm (we use random feature masking in our experiments). More specifically, we mask a group of features with a certain probability. We augment an input graph \(\graph\) to create a weak augmentation \(\graph_w\) with masking probability \(p_w\) and also a strong augmentation \(\graph_s\) with probability \(p_s\), where \(p_s > p_w\). We pass \(\graph_s\) through a learnable prompting function \(f(.; \theta_f)\) to obtain a prompted graph \(\graph_p=f(\graph_s; \theta_f)\). We keep \(\graph_w\) unchanged and call it a non-prompted augmentation graph.

Learnable prompting. We use a prefix prompting module to transform input samples for the base . Our approach is generic enough to allow the integration of different prompting functions. For our experiments, we choose a function \(f\) that enriches node feature vectors, as this design aligns with our random feature masking augmentation technique. This way of adding learnable parameters is used in GPF-Plus [15]. We discuss the choice of input-level prompt tuning and its relation to other parameter-efficient adaptation methods such as LoRA [37], in Appendix 6.6.6.

Specifically, we learn a prompting function \(f\) with parameter set \(\theta_f=\{t^*_j: t^*_j \in \realset^d\}_{j=1}^{n_t}\). For input graph \(\graph\) of \(N\) nodes with nodes features set \(X = \{x_i: x_i \in \realset^d\}_{i=1}^N\), the function \(f\) makes prompted graph \(\graph_p\) with feature set \(X_p = \{x_i+t_i: x_i \in X, \;x_i, t_i\in \realset^d\}_{i=1}^{N}\) such that \(t_i = \sum_{j=1}^{n_t}\alpha_{i,j}t^*_j\) and \(\alpha_{i,j}=\frac{\exp(x_i^Tt^*_j)}{\sum_{l=1}^{n_t}\exp(x_i^Tt^*_l)}\) .

Consistency-based Objective. We optimize \(\theta_f\) to minimize the discrepancy between the ’s prediction scores for the non-prompted augmented graph \(\graph_w\) and the prompted graph \(\graph_p\). This would lower the entropy of the ’s scores for the target unlabeled data. Intuitively, a well-trained frozen model \(\varphi\) makes accurate predictions for samples close to the source distribution. Therefore, as training proceeds and \(\varphi\) scores different random perturbations of the same samples, we utilize its confident predictions as pseudo-labels for optimization, as this helps \(f\) to capture the distribution shift and make the predictions robust. We achieve this by passing \(\graph_p\) and \(\graph_w\) to \(\varphi\) for prediction as:

\[\begin{align} \label{eq:predscores} \tilde{\mathbf{p}}^{\graph}_{\varphi} &= \delta(h(z^{\graph}_a; \theta_h))\hat{\mathbf{p}}^{\graph}_{\varphi} = \delta(h(z^{\graph}_p; \theta_h)) \end{align}\tag{1}\]

where \(z^{\graph}_a = g(\graph_w; \theta_g)\) and \(z^{\graph}_p = g(\graph_p; \theta_g)\), \(\delta(.)\) denotes the softmax function, \(\tilde{\mathbf{p}}^{\graph}_{\varphi}, \hat{\mathbf{p}}^{\graph}_{\varphi} \in \realset^C\), and \(C\) is the number of classes. Pseudo-labels are made as \(\tilde{y}^{\graph}_{\varphi} =\mathop{\mathrm{arg\,max}}\tilde{\mathbf{p}}^{\graph}_{\varphi}\), and finally, the consistency loss is: \[\begin{align} \label{eq:consistencyloss} \loss_c = \frac{1}{|\mathcal{B}|}\sum_{\graph \in \mathcal{B}}\mathbb{1}(\max(\tilde{\mathbf{p}}^{\graph}_{\varphi}) > \tau)CE(\tilde{y}^{\graph}_{\varphi}, \hat{\mathbf{p}}^{\graph}_{\varphi}) \end{align}\tag{2}\]

where CE(.,.) is the cross-entropy function, \(\mathcal{B}=\{\graph_i\}_{i=1}^{|\mathcal{B}|}\) is a sample batch, and \(\tau\) is a confidence threshold. \(\tau\) excludes low-certainty predictions (samples less aligned with the source data distribution) and can be fixed or class-dynamic (see Appendix 6.4.3).

Unlike standard semi-supervised adaptation, ’s use of consistency regularization is novel in that we apply it in a stricter frozen-prompting setting, allowing only learning an external prompt that serves as a strong augmentation to align target samples with the source-trained model.

3.2 Prompt Regularization↩︎

Diversity. Due to the class imbalance, only reducing the consistency loss may cause biased predictions and trivial solutions such that every sample is assigned to the same class to reduce the overall entropy. To mitigate this, inspired by [38], we regularize the model to maximize the entropy of the scores’ expected value over a batch, and encourage diverse predictions. \[\begin{align} \label{eq:diversityloss} \loss_{div} = -H(\hat{\mathbf{q}}) = \mathbf{1}^\top(\hat{\mathbf{q}}\odot\log\hat{\mathbf{q}}); \;\hat{\mathbf{q}} = \frac{1}{|\mathcal{B}|}\sum_{\graph \in \mathcal{B}} \hat{\mathbf{p}}^{\graph}_{\varphi} \end{align}\tag{3}\]

Here \(H(.)\) is the entropy function, and \(\odot\) is the Hadamard product. Employing consistency regularization with an adjusted confidence threshold (\(\tau\)) and integrating a diversity loss (\(\mathcal{L}_{div}\)) to prevent class collapse helps our framework address pseudo-label dependency and potential calibration risks.

Domain Adaptation. While the prompting function \(f\) minimizes the discrepancy of predictions for the same sample, it may also create OOD prompted graphs. Since the source data are unavailable, we cannot directly align prompted representations with the source distribution. Instead, we can exploit the knowledge learned from source samples that is preserved in the frozen parameters \(\theta_{\varphi}\) to mitigate the OOD issue. To achieve this, we train an adversarial discriminator \(d(.; \theta_d)\)—e.g., a simple feedforward network with trainable parameters \(\theta_d\)—to distinguish a prompted graph \(\graph_p\) from a non-prompted augmented graph \(\graph_w\). Formally, we optimize the discriminator as follows:

\[\begin{align} \label{eq:discloss} \theta_d^\star = \underset{\theta_d}{\mathop{\mathrm{arg\,min}}} &- \frac{1}{2|\mathcal{B}|}\sum_{\graph \in \mathcal{B}} [\log \sigma(d(z_a^{\graph}; \theta_d)) + \log (1-\sigma(d(z_p^{\graph}, \theta_d)))] \end{align}\tag{4}\]

where \(\sigma(.)\) is the sigmoid function. We normalize the sum by \(\frac{1}{2|\mathcal{B}|}\) since every graph has two samples. Nevertheless, we regularize \(f\) with the following objective to make \(g\)’s representations for prompted graphs closer to the non-prompted augmented graphs.

\[\begin{align} \label{eq:domainloss} \loss_{adv} = - \frac{1}{|\mathcal{B}|}\sum_{\graph \in \mathcal{B}} \log \sigma(d(z_p^{\graph}; \theta_d^\star)) \end{align}\tag{5}\]

The adversarial discriminator regularizes the prompted representations to stay close to their non-prompted weak augmentations, preventing the prompt from moving them to arbitrary regions of the latent space. Thus, \(L_{\mathrm{adv}}\) should be interpreted as a target-side stability regularizer, while source compatibility is mainly induced by the frozen model and the confidence-filtered consistency objective. We provide an MMD-based diagnostic supporting this interpretation in Appendix 6.7.

3.3 Final Objective & Complexity Analysis↩︎

Our unsupervised objective approach involves three parts. Eq. 2 encourages consistency across predictions to handle distribution shift and exploit the ’s learned knowledge from source data. Eq. 3 and Eq. 5 handle class imbalance and avoid generating OOD prompted graphs, respectively. The final objective to optimize \(\theta_f\) becomes:

\[\begin{align} \label{eq:totalloss} \theta_f^\star = \underset{\theta_f}{\mathop{\mathrm{arg\,min}}} \;\loss; \qquad \loss = \loss_c + \lambda_1 \loss_{div} + \lambda_2 \loss_{adv} \end{align}\tag{6}\]

where \(\lambda_1\), \(\lambda_2\) are hyper-parameters. We bring empirical evidence supporting the logic behind adding each of the regularization objectives terms in Appendix 6.6.2.

During inference, the augmentation step is skipped, allowing the prompting module to produce a prompted graph directly from an input graph to align it with the trained model’s knowledge. This prompted graph is passed to the trained for prediction. The pseudocode of for both the training and inference is presented in Algorithms 3 and 4 of the Appendix.

The time complexity of a regular (e.g. GCN), is \(O(NLd^2 + L|E|d)\), where \(N\),\(E\), \(L\), and \(d\) are the number of nodes, edges, layers, and the dimensionality of node embeddings respectively. A common graph augmentation algorithm, like feature masking, requires \(O(Nd)\) operations. The complexity of the prompting method used in our experiments is \(Ndn_t\), where \(n_t\) is the number of trainable prompting vectors. Thus, the overall complexity is \(O(NLd^2 + L|E|d + Ndn_t)\).

We compare wall-clock training and inference time against the baselines in Appendix 6.6.6 (Table ¿tbl:tab:time95comparison?). has slightly higher training cost due to consistency learning and the discriminator, but this overhead is only used during training. At inference time, the augmentation pipeline and discriminator are removed, and the model only applies the lightweight prompting function before the frozen . Therefore, remains close to the BaseModel inference cost while adding only a small number of trainable prompt parameters and a lightweight discriminator during training.

4 Experiments↩︎

Datasets and Code. We experiment on six standard datasets for graph and node classification. We use ENZYMES [39], PROTEINS [40], DHFR [41], BBBP and BACE [42] datasets for graph classification, which have continuous or discrete features. For node classification, we use Cora, CiteSeer, PubMed [43], Flickr [44], Cornell, Texas, Wisconsin [45]. Please find more details in Appendix 6.4.1.

Distribution Shift in Datasets. Our problem definition () requires evaluating a source-trained on target datasets that exhibit a covariate shift from the source data. To implement this, our main experiments induce shifts based on fundamental graph properties. For graph classification, we generate datasets with varying edge homophily ratios [46], a property known to intrinsically affect information aggregation [47]. For node classification, we use PageRank [48] to create a popularity-based shift, which provides a challenging evaluation scenario. Further details on the generation of these distributions are in Appendix 6.4.2, and additional experiments on other shifts (e.g., graph density, clustering coefficient) are in Appendix 6.5.2.

Evaluation Setting. All datasets are split in half to create source and target sets with shifted distributions. We train a base on the source set and evaluate it on the target set. Since baseline prompting methods rely on supervised training, we allow only the baselines to access labeled data for training in four setups of 25%, 50%, 75%, and 100% (full supervision). Please find experiments on 50%, 75% and 100% labeled data in Appendix 6.8.1. We report the F1-score and the relative improvement in F1-score (IMP%) compared to the BaseModel baseline.

Baselines. We consider several types of baselines.
(1) BaseModel. The base without prompting and fine-tuning, which is expected to be outperformed by prompting methods. We use GCN [49] and GAT [50] as the base . More experiments with recent advanced are in Appendix 6.5.3.
(2) Fine-Tuning. The base model when we fix its encoder and just fine-tune its projection head. The goal is to verify the claim that fine-tuning on new dataset with labels does not necessarily improve performance [12], [15], [16].
(3) Prompting Methods. Our work is the first attempt for graph prompting without labels and updating the base ’s parameters. Thus, we compare with all the SOTA prompting methods used in the recent benchmark [51], namely All-In-One [16] and GPF-Plus [15], GraphPrompt [14], GraphPrompt+ [52], and GPPT [12]. Also, we have compared our method against additional baselines, using GCN as the BaseModel’s architecture, in Tables ¿tbl:tab:few95shot95comparison95graph? and ¿tbl:tab:few95shot95comparison95node? of the Appendix. We use the codebases from the corresponding papers.

We do not allow these methods to update the ’s parameters and only their prompting modules are supposed to be learned on the target dataset. This restriction follows the setting, where the goal is to isolate the effect of the prompting function while keeping the entire source-trained model frozen. We note that this is more restrictive than the original setting of the supervised prompting baselines, which often fine-tune a task-specific prediction head. Therefore, in Appendix 6.6, we additionally evaluate these baselines under a relaxed setting where their decoders are fine-tuned with labeled target data.

4.1 Results on Graph Classification↩︎

For graph classification, 50% of graphs are randomly sampled as the source dataset, with higher-homophily graphs having a greater chance of selection; the remaining 50% form the target dataset. Please see experiments with graph density distribution shift in Table ¿tbl:tab:graphgcndensity? (see the Appendix). The base is trained on the source dataset. We repeat the experiments with two base (GCN and GAT) to demonstrate how the models generalize across different architectures. Note that GPPT is limited to node classification, so we exclude it from this experiment.

Table ¿tbl:tab:graphgcn25? presents graph classification results, where baselines use 25% labeled data, while our method, , uses 0%. Two key observations highlight ’s contribution: first, it consistently surpasses the BaseModel, validating it as a reliable, non-detrimental prompting method. Second, and most notably, ’s use of no labels offers broad applicability to diverse unlabeled datasets, marking a step towards graph foundation models. Interestingly, evaluations with setting reveal that baselines often fail to improve performance and sometimes make it worse. Most of the graph prompting methods, except for GPF-Plus, perform poorly with both architectures. Although GPF-Plus has the same prompting function as ’s, it struggles with adapting to distribution shifts. Conversely, leverages source data knowledge and generates pseudo-labels from highly confident predictions, learning effectively from samples that closely match the source distribution. This ensures consistent improvement across all cases.

4.2 Results on Node Classification↩︎

First, we compute PageRank (PR) for all nodes of each dataset. We sample 50% of nodes from the source dataset based on the normalized PR, so that graphs with higher PR are more likely to be included. The rest are assigned to the target dataset. A 2-hop neighborhood of each node is extracted as a subgraph for task unification to graph classification, and it inherits the main node’s label. More experiments with the distribution shift of the type clustering coefficient are provided in Table ¿tbl:tab:nodegcncc? (see the Appendix).

Node classification results are in Table ¿tbl:tab:nodegcn25?. outperforms on all datasets except PubMed (where it is second-best) and, notably, uses no labeled data, unlike the baselines. Additionally, the ’s performance degrades on target data across all datasets after fine-tuning its projection head (the Fine-Tuning baseline) with 25% of labels. This verifies that fine-tuning a model on a small-sized labeled dataset may introduce noisy information when the downstream data distribution does not align with the model’s learned knowledge [28]–[30]. We also verify that maintains high performance on the source domain and does not cause forgetting of learned knowledge, unlike the other baselines; please find the experiments in Appendix 6.6.3.

An important advantage of an unsupervised method—such as ours—is that it allows the use of large-scale unlabeled datasets. It is important to emphasize that achieves the best results on Flickr, the largest dataset, and second-best with large margins over other baselines on PubMed, the second-largest dataset, indicating that can perform well on large datasets.

Although supervised learning usually provides more informative gradients, this is not necessarily an upper bound in the setting. Since the source-trained is frozen and the target data are shifted, adaptation mainly requires aligning target samples with the source-trained model. uses confidence-thresholded pseudo-labeling and therefore optimizes the prompt only from target samples on which the source-trained model is already certain, reducing the impact of highly shifted samples. Moreover, allows the use of the full unlabeled target set, whereas supervised baselines are limited to training on the labeled subset. The few-shot results in Section 4.4 further show that labels improve when they are used together with our prompt-based alignment.

Additionally, we evaluate the supervised prompting baselines in their more relaxed setting with fine-tuned decoders and 25% labeled target data in Appendix 6.6; remains more stable under distribution shift despite using no labels and keeping the full base model frozen.

4.3 Ablation Study↩︎

4.3.1 The Effect of Regularization↩︎

To evaluate the regularization effect, is trained in four scenarios: (1) without regularization (“w/o”), (2) with only domain adaptation regularization (“domain”), (3) with only diversity regularization (“diversity”), and (4) with both regularizations (“domain + diversity”). Settings (2), (3), and (4) are compared to (1), with Figure [fig:losses] showing IMP improvements. During evaluation, if validation performance indicates that a specific regularization term fails to improve the model, its corresponding weight (\(\lambda_1\) or \(\lambda_2\) in Equation 6 ) is set to zero, and the test IMP for that term resolves to zero. Results show that both regularization factors have positive effects across most datasets; however, their combination is not always superior to their individual applications.

Domain adaptation regularization is beneficial across all graph classification datasets. We conjecture that the distributions of node classification datasets are more likely to be densely populated, whereas graph datasets often exhibit scattered hollow spaces in the latent space between classes, increasing the likelihood of generating OOD-prompted graphs. Therefore, domain adaptation regularization would be more beneficial. Moreover, a key finding of this experiment is the importance of adding diversity regularization. More specifically, when we have severe class imbalance, for example, in PROTEINS, Cora, and CiteSeer, “diversity” empowers the base significantly, which supports our claims. Appendix 6.6.2 provides more discussion on the effect of these objectives.

4.3.2 The Effect of Augmentation↩︎

We mask node features in weakly augmented graphs (\(\graph_w\)) for pseudo-labeling with probability \(p_w\) and in strongly augmented graphs \(\graph_s\) for prompting with probability \(p_s\). Since \(\graph_s\) resembles a distribution shift from \(\graph_w\), a higher \(p_s\) enables learning from distribution shifts across more feature groups, potentially offering greater robustness. Here, we fix \(p_w = 0.1\) and evaluate the impact of varying \(p_s\). The IMP for all \(p_s\) values is compared to no augmentation \(p_s = 0\). The results in Table ¿tbl:tab:augratio? support our intuition that augmenting, confidence pseudo-labeling, and consistency training are advantageous. Higher \(p_s\) yields better performance over PROTEINS, Cora, CireSeer, and PubMed because augmentation by feature masking—replacing a group of feature values with 0.0—is semantically more aligned with the discrete and binary features of these datasets; however, for ENZYMES and DHFR with continuous features, this augmentation is not as effective and higher values of \(p_s\) may be detrimental and cause learning from noise. Additional experiments on evaluating the effect of augmentation type (e.g. modifying graph structure) are in Appendix 6.6.7. Besides, we show the versatility and generalization of our framework in Appendix 6.6.6 using All-In-One’s prompting function as a choice of function \(f\) (see Section 3.1). Also, additional diagnostics on pseudo-label accuracy, calibration, and sensitivity to confidence threshold (\(\tau\)) are provided in Appendix 6.8.

4.3.3 The Effect of Homophily Shift↩︎

To evaluate how performance scales with the increase in covariate shift (or difficulty), we extend our experiment by constructing explicit degrees of shift. Instead of a random 50–50 split (as in the original setting), we sort all samples by homophily (for the graph classification dataset) and PageRank (for node classification datasets). The top 50% (highest homophily) are used as the source dataset. The remaining 50% are divided into four target sets with increasing difficulty (amount of covariate shift with respect to source set): (i) Set 1: 50–67.5 (closest to source), (ii) Set 2: 67.5–75, (iii) Set 3: 75–87.5, and (iv) Set 4: 87.5–100 (lowest homophily). Table 1 shows the results. maintains strong gains on the easier shifts (Sets 1 & 2) and continues to deliver improvements even under severe distribution shift (Sets 3 & 4). In contrast, GPF-Plus is worse and frequently induces negative transfer (negative IMP%).

4.4 Few-shot Learning for↩︎

is an unsupervised prompting method. However, it can potentially utilize labeled data efficiently when it is available. Assuming every batch \(\mathcal{B}\) is composed of a set of labeled samples \(S_l\) and unlabeled samples \(S_u\) s.t. \(\mathcal{B}=S_l\cup S_u\), we can replace \(\loss_c\) in Equation 2 by \(\loss_c = \frac{1}{|\mathcal{B}|}(\loss_l + \lambda_3\loss_u)\) in which \(\loss_l = \sum_{\graph \in S_l} CE(y^{\graph}, \hat{\mathbf{p}}^{\graph}_{\varphi})\) is the supervised objective term and \(\loss_u = \sum_{\graph \in S_u}\mathbb{1}(\max(\tilde{\mathbf{p}}^{\graph}_{\varphi}) \geq \tau)CE(\tilde{y}^{\graph}_{\varphi}, \hat{\mathbf{p}}^{\graph}_{\varphi})\) is the unsupervised term.

Here we evaluate in a few-shot setting. Table ¿tbl:tab:few95shot? presents the results using GCN as the base under homophily and PR distribution shifts. significantly performs better when labels are provided. For node classification datasets, IMP in the 25% labels setting is notably higher than in the unsupervised case (0% labels), while improvements on PROTEINS and DHFR are smaller. Meanwhile, performance on ENZYMES remains comparable in the absence of labels, likely due to a significant covariate shift, which makes learning from highly heterophilic data challenging even with labels. The key takeaway emerges by comparing Table ¿tbl:tab:few95shot? with Tables ¿tbl:tab:graphgcn25? and ¿tbl:tab:nodegcn25?, showing that with 25% of labeled data, outperforms all baselines on most datasets except DHFR. Notably, its superiority is also evident in the 0% label setting.

4.5 Exploratory Robustness Beyond the UGPP Assumption↩︎

The formal setting assumes covariate shift with a shared conditional label distribution. Here, we provide an exploratory stress test beyond this assumption. The goal is to evaluate whether prompting can still improve a frozen model when the target categories are semantically disjoint from those used to train the head. We use the Cornell, Texas, and Wisconsin datasets, which share the same feature dimensionality and number of classes. For each dataset, the base is trained only on source samples from classes \(\{0,1,2\}\), and then the base model is frozen. We then prompt and evaluate it on samples from classes \(\{2,3,4\}\). Since the projection head remains frozen with output dimension \(C=3\), the target labels are reindexed to \(\{0,1,2\}\) only for computing evaluation metrics, and no classifier weights are updated.

Table ¿tbl:tab:exp195label95shift? summarizes the findings. The BaseModel collapses under this category-disjoint stress test, showing that the original semantic mapping of the frozen head is no longer reliable. Nevertheless, recovers substantial performance across all datasets, outperforming GPF-Plus, which occasionally leads to negative transfer.

Our second experiment is designed to evaluate robustness to real-world covariate shift. We consider a more extreme scenario than the simulated covariate shift discussed before. This setting is standard (particularly in transfer learning) and more challenging. Here, we train the base on a source dataset, such as Cornell, and then prompt and evaluate it on another dataset, such as Texas, which is denoted as Cornell \(\rightarrow\) Texas. Results are shown in Table 2. consistently delivers positive transfer, whereas GPF-Plus potentially produces negative transfer. This demonstrates ’s robustness even under severe shifts.

5 Conclusions↩︎

In conclusion, we have introduced , a novel unsupervised prompting framework for that overcomes the limitations of existing prompting methods, particularly in scenarios where labeled data is unavailable. eliminates the need for updating the base ’s parameters on new downstream tasks. also enhances the generalization of the base source-trained without supervision. Experimental results over various datasets validate the effectiveness of which outperforms the state-of-the-art prompting methods that rely on labeled data on both graph and node classification tasks in many settings.

Limitations & Future Work. Since we do not involve training the projection head of the source-trained for downstream tasks, assumes that the source and target domains share the same label space. Thus, it is not designed for a general label distribution shift with new target classes. This limitation is tied to ’s frozen-head requirement. The lack of a unified discrete output space across graph tasks and datasets requires retraining or fine-tuning a task-specific projection head to handle new labels, which falls outside our setting. Developing universal graph vocabularies or output spaces for open-vocabulary graph prompting is an important future direction. Another interesting future direction would be to develop a method that selects high-quality pseudolabels in the presence of a severe covariate shift.

6 Appendix↩︎

6.1 Additional Related Work↩︎

6.1.0.1 Prompting for LLMs.

The promising results of early LLMs [6], [7] have inspired prompt-tuning approaches that leverage LLMs’ reasoning capabilities with minimal parameter tuning. These modular methods [1], [2] integrate trainable prompting prefixes with LLMs, with their parameters frozen, offering remarkable performance across various tasks while reducing complexity. Besides, [8] suggested pre-training on a large, diverse corpus (WebText) and demonstrated strong zero-shot performance across various tasks. Following that, [30] coined the term in-context learning, which refers to the effort to help models, particularly LLMs, generalize to new tasks without any parameter training, only by constructing informative prompts with task descriptions, examples, and instructions. The approaches propose specific language hints to guide the reasoning process, for example, by providing fine-grained and conditionally specific instructions [53], or by encouraging the LLM to engage in sequential reasoning with [54] or without [55] examples. Additionally, some methods iteratively score, evaluate, and update prompts in refinement loops [56], [57].

6.1.0.2 Prompting for GNNs.

A few studies have also adopted prompt tuning for . Specifically, the main track of these works starts with the “pre-train, prompt, fine-tune” paradigm proposed by [12]; they address the common issue of a discrepancy between training and test objectives that causes performance drop in downstream applications. They design a pre-training task, specifically edge prediction, that can align with the downstream task. Nonetheless, their approach is only applicable to node classification. Later, GraphPrompt [14] and GraphPrompt+ [52] propose subgraph similarity detection as a more general task unification for different graph tasks to pre-train a encoder. Their prompting method involves task-specific trainable readout functions. Similarly, PRODIGY [13] generates prompts as a combination of example subgraphs, connected to label nodes, and query subgraphs, awaiting connection to label nodes, and pre-trains with a neighborhood-matching task. Unification to subgraph classification is also proposed by GPF-Plus [15] and ALL-In-One [16]. However, these methods resemble LLM prompting methods more, in that they prompt input graphs before a frozen encoder. Specifically, the former adds trainable prompting parameters to the feature matrix of an input graph, and the latter combines a subgraph with a trainable feature matrix and structure with the input. More recently, GCOPE [58] utilizes graph coordinators to align domain-shifted datasets and remedy negative transfer in cross-domain pre-training. Also, DAGPrompt [18] introduces distribution-aware prompting by utilizing low-rank adaptation for heterophilic graphs. Similarly, to capture diverse node-specific patterns in non-homophilic graphs, PRONOG [59] introduces a conditional prompting. In addition, Prompting GNNs for dynamic graphs has also been studied lately [60].

There are two shortcomings with the above prompting methods. First, all of them require labeled data for training their prompting functions or for testing. Second, they mostly train a new projection head/decoder for the pre-trained along with the prompting parameters. However, we propose a fully unsupervised prompting method that does not require fine-tuning and achieves promising performance even when competitors have access to all or a subset of the labels.

6.1.0.3 Consistency Regularization and Pseudo-labeling.

In the context of Semi-Supervised Learning (SSL), pseudo-labeling [31], [32] and consistency regularization [33], [34], [61] for training neural network models when labeled data are scarce. The first technique augments the labeled dataset with the model’s predictions on the unlabeled dataset, and the second aims to minimize the discrepancy in a model’s predictions across random perturbations of the same samples generated by augmentation and dropout layers. In particular, these methods have been applied to domain adaptation to mitigate distribution shifts between source and target datasets [62], [63]. A widely studied approach to integrating pseudo-labeling and consistency regularization is to first generate random weakly and strongly augmented instances from the same dataset samples, and then assign pseudo-labels to the weakly augmented samples whenever the model makes confident predictions for them. Models are trained by these pseudo-labels as labels for the strongly augmented samples, along with labeled data if available. To select confident samples, some methods use a fixed certainty threshold [64] while others set dynamic class-wise thresholds [35], [36].

Unlike previous work that used consistency regularization and pseudo-labeling, we are not interested in training or fine-tuning a model. Therefore, our novelty lies in interpreting a prompted graph as a strongly augmented instance and using the pseudo-labels from weak augmentation to train the prompting parameters.

6.2 Algorithm↩︎

Algorithm 3 presents our method’s prompting procedure during training. In line 1, we initialize the base model with trained parameters tuned on the source and target datasets, the augmentation and prompting functions, and the method’s hyperparameters. We initialize the prompting parameter and fix the base ’s parameters in lines 3-4. Line 6 shows sampling of a batch of graphs and in line 7 a strong and a weak augmentation is generated from each graph in the batch. Next, the strongly augmented graph is prompted in line 8. This prompted graph, along with the weakly augmented graph, is encoded by , as in line 9. In line 10, the representation of the prompted and weakly augmented graphs is used to optimize the discriminator. Finally, in lines 9-10, the ’s projection head (decoder) decodes the representation, and the diversity, domain adaptation, and consistency objective functions are used to optimize the prompting parameters. Algorithm 4 shows the inference stage of our method. At inference time, a test sample is passed directly to the prompting function without augmentation, and the GNN scores the prompted graph.

6.3 Remark on UGPP definition.↩︎

Firstly, our problem setting differs from out-of-distribution (OOD) generalization methods [65]–[67]. In OOD generalization, given samples of dataset \(\data=\{(x_i, y_i)\}_{i=1}^{N}\) drawn from training distribution \(P_{train}(X, Y)\), the goal is to train an optimal model \(f(.; \theta)\) to have the best generalization to the test samples drawn from the distribution \(P_{test}(X, Y)\), where \(P_{train}(X, Y) \neq P_{test}(X, Y)\). This differs from our problem setting, as our goal is to propose a prompting method for GNNs that aligns with the in-context learning paradigm of LLMs. As discussed, LLM prompting methods commonly 1) do not retrain or fine-tune the LLM, 2) do not necessarily use labeled data, and 3) do not assume access to the data used for training the LLM. Ugpp directly encourages these properties. We assume the GNN \(\varphi(.; \theta_g, \theta_h)\) is first trained on distribution \(\prob_X^s\prob_{Y|X}^s\) and \(\data_s\) has the same train and test distributions, i.e. \(\prob_{X,Y}^{s_{train}} = \prob_{X,Y}^{s_{test}} = \prob_X^s\prob_{Y|X}^s\), while this model is aimed to be used on \(\data_t\) with unlabeled training distribution \(\prob_{X,Y}^{t_{train}}\) and test distribution \(\prob_{X,Y}^{t_{test}}\), such that \(\prob_{X}^{t_{train}} = \prob_{X}^{t_{test}} \neq \prob_{X}^{s}\), but \(\prob_{Y|X}^{t_{train}} = \prob_{Y|X}^{t_{test}} = \prob_{Y|X}^{s}\) and \(\prob_{Y|X}^{t_{train}}\) is not available. So far, this assumption of Ugpp makes it close to the unsupervised domain adaptation (UDA) problem [68], [69] and satisfies unsupervised learning on \(\data_t\), the second property of LLM in-context learning methods discussed above. For the third property, we assume the source dataset \(\data_s\) is unobservable after training, which is known as source-free domain adaptation (SFDA) [19], [70]. For the first property, we assume all the trained GNN’s parameters (\(\theta_g, \theta_h\)) are fixed and we do not fine-tune them on dataset \(\data_t\), analogous to LLM in-context learning. So, our work also differs from SFDA methods, which allow fine-tuning all or a portion of the parameters on \(\data_t\). In summary, our problem setting differs from UDA, SFDA, and OOD generalization, and it encourages the evaluation and design of generalized prompting methods for GNNs.

6.4 Additional Details on Experimental Setup↩︎

6.4.1 Datasets Statistics↩︎

In this work, we use datasets with different tasks and types of features. For graph classification, we evaluate on bioinformatics and molecular datasets, specifically ENZYMES [39] for multi-class classification and DHFR [41] for binary classification (both with continuous features), PROTEINS [40] with discrete features for binary classification, also on BBBP and BACE [42]. For node classification, we use common citation networks Cora, CiteSeer, PubMed [43], and Flickr [44] with discrete features of online images for multi-class classification. We also experimented on Cornell, Texas, and Wisconsin [45] as semantic web datasets with discrete features. Table ¿tbl:tab:datasets? shows the statistics of these datasets.

6.4.2 Details on the Distribution Shift↩︎

Introducing distribution shift to graphs is challenging. First, the position of nodes does not matter in graphs, therefore, introducing distribution shift by geometric transformations is not an option. Second, it is hard to find invariant features among all graphs for universal manipulations to inject shift—like color manipulations in image domain since color channels are common features. Moreover, random perturbation to node features cannot be seen as a distribution shift because it may lead to noisy datasets rather than a distribution shift with some patterns. We study different distribution shifts in the graph domain to make a comprehensive evaluation for the setting of . Here is a brief review of these distribution shifts.

Node-level. Previous works study two main categories of distribution shift for nodes: 1- based on added random noise to node features [71] 2- based on structural properties such node degrees [72], [73], clustering coefficient [48], Page Rank (PR) [48], and Personalized Page Rank (PPR) [48], [74]. Because applying random noise to node features cannot necessarily represent distribution shifts, we use structural properties in our work. For that, we choose PR for main experiments as a popularity-based property since it implies challenging distribution shift [48], [74]. We also study the clustering coefficient as a density-based property and provide results.

Graph-level. Apart from datasets with inherently shifted distributions [42], [72], [75]–[78], number of nodes [65], [79], average node degrees [80] and other graph properties can be utilized for introducing distribution shift for graph-level tasks. However, for main experiments, we choose edge homophily ratio [46] for generating graph datasets with distribution shift because are intrinsically affected by this property for information aggregation [45]–[47], [81], and also our experiments on common graph classification datasets show higher variance of homophily ratio among graphs of these datasets. Additionally, we evaluate our method against graph density as another distribution shift as well.

6.4.3 Experimental Details↩︎

Task Unification. A common practice for prompt tuning and in-context learning methods in graph domains is unification of different tasks [12], [13], [16]. However, defining a unified task in the graph domain is more challenging than in text (e.g., using LLMs), for two main reasons. First, proposing a unified task requires an enormous amount of data, labeled or unlabeled, while, in contrast to text gathering, these large graph datasets are not feasible. Second, there is a wide variety of downstream graph tasks. Nonetheless, we unify the tasks to graph classification [16], considering the message-passing intrinsic of [47].

To reduce node classification to graph classification, we select the induced subgraph of each ego node within its \(k\)-hop neighborhood and assign the label of the ego node to this subgraph. One can also reduce edge-level tasks by selecting the \(k\)-hop neighborhoods around the nodes lying on the endpoints of each edge [16].

6.4.3.1 Implementation Details.

We have implemented our experiments in Pytorch [82] and used a single GPU core NVIDIA GeForce RTX 3090. To make results reliable, we run each experiment with 10 different random initializations of seeds before dataset creation and with 5 trials of model parameter initialization for every seed, which sums up to a total of 50 runs for every experiment.

We split all datasets based on properties to make a distribution shift to a 50% source dataset and a 50% target dataset. Since we split source and target datasets in the beginning in favor of making distribution shifts, we randomly make our own train, validation, and test splits for every trial, although the node classification datasets have original splits. Therefore, the train, validation, and test split is set to 0.6, 0.1, 0.3 for graph classification datasets and to 0.3, 0.1, 0.6 for node classification datasets—to reflect more on their original splits as they have many more test examples compared to train ones. Besides, for node classification, we find the induced graphs within nodes selected from datasets after the source-target split.

We tune hyper-parameters based on the average F1 score on validation sets as follows. For we select learning rate from \(\{0.01, 0.001\}\), batch size from \(\{16, 32, 64\}\), number of epochs from \(\{30, 50, 60\}\), loss function weights for domain adaptation and diversity (\(\lambda_1, \lambda_2\)) from \(\{0.25, 0.5, 0.75, 1.0, 1.25, 1.5\}\), the \(L_2\) regularization factor \((\lambda_3)\) from \(\{0.1, 0.2\}\), the augmentation probability \(p_u\) from \(\{0.1, 0.2, 0.3, 0.4, 0.5\}\) and for \(p_w\) from \(\{0.05, 0.1, 0.2\}\), the number of trainable prompting parameter vectors \(n_p\) from \(\{10, 20, 30, 50, \expect_{N_G}\}\) where \(\expect_{N_G}\) is the average number of nodes in graphs for the graph datasets. For the certainty threshold \(\tau\), we either chose a fixed threshold following FixMatch [64] or a dynamic class-wise threshold following FlexMatch [35], then we select the threshold from \(\{0.1, 0.3, 0.5, 0.7\}\). The final selection of all hyper-parameters for the GCN as base and for main distribution shifts (edge homophily and PR) is provided in the codes provided by the link before.

6.5 Additional Experimental Results↩︎

6.5.1 Comparison with More baselines↩︎

To validate whether with no label constraint can offer practical advantages over few-shot methods, in this section, we evaluate our method against more recent state-of-the-art models, namely DAGPrompt, PRONOG, and GCOPE. As evidenced in Tables ¿tbl:tab:few95shot95comparison95graph? and ¿tbl:tab:few95shot95comparison95node?, generally outperforms the few-shot baselines in new experiments.

Specifically, consistently surpasses all baselines on graph classification and achieves competitive (first or second best) results on node classification. The robustness of zero-label acts as a powerful regularizer, forcing distribution alignment for positive transfer, which contrasts sharply with few-shot methods that often suffer from catastrophic negative transfer and overfitting to biased samples under distribution shifts. In addition, has notably less complex (lower number of trainable parameters) than the baselines. Therefore, heavier models like DAGPrompt (which has separate trainable low-rank matrices for each input and GNN encoder layer) fail catastrophically on smaller graph classification datasets—despite having superior performance on large and heterophilic node classification tasks—and undesirably show negative improvement as it requires a larger number of samples.

6.5.2 Effectiveness across Different Distribution Shifts.↩︎

The problem setup validates how well prompting methods can relieve the performance drop of a facing distribution shift. Therefore, we aim to show the generalization of across different kinds of distribution shifts. Previously, we evaluated our method on distribution shift based on edge homophily ratio and Page Rank (PR). Here we add shifts based on graph density for graph classification and clustering coefficient for node classification.

Graph Density Distribution Shift. Table ¿tbl:tab:graphgcndensity? illustrates generally attains the best results on the graph datasets with consistent positive performance gain, while the competitors mostly have negative performance gain on DHFR and PROTEINS in all cases even when they take advantage of 50% of labeled data of the target distribution. Besides, we have the second-best performance on ENZYMES after Fine-tuning in 50% label setting while beating this baseline in 25% label setting.

Node Clustering Coefficient Density Distribution Shift. The results in Table ¿tbl:tab:nodegcncc? reflect on the superiority of over all competitors on CiteSeer and PubMed. However, GPF-Plus outperforms our method on Cora when it has access to 50% of labeled samples while is beaten by when 25% of labels are available. Notably, we excel over all of the baselines on PubMed with a high margin and with no labels.

6.5.3 Generalization to Advanced GNN Backbones↩︎

A core principle of a prompting method is that it should be agnostic to the specific GNN backbone, demonstrating effectiveness across a range of architectures. To validate this, we extend our experiments beyond the foundational GCN and GAT models by incorporating two powerful, recent . We specifically chose: 1) GATv2 [83], a model that uses dynamic attention to better handle graphs with varying levels of homophily and heterophily. 2) GraphGPS [84], a graph transformer that uses a global attention mechanism to address common challenges like oversmoothing and oversquashing.

The results of these new experiments are presented in Table ¿tbl:tab:graphgps95gatv2? for graph and node classification tasks, respectively. The findings clearly demonstrate that our method, , consistently outperforms all baselines when applied to these advanced GNN backbones. This is particularly noteworthy as the baselines retain the significant advantage of access to labeled data from the target domain, whereas our method operates in a completely unsupervised manner. These results confirm that the effectiveness of our prompting framework is independent of a specific architecture and that it generalizes robustly to more powerful and modern backbones.

6.5.4 Comparison with Source-Free Domain Adaptation Methods↩︎

Since is a restrictive prompt-based setting within the broader SFDA landscape, the shared goal of adapting a model without source data motivates a direct empirical comparison. To this end, we evaluate against two recent, state-of-the-art graph SFDA methods: SOGA[20] and GraphCTA[21]. Since these methods are designed for node classification, we conduct the evaluation on our node classification datasets.

To create a fair comparison within our prompting-focused problem setup, we adapt these baselines. SFDA methods typically fine-tune the entire model; instead, we align them with the “lightweight fine-tuning” paradigm common to other baselines by freezing the source-trained GNN’s encoder and only allowing the decoder (prediction head) to be trained on the target data. This contrasts with our method, , where both the encoder and decoder remain fully frozen.

The results of this comparison are presented in Table ¿tbl:tab:sfda95eval?. consistently and significantly outperforms both adapted SFDA methods across all datasets. This outcome is powerful, as it demonstrates that our parameter-efficient approach of training only a prompt is more effective for adaptation than the more common fine-tuning strategy for the prediction head. In addition, our method is significantly lighter since we only train a small set of trainable prompting vectors, but these methods are originally proposed to fine-tune all parameters of the pretrained model.

6.6 Comparison with Decoder Fine-tuned Prompting Baselines↩︎

In the main experiments, we restrict all prompting baselines to the frozen-setting required by . This allows us to isolate the effect of the prompting function without conflating it with fine-tuning the source-trained encoder or prediction head. However, several supervised GNN prompting methods were originally designed to fine-tune a task-specific decoder. To provide a more comprehensive comparison, we further evaluate the supervised baselines in this relaxed setting, where their decoders are fine-tuned using 25% labeled target data.

Table ¿tbl:tab:decoder95finetune95baselines? reports the performance improvement over the frozen BaseModel. Even with access to labeled target data and decoder fine-tuning, the supervised baselines often suffer from negative transfer under distribution shift. In contrast, keeps both the encoder and decoder frozen, uses no labels, and still achieves stable positive improvements in most cases. These results further support that the gains of come from effective unsupervised prompting rather than from additional fine-tuning.

6.6.1 Embedding distribution analysis under distribution shift↩︎

Figure 5 illustrates the distribution of embeddings produced by the base ’s encoder for the PubMed dataset. The setting is for PR distribution shift with GCN as the base . It compares the embeddings of the BaseModels on source (BaseModel-source) and target (BaseModel-Target) test data, with graphs not prompted in either case. Additionally, it shows the embeddings for and GPF-Plus when prompted with graphs from the target test data, while is fully unsupervised and GPF-Plus consumes 50% of the labeled data.

This figure provides empirical evidence supporting our claims. First, , which employs consistency regularization without labeled data, achieves equal or better performance in mitigating distribution shift compared to GPF-Plus, although the latter has access to labeled data. Second, when using confident pseudo-labels and keeping the ’s parameters fixed after source-training, the prompting function aligns with the knowledge learned from the source data. This approach preserves samples within densely homophilous regions while pushing uncertain samples (those in overlapping regions) away, replicating the source data distribution as seen in BaseModel-Source. As a result, the method produces well-separated representations, enabling the projection head to effectively discriminate between classes using the same source-trained weights.

6.6.2 Effect of regularization methods↩︎

We study the effect of domain adaptation and diversity regularization methods introduced in Section 3.2. Minimizing the entropy of the score’s expected value (as a diversity regularization term) has also been used in prior representation learning studies [38], [85]. This regularization term relieves the harmful effect of class imbalance. Meanwhile, the domain adaptation regularization term is more beneficial for smaller datasets, since inferring their distribution from a small number of samples is difficult, increasing the likelihood of generating OOD prompted graphs. To showcase the effect of these objectives, we introduce two common methods of measuring class imbalance: the Imbalanced Ratio (IR) and Normalized Entropy (NE). A higher IR and a lower NE show more class imbalance in the datasets. Denoting the frequency of each class of a dataset as \(f_i\), while we have \(C\) classes, IR and NE are defined as below:

\[\begin{align} IR = \frac{f_{max}}{f_{min}}; \qquad f_{min}=\underset{i}{\mathop{\mathrm{arg\,min}}}\;f_i, \quad f_{max}=\underset{i}{\mathop{\mathrm{arg\,max}}}\;f_i\\\ NE = \frac{H}{log(C)}; \qquad H=-\sum_{i=1}^{C}p_ilog(p_i), \quad p_i=\frac{f_i}{\sum_{i=1}^Cf_i} \end{align}\]

Tables ¿tbl:tab:ds95reg95stats? shows the total number of samples in the target datasets (50% of the actual dataset sizes because we split the datasets into half source and half target), the statistics of their corresponding test sets, and the IMP% of on these datasets (we fix GCN as the base GNN).

When looking at both graph and node classification results, the general trends support our claims that the regularization methods have a positive impact. The first main trend observed in each task group is that diversity regularization is more effective as dataset imbalance increases; specifically, on PROTEINS and Cora, which are the most imbalanced datasets for graph and node classification respectively, diversity regularization shows the highest IMP%. Furthermore, when considering dataset size, domain regularization generally demonstrates its greatest effect on smaller datasets. For example, Cora, the node classification dataset with the fewest samples, benefits the most from domain regularization. An exception regarding domain regularization’s effect might be seen when comparing PROTEINS with DHFR and ENZYMES; this could be because PROTEINS has discrete features, whereas ENZYMES and DHFR also include continuous features, potentially reducing the impact of domain regularization.

To better show the effects of each regularization method, we also conduct a more comprehensive ablation study on Cora and PROTEINS, which serve as representatives of datasets for node and graph classification. Table ¿tbl:tab:lambdaeffect? shows how increasing the weight of each regularization term, as in Equation 6 , improves the performance on both datasets, especially on PROTEINS.

6.6.3 Effectiveness under no distribution shift↩︎

A key concern for any adaptation method is whether adapting to a new target distribution causes “catastrophic forgetting,” degrading performance on the original source domain. To measure this, we adapted models on the target data (with a distribution shift) and then evaluated their performance on both the original source test set and the target test set. The results are presented in Table ¿tbl:tab:source95target?, with columns marked (S) for source and (T) for target evaluation. The findings show that not only adapts effectively to the target distribution but also maintains higher performance on the source data compared to traditional lightweight fine-tuning, demonstrating its robustness against catastrophic forgetting.

To further understand our method’s behavior, we also investigated its effect in a no-shift setting, where the model is prompted with a target domain that shares the same distribution as the source domain. As shown in Table ¿tbl:tab:no95shift95eval?, we measured the F1 score improvement (IMP) over the base model in this scenario. The results demonstrate that consistently improves performance even without a distribution shift. This benefit is more evident than that of competing baselines, which show smaller gains and occasionally fail to improve performance at all. This indicates that our prompting approach serves as a general performance enhancer, not just a tool for mitigating distribution shifts.

6.6.4 Effect of the number of trainable prompting vectors↩︎

As the final ablation study, we evaluate the effect of increasing the number of trainable prompting vectors. For this experiment, we also fix GCN as the base GNN and report the results in Table ¿tbl:tab:tokeneffect?. This empirical evaluation clarifies that increasing the number of does not have a significant impact on ’s performance. This conclusion is indeed favorable, meaning that our method can achieve desirable results even with a considerably low number of trainable parameters.

6.6.5 Analysis of Computational Cost↩︎

A practical consideration for any adaptation method is its computational cost. The unsupervised nature of , which relies on data augmentation and multiple regularization components, introduces manageable overhead during training. This is an expected trade-off for the significant advantage of operating without labeled data. Table ¿tbl:tab:time95comparison? shows the seconds of the average full dataset training epoch time and test time across node classification datasets. Our training times are marginally higher than supervised prompting baselines, but scale reasonably on large graphs like PubMed and Flickr.

However, the more critical metric for real-world deployment is inference efficiency. Once the prompt is trained, the adaptation process is complete. At test time, the expensive training components, such as data augmentation and the discriminator, are no longer required. The inference step simply involves a forward pass through the frozen GNN with the learned prompt, resulting in a computational complexity of \(O(NLd^2 + L|E|d + Ndn_t)\)—as discussed in Section 3.3—compared to \(O(NLd^2 + L|E|d)\) of a regular models and \(n_t\) is number of trainable prompting vectors. Since, in all our experiments, even for large graphs \(n_t\leq 50\), the overhead can be neglected. Our empirical results confirm this efficiency, showing that ’s test time is consistently below the average of competing prompting methods, making it a lightweight and practical solution for deployment.

6.6.6 Effect of changing the prompting function↩︎

An advantage of , as discussed in Section 3.1, is its versatility, as it serves as a general framework agnostic to the base and the prompting function. This means our unsupervised framework can potentially enhance a prefix prompting function such as [15], [16]. To support this claim, we design an experiment where we substitute our experimental prompting function (which is similar to GPF-Plus) with All-In-One’s prompting function and present the results in Table ¿tbl:tab:allinone?. These results show that All-In-One’s prompting performs better when integrated into our framework. Notably, this improvement occurs without the use of labeled data.

6.6.6.1 Remark on prompt tuning and other PEFT methods.

Other parameter-efficient fine-tuning (PEFT) methods for adaptation, such as LoRA, may also be considered for source-trained adaptation. However, input-level prompt tuning is more aligned with the setting for two reasons. First, it treats the source-trained as a frozen, potentially inaccessible model, since the prompt only modifies the input graph representation and does not require accessing or changing its internal layers. In contrast, methods such as LoRA inject trainable parameters into specific layers and therefore require architectural access to the model. Second, our consistency-based objective compares a weakly augmented graph with a strongly augmented and prompted graph, both of which are processed by the same frozen . This keeps adaptation isolated in the prompting function. Moreover, standard are usually much smaller than LLMs, so layer-wise adapters may add a non-negligible number of parameters compared to our small set of trainable prompting vectors, which is important when adaptation relies only on unlabeled target data. We leave a systematic study of other PEFT methods under as future work.

6.6.7 Effect of types of augmentation↩︎

Augmentation is a key component of our framework. As discussed in Section 3.1, the type of augmentation should align with the prompting function. For instance, if the prompting function applies feature modifications, as in our main experimental prompting function, feature augmentation is expected to be more beneficial than structural augmentation (e.g., adding or removing edges), and vice versa.

As an ablation study on the type of augmentation, Table ¿tbl:tab:augmentation95type? shows how different types of augmentation impact ’s performance. Here, our prompting function transforms the feature matrix, the feature augmentation masks features, and structural augmentation drops edges. Results meet our expectations that feature augmentation improves performance, as it better aligns with the prompting function. Since none of the existing prompting functions can be categorized solely as structural prompting (without changing the node representations), we leave experimenting with such a prompting method for future work.

6.7 Source Alignment of Confidence-filtered Target Samples↩︎

Although the source data are not used during training, we perform a diagnostic analysis to understand the role of confidence filtering. Therefore, source compatibility is induced indirectly through the frozen source-trained and the confidence threshold used in \(L_c\).

To verify this effect, we measure the Maximum Mean Discrepancy (MMD) between the latent embeddings of the source data and two subsets of weakly augmented target samples: confident samples with \(\max(\tilde{\mathbf{p}}^{\graph}_{\varphi}) \ge \tau\), and unconfident samples with \(\max(\tilde{\mathbf{p}}^{\graph}_{\varphi}) < \tau\). Table 3 shows that confident target samples are generally closer to the source distribution than unconfident ones. This supports our interpretation that confidence filtering selects the portion of the target distribution that is more compatible with the source-trained model.

Using the adversarial regularizer in \(L_{\mathrm{adv}}\) helps align the prompted graphs with their non-prompted weak augmentations on the target domain. According to the results, although this objective does not directly align the prompted representations with the unobserved source distribution, it prevents them from drifting too far from the target-side anchors, which are closer to the source samples.

6.8 Pseudo-label Reliability and Sensitivity to Confidence Threshold↩︎

Pseudo-labeling under covariate shift may suffer from confirmation bias when high confidence predictions are incorrect. To evaluate this risk, we report pseudo-label accuracy, calibration before and after prompting, and sensitivity to the confidence threshold \(\tau\) in Table 4. For binary classification datasets, we have \(\tau \in \{0.6, 0.7, 0.8, 0.9\}\) and for multi-class classification we use \(\tau \in \{0.2, 0.4, 0.6, 0.8\}\). First, increasing \(\tau\) consistently improves pseudo-label accuracy, indicating that the threshold successfully filters out noisy predictions caused by distribution shift. Second, comparing the expected calibration error (ECE) before and after prompting reveals that generally improves model calibration rather than amplifying overconfidence. Together, our regularized prompting framework extracts reliable signals and mitigates confirmation bias. We will include this analysis in the revised appendix.

6.8.1 Unsupervised Prompting vs Fully Supervised Prompting↩︎

Since achieves significant improvements across different experiments, we are interested in evaluating competitors while allowing them to access 50%, 75%, and 100% of the labeled data from the target distributions. We show the results of these settings in Tables ¿tbl:tab:graphgcn50? and ¿tbl:tab:nodegcn50?, and Figures 6 and 7. All results are reported for both GCN and GAT under distribution shifts induced by edge homophily for graph classification and PR node classification.

Graph classification results for both GCN and GAT base models in Table ¿tbl:tab:graphgcn50? illustrate the superior performance of as an unsupervised method compared to the baselines in most cases, even when they have access to 50% of the labeled samples for training on the target datasets. On the node classification datasets, achieves the second-best performance as shown in Table ¿tbl:tab:nodegcn50?. Specifically, for larger node classification datasets, it is noteworthy that we beat all baselines on Flickr and only underperform GraphPrompt+ on PubMed.

Observing Figure 6, when we use GAT as the base model, outperforms GPF-Plus and GraphPrompt on graph classification datasets even when they utilize fully labeled target datasets. This clearly shows the effectiveness of our proposed method, . Additionally, we achieve closely competitive results with GCN as the base model in 100% label setting overall. Besides looking at the node classification results for both GNNs, we generally obtain the second-best improvement in 75% label setting and perform favorably with 100% compared to the best baseline.

Next, we evaluate our method in distribution shifts of graph density and clustering coefficient under the 75% and 100% label settings. Here, we fix GCN as the base GNN. Results are in Figure 7. Similar to the previous distributions, for graph density and clustering coefficient, achieves competitive or better performance on graph classification datasets and ranks second after GPF-Plus on node classification, even though our method does not see any labels.

Finally, an interesting finding of these experiments is that competitors can occasionally cause a performance drop compared to the base models, which is unexpected and undesirable. On the other hand, , although it is an unsupervised method, does not negatively affect any dataset, any architecture, or any type of distribution shift. Also, all the above results lead to the same conclusion: can enhance base GNNs that encounter different distribution shifts across various downstream tasks. Since achieves promising results in a fully unsupervised manner, it offers new avenues for leveraging large unlabeled datasets to improve the generalization of .

References↩︎