1 Introduction↩︎

In recent years, pre-trained language models (PLMs) in natural language processing (NLP) are blooming everywhere [1]–[7]. However, not only the number of PLMs but also their size is rapidly growing, making it harder to perform full fine-tuning. To address the issue, tons of parameter-efficient fine-tuning (PEFT) methods have bubbled up, such as adapters [8]–[11], or prompts [12], [13]. These methods have made it equitable for researchers with insufficient computational resources.

However, there is still a long way to go for these PE methods to reach their full potential. Because the pre-training objectives are not directly related to PE, it is foreseeable that there is a mismatch between the PLM and PE methods, which may prevent PE methods from unleashing their full power. To address this problem, we introduce an additional "priming" stage between pre-training and downstream fine-tuning. As shown in Fig.1, we prime the PLM on extra few-shot source tasks with multitask learning (MTL) or meta-learning, and then fine-tune PE elements to target tasks. Compared with traditional PEFT methods, the PLM and PE elements can fit each other better after priming. In other words, instead of the conventional PEFT paradigm (pre-train \(\rightarrow\) PEFT), we adopt the "pre-train \(\rightarrow\) priming \(\rightarrow\) PEFT" pipeline.

Some recent studies explore how to bridge the gap between pre-training tasks and target tasks. [14] pre-trains the soft prompt tokens with self-supervised tasks to give a better initialization. [15], [16] exploits optimization-based meta-learning to find an initialization for soft prompts to facilitate faster adaptation to new tasks. [17] tweaks the meta-learning algorithm MAML [18] for priming and simulates the PEFT procedure in the inner loop. However, they only focus on priming for a single PE method with a specific algorithm. In contrast, our work views priming as a general method to boost PEFT from a higher perspective. Moreover, previous works explore only single-domain tasks, which lack the exploration of generalization ability. On the contrary, our work evaluate PE methods with diverse NLP tasks in various domains.

On top of that, we conduct comprehensive experiments over well-known PE methods like adapters and prompt tuning under different settings. The experiment result reveals that priming by tuning only PLM leads to the best adaptation result on target tasks. In addition, we shows priming does help the whole model to converge more easily, which validates the necessity of priming.

2 Related Work↩︎

2.1 Adapter↩︎

Adapters [8]–[11], [19]–[24] are lightweight modules introduced for the transformer architecture. Adapters add extra trainable parameters and freeze the original PLM parameters during fine-tuning.

2.2 Prompt↩︎

Prompt-based tuning [12]–[16], [25]–[29] is an innovative method to use the power of PLMs efficiently. [14] proposed the concept of prompt initialization pre-training. [15] proposed Meta-learned Prompt Tuning (MetaPT) to further improve prompt initialization.

3 Methodology↩︎

3.1 Framework↩︎

Our work aims to comprehensively analyze the priming strategies by comparing the performance of PE methods under few-shot scenarios. We introduce a general framework to prime the whole model (may include PLMs) to better adapt to downstream tasks. Our training approach consists of two distinct stages: the upstream priming stage and the downstream fine-tuning stage. Initially, the model acquires knowledge from source tasks during the upstream priming stage, followed by few-shot fine-tuning on target tasks in the downstream stage.

Specifically, we name parameters fine-tuned in the upstream stage as upstream tunable elements, while those in the downstream stage as downstream tunable elements. Upstream tunable elements and downstream tunable elements may be fully-overlapping, partially-overlapping or non-overlapping.

3.2 Upstream Priming Stage↩︎

The upstream priming stage is designed to prime the model’s upstream tunable elements, enabling it to quickly adapt to a range of downstream few-shot tasks. We employ a priming algorithm, predominantly Meta Learning or Multitask Learning (MTL), to update the upstream tunable elements on source tasks. Upstream tunable elements comprise Pre-trained Language Models (PLM), adapters, and prompts. In other words, the combination of upstream tu

3.2.1 Multi-task Learning↩︎

In Multi-task Learning (MTL), multiple tasks are learned concurrently by minimizing their combined loss. This method enhances the model’s capability to learn cross-task features and accelerates adaptation. Our implementation of MTL focuses on training the upstream tunable elements during the upstream priming stage. We define \(\psi\) as the whole model’s parameters, with subsets \(\psi_u\) for upstream and \(\psi_d\) for downstream tunable elements. The objective is to minimize loss across training tasks while adjusting only \(\psi_u\):

\[\small \psi_u^\prime = \mathop{\arg\min}_{\psi_u} \sum_{\mathcal{T}_i\in \mathcal{T}}\mathcal{L}\left(\psi, \mathcal{T}_i\right)\]

Here, \(\mathcal{L}\) is the loss function, and \(\mathcal{T}_i\) represents the \(i^{th}\) task from the set of source tasks \(\mathcal{T}\).

It is important to note that if \(\psi_d\) is not included in \(\psi_u\), it is initialized but remains unchanged during the upstream priming stage. For example, if PLM is selected as the upstream tunable element and adapters as the downstream element, the adapters are initialized in the upstream stage but only tuned during the downstream fine-tuning phase.

3.2.2 Meta Learning↩︎

In our study, we utilize the MAML[18] algorithm, for our priming process. MAML is distinctive in its dual-phase training approach: the inner loop and the outer loop. The inner loop is designed for task-specific adaptation, while the outer loop focuses on finding an optimal initialization for quick adaptation in the inner loop. We modify some parts of MAML algorithm, which are outlined in Alg.3 and illustrates in Fig. 2. It starts by copying the current model parameters \(\psi\) as the initial state for the inner loop. In this loop, we specifically tune the downstream tunable element \(\psi_d\). The tuned parameters for the \(i^{th}\) task in the inner loop are represented as \(\psi_{i}^{\prime}\). The final step involves calculating the loss from the adapted model \(\psi_{i}^{\prime}\) and the source tasks \(\mathcal{T}_i\), which is then used to update \(\psi_u\). The updated \(\psi_u\) are initialized for the subsequent inner loop.

3.3 Downstream Fine-Tuning Stage↩︎

In downstream fine-tuning stage, with the primed initialization obtained from the upstream priming stage, we directly fine-tune the downstream tunable elements on the target tasks. Since the backbone of our work is to explore the cross-domain few-shot ability of PEFT methods w & w/o priming, only prompt or adapter are tunable in downstream stage.

4 Experiment↩︎

4.1 Dataset↩︎

We choose CrossFit Challenge [30] as our benchmark, which provides 160 different NLP few-shot tasks with a unified text-to-text format gathered from existing open-access datasets.

In CrossFit Challenge, they divide all tasks into non-overlapping Train, Dev, and Test tasks. We select random split in [30] to be the task split setting in our work. We select the Train tasks as the source tasks for upstream priming and the Test tasks as the target tasks for downstream fine-tuning. More explicit explanations of tasks can be found in [30]. Briefly speaking, CrossFit Challenge is able to evaluate the authentic few-shot generalization ability of models.

4.2 Setup↩︎

4.2.1 Tunable Elements↩︎

Our experiment setup mainly follows [30]. In the upstream priming stage, we can tune prompt, adapter and PLM, but we only tune prompt or adapters during the downstream fine-tuning stage to accord with the spirit of PE methods. It’s crucial to emphasize that the initialized parameters obtained from the upstream priming stage are carried forward to the downstream fine-tuning stage.

Adapter↩︎

In this work, we mainly adopt AdapterBias [11] as our adapter module. AdapterBias adds a token-dependent shift to the hidden output of transformer layers, parameterized by only a vector and a linear layer. Compared with the original adapter design [8], the trainable parameters are further reduced while obtaining comparable performance.

Prompt↩︎

Prompt is one of our tunable elements. In our settings, we applied prompt tuning proposed by [12], which concatenates tunable tokens before the input sentence and ask the PLM to generate corresponding output text. Following the settings in [12], we set the prompt length to 100 tokens.

4.2.2 Hyperparameters↩︎

In our research, we use the BART-base model [31] as our primary language model. For both the MTL and the outer loop of meta-learning, we employ the AdamW [32] with a weight decay of 0.01. Specifically in meta-learning, we set different outer loop learning rates for various elements: \(8\times10^{-5}\) for PLMs, \(8\times10^{-3}\) for prompts, and \(1\times10^{-5}\) for adapters. The inner loop has its learning rates set at 0.025 for prompts and 0.001 for adapters. The training is conducted over 80 epochs, with a batch size of 1 for training and an inner batch size of either 4 or 8, contingent on GPU memory limits. For MTL, we maintain a consistent learning rate of \(3\times10^{-5}\) for PLMs, prompts, and adapters. The MTL training spans 10 epochs with a train batch size of 32.

4.3 Metrics↩︎

For the evaluation metric, we also follow [30], adopting Average Relative Gain (ARG) as one of the performance indexes, and the definition for ARG is: \[\small \text{ARG} = \frac{1}{n}\sum_{i=1}^{n}(\frac{\text{P^i} - \text{P_0^i}}{\text{P_0^i}})\] \(\theta\) : Adapter and downstream model \(P_0^i\) represents the performance of directly fine-tuning PLMs on \(i^{th}\) target tasks, and \(P_i\) is that of our experiment combination. Since the comparing target is directly fine-tuning PLMs, baselines with ARG greater than 0 are those that surpass fine-tuning PLMs.

max width=1

4.4 Annotation↩︎

We use abbreviations to make the result more concise, including M for PLM, P for prompts, and A for adapters. Additionally, characters before the underline represent the upstream tunable elements, while those after the underline represent the downstream tunable elements of the baseline. For example, P_P represents upstream and downstream tunable elements are both prompts; M+A_A represents upstream tunable elements are PLM and adapters, and downstream tunable element is adapters. Lastly, we use FT_M, FT_A, and FT_P to represent directly fine-tuning the PLM, adapter, and prompt, respectively.

4.5 Main Result↩︎

The first block in Table ¿tbl:tab:main-experiment-combinations? showcases baselines without priming, while subsequent blocks feature combinations of different priming strategies categorized by the priming algorithm. Among the direct fine-tuning results, fine-tuning PLM serves as a competitive baseline with high training costs, while direct fine-tuning prompts/adapters are considered as primary baselines against each priming prompts/adapters baseline.

Table ¿tbl:tab:main-experiment-combinations? underscores the effectiveness of priming, with most baselines showing noticeable improvements (indicated by asterisks\(^\ast\)) across various priming algorithms. Notably, some combinations outperform direct fine-tuning of prompts, and a few even surpass fine-tuning PLM, such as M_P and M+P_P in multi-task learning (ARGs greater than 0). For adapters, certain combinations demonstrate remarkable progress, like M_A in both meta-learning and multi-task learning, while others experience slight drops in performance, such as A_A in both meta-learning and multi-task learning.

4.6 Parameter Efficiency↩︎

Fig 4 visually depicts the relationship between the performance of each method and its tuned parameter scale. The best-performing combinations from Table ¿tbl:tab:main-experiment-combinations? represent priming prompts/adapters (green ones). Other baselines include fine-tuning prompts/adapters without priming, fine-tuning PLM (BART-base), and existing works like LoRA[23] and BitFit[10]. Fig 4 demonstrates that priming prompts/adapters baselines locate at the upper-left region, indicating superior results and higher parameter efficiency brought by priming.

5 Analysis↩︎

In this section, we emphasize the advantages of priming by examining training loss during downstream fine-tuning. We compare loss curves between models w/ or w/o priming (FT_P and MTL M_P, respectively) across three diverse tasks from the target task training set. These tasks, unseen by prompts/adapters, feature distinct evaluation metrics. Figure 5 presents the results, where the blue curves represent FT_P (w/o priming) and the orange curves represent MTL M_P (w/ priming). The primed model not only converges faster but also achieves a superior final level of loss. Furthermore, the orange curves exhibit steadier convergence compared to the fluctuating and glitch-prone blue curves. These findings underscore the benefits of priming for prompts/adapters, facilitating rapid adaptation to various target tasks.

6 Conclusion↩︎

In this paper, we systematically analyze priming PEFT within a comprehensive framework. Our framework not only incorporates existing priming approaches but also explores previously uncharted strategies. Our experimental results demonstrate that the majority of priming strategies enhance the performance of PE methods. Notably, "Priming PLM only" emerges as the top-performing strategy when used in conjunction with multi-task learning. Crucially, our study provides concrete evidence that priming significantly facilitates the convergence of fine-tuning prompts/adapters on unseen tasks, underscoring the efficacy of priming.

7 Limitation↩︎

We provide a systematic analysis of different priming strategies on PE methods and successfully improve the few-shot performance on diverse downstream tasks. However, there are some limitations to our work. Though we empirically show that MTL outperforms meta-learning, there are no further explanations for it. Besides, a small proportion of the priming strategies lead to a performance drop, but the actual reason remains unexplained. In addition, all the experiments are conducted on the pre-trained BART-base model. Extra experiments on other large language models may strengthen the results.

Acknowledgement↩︎

We thank National Center for High-performance Computing (NCHC) of National Applied Research Laboratories (NARLabs) in Taiwan for providing computational and storage resources.

References↩︎