1 Introduction↩︎

PEFT is a fundamental strategy for adapting PFMs, which updates only a small fraction of parameters in large models. Among PEFT approaches, LoRA is arguably the most prominent: it employs a low-rank decomposition to parameterize weight updates, typically written as \(\Delta W = AB\). Many PEFT methods can be viewed as learning task-specific updates under a prior over \(\Delta W\), which can be written as \[\label{eq:old95assumption} \Delta W^* := \arg\max_{\Delta W \in \mathcal{S}} P(\mathcal{D} \mid W_0 + \Delta W) P(\Delta W).\tag{1}\] This formulation treats \(\Delta W\) as the main trainable object, while the pretrained weights \(W_0\) mainly serve as the fixed backbone to which \(\Delta W\) is added. Recently, several PEFT methods have started to explicitly exploit \(W_0\). For example, PiSSA [1] uses the SVD of \(W_0\) to initialize adapters, while DoRA [2] decomposes weights into magnitude and direction. However, these methods usually exploit \(W_0\) through initialization, decomposition, or element-wise interaction [3], rather than using \(W_0\) as a direct condition for generating task-specific updates throughout optimization. Motivated by this observation, we revisit PEFT from a conditional weight-generation perspective. Instead of learning \(\Delta W\) largely independently of \(W_0\), we consider a conditional prior over updates: \[\label{eq:assumption} \Delta W^* :=\arg\max_{\Delta W \in \mathcal{S}} P(\mathcal{D} \mid W_0 + \Delta W) P(\Delta W \mid W_0)P(W_0).\tag{2}\] Here, Eq. 2 is used as a conceptual motivation rather than a full probabilistic generative model. In practice, we instantiate this conditional perspective with a deterministic generator \(G_{\theta}(W_0)\), which produces task-specific updates by extracting structured information from the pretrained weights.

Finally, we introduce Generative Parameter-Efficient Fine-Tuning (GenFT), a \(W_0\)-conditioned PEFT method that uses a deterministic generator to produce task-specific updates \(\Delta W\). Specifically, GenFT applies row and column transformations with activation functions to extract structured information from \(W_0\) for update generation. Moreover, we introduce a shared-specific decomposition of the transformation factors, enabling information reuse across layers while preserving layer-specific flexibility. Our contributions are summarized as follows:

• We propose GenFT, a \(W_0\)-conditioned PEFT framework that generates task-specific updates through row and column transformations, improving adaptation across NLP and CV tasks.

• We introduce a shared-specific decomposition of the transformation factors, capturing cross-layer shared patterns and layer-specific variations with a small parameter budget.

• Experiments on VTAB-1K with ViT and GLUE with RoBERTa\(_{\textit{Base}}\) show that GenFT achieves competitive or better average performance than representative PEFT baselines, and a pilot study on Alpaca with LLaMA-7B further examines its feasibility for generative models.

2 Related Work↩︎

Additive PEFT methods introduce lightweight layers into transformers to enhance adaptability for customized tasks. Adapter [4] inserts a downsampling-activation-upsampling layer after the feedforward module. AdaptFormer [5] targets the MLP layer, improving transferability without modifying its parameters. AdapterDrop [6] dynamically disables adapters in lower layers during training to eliminate ineffective modules and boost efficiency. UniPT [7] accelerates training by adding parallel learnable modules to adapters, enabling simultaneous updates across layers. Tiny-Attention Adapter [8] designs compact adapters with minimal per-head dimensionality, directly modifying latent representations based on cross-position dependencies. These methods improve adaptation by modifying activations or inserting additional modules, but they do not explicitly generate weight updates from the structure of pretrained weights.

Selective PEFT methods are designed to update a subset of parameters in a pretrained model based on their importance or task relevance. BitFit [9] tunes only the bias terms of each layer, achieving promising results on small-to-medium-scale datasets. SPT [10] identifies sensitive parameters using gradient-based criteria, applying unstructured tuning to these parameters or switching to LoRA if the criteria are not met. GPS [11] employs a gradient-based method to select and tune a small subset of sensitive parameters while keeping the remaining parameters frozen. LoRAPrune [12] uses LoRA-guided pruning to estimate structural importance, iteratively removing redundant channels and heads. SH-PEFT [13] adopts a task-specific approach, selecting weights sensitive to the target task while discarding those responsive to unrelated tasks. These methods focus on identifying which parameters should be updated, while the generation of structured task-specific updates from \(W_0\) remains less explored.

Reparameterized PEFT methods restructure model parameters using low-rank or low-dimensional representations. LoRA [14], a prominent method, reparameterizes updates to pretrained weights \(W_0\) as \(W_0^{\prime} = W_0 + AB\), where \(A\) and \(B\) are low-rank matrices. DoRA [2] decomposes \(W_0\) into magnitude and direction components to enhance parameter learning. HiRA [3] applies the Hadamard product to combine LoRA weights with \(W_0\), aiming to improve expressiveness but potentially disrupting neuron correlations. FourierFT [15] transforms reparameterized weights into the Fourier domain, learning a small set of spectral coefficients. VB-LoRA [16] introduces a shared vector bank with a differentiable top-\(k\) module, enabling parameter sharing across layers. These methods exploit low-dimensional parameterization, initialization, decomposition, or element-wise interaction, but they do not explicitly model row- and column-wise transformations of \(W_0\) for update generation.

Overall, existing PEFT methods mainly improve efficiency through aforementioned PEFT methods. In contrast, GenFT is complementary to these directions: it uses \(W_0\) as the input to a deterministic update generator and extracts structured row- and column-wise information for task-specific adaptation.

3 GenFT↩︎

Building on Eq. 2 , the GenFT is illustrated in Fig. 1. Instead of learning \(\Delta W^{(\ell)}\) at \(\ell\) layer independently of \(W^{(\ell)}_0\), GenFT uses a deterministic generator to extract structured information from \(W^{(\ell)}_0\) for update generation.

Paradigm. We assume that pretrained weights \(W^{(\ell)}_0\) encode transferable structural information in their row and column spaces. GenFT therefore conditions the task-specific update \(\Delta W^{(\ell)}\) on \(W^{(\ell)}_0\) through a learnable generator. The core paradigm is: \[h^{(\ell)} = W^{(\ell)}_0 X^{(\ell)} + \Delta W^{(\ell)} X^{(\ell)} = W^{(\ell)}_0 X^{(\ell)} + \mathcal{G}_{\theta}(W^{(\ell)}_0) X^{(\ell)},\] where \(\mathcal{G}_{\theta}(\cdot)\) produces \(\Delta W^{(\ell)} \in \mathbb{R}^{D\times D}\) from \(W^{(\ell)}_0 \in \mathbb{R}^{D\times D}\). For clarity, we present the square-weight case; the formulation can be directly extended to non-square linear layers by matching input and output dimensions.

Generator. Since rows and columns of \(W^{(\ell)}_0\) correspond to output-channel and input-feature structures, respectively, GenFT applies row and column transformations to extract complementary information from \(W^{(\ell)}_0\). These transformations are controlled by learnable factors in \(\theta\), and the resulting features are used to generate the task-specific update \(\Delta W^{(\ell)}\).

Row Transformation. To balance shared and layer-specific information, we decompose the row-transformation matrix into a layer-shared component and a layer-specific component. The shared factors are reused across layers, while the layer-specific factors are learned independently for each layer. For the \(\ell\)-th layer, the row-transformation matrix is formulated as: \[\begin{align} U_1^{(\ell)} &= \begin{pmatrix} U_s & A^{(\ell)} \end{pmatrix},\quad U_2^{(\ell)} = \begin{pmatrix} U_s & B^{(\ell)} \end{pmatrix}, \\ U^{(\ell)} &= U_2^{(\ell)} (U_1^{(\ell)})^\top = U_s U_s^\top + B^{(\ell)} (A^{(\ell)})^\top, \end{align}\] where \(U_s \in \mathbb{R}^{D \times a}\) captures layer-shared row-wise information, and \(A^{(\ell)}, B^{(\ell)} \in \mathbb{R}^{D \times b}\) capture layer-specific information. The row transformation is: \[F^{(\ell)}_{\textit{row}} = \sigma_1(\textit{ratio} \cdot W_0 U^{(\ell)})\odot M_p,\] where \(\sigma_1\) is an activation function, ratio controls the proportion of \(W^{(\ell)}_0\) used for update generation, and \(M_p\) is a binary mask matrix that disables \(p\%\) of the corresponding weights.

Column Transformation. Complementary to the row transformation, the column transformation extracts input-feature information from \(W^{(\ell)}_0\). We use distinct layer-specific factors for the column transformation: \[\begin{align} V_1^{(\ell)} &= \begin{pmatrix} V_s & A^{(\ell)} \end{pmatrix},\quad V_2^{(\ell)} = \begin{pmatrix} V_s & B^{(\ell)} \end{pmatrix}, \\ V^{(\ell)} &= V_2^{(\ell)} (V_1^{(\ell)})^\top = V_s V_s^\top + B^{(\ell)} (A^{(\ell)})^\top, \end{align}\] where \(V_s \in \mathbb{R}^{D \times a}\) captures layer-shared column-wise information, and \(A^{(\ell)}, B^{(\ell)}\) is shared parameters from row transformation. The update is obtained by: \[\begin{align} F^{(\ell)}_{\textit{col.}} &= \sigma_2\left(\left(F^{(\ell)}_{\textit{row}}\right)^{\top} V^{(\ell)}\right) \odot M_p,\; \Delta W^{(\ell)} &= \textit{scaling} \cdot F^{(\ell)}_{\textit{col.}}, \end{align}\] where \(\sigma_2\) is an activation function and scaling controls the magnitude of \(\Delta W^{(\ell)}\). The latent dimension \(a+b\) controls the transformation space of GenFT, but it should not be interpreted as the exact algebraic rank of the final \(\Delta W^{(\ell)}\) after nonlinear activations and masking.

Finding. Given a model with \(L>1\) layers, let \(a\) denote the layer-shared dimension, \(b\) the layer-specific dimension, and \(r\) denote LoRA’s low-rank dimension. Under a comparable parameter budget, GenFT can use a larger latent transformation dimension than LoRA. See Appendix section A for more explanation.

As shown in Fig. 2, LoRA’s parameter count grows rapidly with \(r\), whereas GenFT grows more gradually with \(a+b\). Here, \(a+b\) denotes the latent transformation dimension rather than the exact algebraic rank of the final \(\Delta W^{(\ell)}\), since nonlinear activations and masking may change the resulting matrix rank.

4 Experiments↩︎

Natural Language Understanding. We fine-tune the RoBERTa\(_{\textit{Base}}\) [18] with GenFT on GLUE benchmark [17]. Our experimental setup follows [19]. We perform random hyperparameter search over the layer-shared dimension \(a\), layer-specific dimension \(b\), activation functions \(\sigma_1\) and \(\sigma_2\), initialization schemes for \(A^{(\ell)}\) and \(B^{(\ell)}\), the \(W^{(\ell)}_0\) ratio, and the scaling factor for \(\Delta W^{(\ell)}\), using the AdamW optimizer with a preset weight decay rate. To ensure reproducibility, we fix all random seeds at 42.

Results on GLUE. Results are reported in Table 1. GenFT ranks fourth on STS-B (behind PROPETL\(_{\textit{LoRA}}\)) and third on QQP and QNLI (behind Adapter\(^S\) and PROPETL\(_{v1}\)), likely due to the sensitivity of these tasks to hyperparameter configurations. Nevertheless, GenFT achieves the best average score (85.87%) with only 0.24M parameters, outperforming LoRA and other baselines on the remaining tasks. This improvement stems from its generative PEFT design: row–column transformations and the rank-decomposition policy, together with \(\sigma_1\) and \(\sigma_2\), extract structured information from \(W_0^{(\ell)}\) into \(\Delta W^{(\ell)}\), leading to robust performance across diverse NLU tasks.

Image Classification. We adopt the ViT-Base model [30] (85.8M parameters), for multi-class and fine-grained image classification. We evaluate GenFT on VTAB-1K [31], which contains 19 sub-datasets spanning natural, specialized, and structured images. We also validate GenFT on FGVC, please refer to Appendix section D for more details. We report top-1 accuracy as the primary evaluation metric. Our experimental setup follows V-PETL Bench [32]. We conduct random hyperparameter search over the layer-shared dimension \(a\), layer-specific dimension \(b\), activation functions \(\sigma_1\) and \(\sigma_2\), and initialization schemes for \(A^{(\ell)}\) and \(B^{(\ell)}\), using the AdamW optimizer with a preset weight decay rate. To ensure reproducibility, we fix all random seeds.

Results on VTAB-1K. Results are reported in Table 2. GenFT achieves the highest average accuracy, outperforming all baselines on natural and specialized images and remaining competitive on structured datasets. Compared to full fine-tuning, GenFT avoids excessive computational costs. Moreover, GenFT’s row and column transformations with activation functions \(\sigma_1\) and \(\sigma_2\) enable flexible adaptation, surpassing additive PEFT methods such as AdaptFormer, while its rank-decomposition policy broadens the adaptation scope relative to selective PEFT. Compared to LoRA and its variants, GenFT further leverages \(W^{(\ell)}_0\) via row and column transformations for robust \(\Delta W^{(\ell)}\) generation.

Pilot Validation on LLaMA-7B. For generative task, we fine-tune LLaMA-7B [41] on the Alpaca dataset [42], which contains 52,000 instruction-following demonstrations generated by OpenAI’s text-davinci-003. To evaluate GenFT on LLM under resource constraints, we conduct a pilot study on a randomly selected subset of 1,000 samples and compare GenFT with LoRA and PiSSA in terms of perplexity. Results are reported in Table 3. We also observe that GenFT requires a larger learning rate (3\(\times\)​10\(^{-3}\)) than LoRA and PiSSA (1\(\times\)​10\(^{-4}\)) to obtain an improved learning curve, which may be attributed to its expanded rank space. This observation suggests that GenFT exhibits distinct optimization dynamics, warranting further investigation in future large-scale experiments.

Ablation Study. GenFT comprises four main components: the layer-shared configuration (\(U_{\textit{s}}, V_{\textit{s}}\)), the layer-specific configuration (\(A^{(\ell)}, B^{(\ell)}\)), the row transformation of \(W_0^{(\ell)}\), and the column transformation of \(W_0^{(\ell)}\). We ablate each component individually and evaluate performance on the FGVC and VTAB-1K benchmarks. To disable the layer-shared or layer-specific module, we set the corresponding dimension to zero; for the row and column ablations, we remove the corresponding transformation. In cases where a dimension is originally zero, we set it to 8 to ensure functionality.

Results in Table 4 show that removing any component degrades performance, with FGVC decreasing by up to 6% and VTAB-1K by up to 11%. In particular, removing the layer-shared configuration causes a substantial drop due to its role in cross-task generalization. Moreover, the row and column transformations, together with the rank-decomposition policy and \(\sigma_1, \sigma_2\), are crucial for extracting structured information into \(\Delta W^{(\ell)}\).

Computational Cost Analysis. For LoRA, we assume \(\Delta W^{(\ell)} \in \mathbb{R}^{D \times D}\) with \(A^{(\ell)} \in \mathbb{R}^{D \times r}\) and \(B^{(\ell)} \in \mathbb{R}^{r \times D}\). The dominant per-layer cost is the matrix multiplication, i.e., \(\mathcal{O}(rD^2)\); we ignore the cost of the element-wise addition, which is typically negligible compared to multiplication. For the row transformation in GenFT, we assume \(W_0^{(\ell)} \in \mathbb{R}^{D \times D}\), \(U_{\textit{s}} \in \mathbb{R}^{D \times a}\), and \(A^{(\ell)}, B^{(\ell)} \in \mathbb{R}^{D \times b}\), where \(a\) and \(b\) denote the layer-shared and layer-specific dimensions, respectively. Using the simplified row transformation \(F^{(\ell)}_{\textit{row}} = (W^{(\ell)}_0 U^{(\ell)}_{\textit{s}})(U_{\textit{s}}^{(\ell)})^\top + (W^{(\ell)}_0 B^{(\ell)})(A^{(\ell)})^\top,\) whose cost is dominated by matrix multiplications and is \(\mathcal{O}((a+b)D^2)\) up to constant factors. The column transformation has the same order. Therefore, GenFT may introduce slightly higher training cost than LoRA when using a larger latent transformation dimension \(a+b\), while both methods remain quadratic in \(D\). Empirically, we select three time-consuming subsets across three benchmarks and compare LoRA and GenFT under different parameter budgets. As shown in Table 5, LoRA becomes comparable to GenFT in time cost only when using substantially more parameters, while GenFT maintains competitive or better performance with a smaller parameter budget.

We also measure peak GPU memory during training. GenFT introduces only small memory overhead compared with LoRA: 6.54 GB vs. 6.46 GB on VTAB-1K, 6.56 GB vs. 6.46 GB on FGVC, and 7.09 GB vs. 6.77 GB on GLUE. Moreover, after training, the generated update \(\Delta W\) can be materialized and merged into \(W_0+\Delta W\), so the additional generator cost mainly affects training rather than inference.

Benefit from a Larger Latent Transformation Space. We select three representative subsets from the three benchmarks for evaluation. All training hyperparameters are kept identical for LoRA and GenFT. Here, \(r\) denotes the LoRA rank, while \(a\) and \(b\) denote the shared and layer-specific dimensions in GenFT, respectively. Results are summarized in Table 6. As shown, GenFT can use a larger latent transformation dimension \(a+b\) with fewer trainable parameters than LoRA, leading to better or comparable performance under a small parameter budget.

More Experiments Results. To facilitate a comprehensive evaluation of GenFT, we present supplementary empirical results. Specifically, Appendix E analyzes the impact of the layer-shared dimension on final performance. Appendix F elucidates the distinctions between GenFT and LoRA regarding row and column information. Furthermore, Appendix G investigates how different initialization strategies influence model effectiveness.

5 Conclusion↩︎

In this work, we propose GenFT, a \(W_0\)-conditioned PEFT method that uses a deterministic generator to produce task-specific updates \(\Delta W\). GenFT extracts structured row- and column-wise information from \(W_0\) and uses a shared-specific decomposition to balance cross-layer reuse and layer-specific flexibility. Experiments on NLP and CV benchmarks show competitive or better average performance than representative PEFT baselines, while a pilot study on LLaMA-7B suggests its feasibility for generative models.

6 Acknowledgment↩︎

This work is supported by RGC Research Matching Grant (RMGS-2025-01-11), Guangdong Basic and Applied Basic Research Foundation (2025A1515011674), Guangdong-Hong Kong Universities “1+1+1” Joint Research Scheme (UICR0800008-24), National Key R&D Program of China (2024YFE0202900), and RGC GRF (12300125).

7 Explanation of Finding 1↩︎

Finding 1. Given a model with \(L>1\) layers, let \(a\) denote the layer-shared dimension, \(b\) the layer-specific dimension, and \(r\) denote LoRA’s low-rank dimension. Under a comparable parameter budget, GenFT can use a larger latent transformation dimension than LoRA.

Explanation. For the \(\ell\)-th layer, LoRA parameterizes the update as \[\Delta W_{\text{LoRA}}^{(\ell)} = A_{\text{LoRA}}^{(\ell)} B_{\text{LoRA}}^{(\ell)},\] where \(A_{\text{LoRA}}^{(\ell)} \in \mathbb{R}^{D \times r}\) and \(B_{\text{LoRA}}^{(\ell)} \in \mathbb{R}^{r \times D}\) are learnable parameters. Thus, the total number of LoRA parameters over \(L\) layers is \[\text{Total \# of LoRA} = 2LDr .\]

For GenFT, the update of the \(\ell\)-th layer is generated from \(W_0^{(\ell)}\) as \[\begin{align} U^{(\ell)} &= U_s U_s^T + B^{(\ell)}(A^{(\ell)})^T,\\ F_{\textit{row}}^{(\ell)} &= \sigma_1(\textit{ratio} \cdot W_0^{(\ell)} U^{(\ell)})\odot M_p,\\ V^{(\ell)} &= V_s V_s^T + B^{(\ell)}(A^{(\ell)})^T,\\ F_{\textit{col.}}^{(\ell)} &= \sigma_2((F_{\textit{row}}^{(\ell)})^T V^{(\ell)}) \odot M_p,\\ \Delta W_{\text{GenFT}}^{(\ell)} &= \textit{scaling} \cdot F_{\textit{col.}}^{(\ell)}, \end{align}\] where \(U_s,V_s \in \mathbb{R}^{D \times a}\) are shared across layers, and \(A^{(\ell)},B^{(\ell)} \in \mathbb{R}^{D \times b}\) are layer-specific parameters. Therefore, the total number of GenFT parameters is \[\text{Total \# of GenFT} = 2Da + 2LDb .\]

Suppose GenFT and LoRA have the same parameter count: \[2Da + 2LDb = 2LDr .\] Then, \[a = L(r-b).\] If \(r>b\) and \(L>1\), we have \[a+b-r = L(r-b)+b-r = (L-1)(r-b) > 0,\] and hence \[r < a+b .\] This indicates that, under the same parameter budget, GenFT can allocate a larger latent transformation dimension than LoRA. Here, \(a+b\) should not be interpreted as the exact algebraic rank of \(\Delta W_{\text{GenFT}}^{(\ell)}\), since nonlinear activations and masking may change the resulting matrix rank.

8 Benchmarks Details↩︎

Here, we introduce the details of the selected benchmarks: VTAB-1k, FGVC, and GLUE, which are widely used to evaluate the generalization and robustness of machine learning models across diverse tasks and domains.

The VTAB-1k Benchmark is designed to assess the adaptability of visual representation learning models. It comprises 19 diverse visual tasks, grouped into three categories: natural, specialized, and structured. Each task is limited to 1,000 training examples to simulate low-data scenarios, challenging models to leverage pre-trained representations effectively. Table 7 provides detailed statistics and descriptions of the VTAB-1k benchmark, including task categories, and dataset sizes.

The FGVC benchmark focuses on tasks requiring fine-grained discrimination within specific categories, such as identifying species of birds, models of cars, or types of aircraft. FGVC datasets typically involve high intra-class similarity and inter-class variability, making them challenging for visual recognition systems. Notable datasets include CUB-200-2011 (birds), NABirds, Flowers102, Stanford Cars, and Stanford Dogs. Table 8 summarizes the FGVC benchmark, including dataset sizes and number of classes.

The GLUE benchmark is a collection of nine natural language understanding (NLU) tasks aimed at evaluating models’ ability to generalize across diverse linguistic tasks. These tasks cover single-sentence classification, similarity and paraphrase detection, and natural language inference (NLI), with varying dataset sizes and domains to test sample-efficient learning and cross-task knowledge transfer. Table 1 presents the details of the GLUE benchmark, including training and test set sizes, task, task types, evaluation metrics.

9 Experiments Hyperparameter Details↩︎

The experiments across the VTAB-1K, FGVC, and GLUE benchmarks utilize a comprehensive set of hyperparameters to fine-tune model performance and training efficiency. Under the GenFT category, hyperparameters such as \(W_0\) Ratio, Initial method of \(A\) (Init. \(A\)), Initial method of \(B\) (Init. \(B\)), \(\sigma_1\) and \(\sigma_2\) Activations, Shared and Specific Dimensions, Bias, Dropout, and Scaling control the feature transformation and adaptation processes, ensuring the model effectively captures task-specific patterns while maintaining generalization. The Training category includes hyperparameters like Seed, Optimizer, Number of GPUs, GPU Memory, Label Smoothing, Batch Size, Learning Rate, Weight Decay, Epochs, Warmup Epochs, Cycle Decay, and Training Time, which govern the optimization process, resource allocation, and training dynamics to achieve stable and efficient convergence. Detailed hyperparameter configurations for each benchmark are provided in their respective tables: Table 10 for VTAB-1K, Table 11 for FGVC, and Table 12 for GLUE.

For initialization methods, we search Kaiming Uniform (K-U), Xavier Uniform (X-U), Normal (N), and Zeros (Z). For activation functions, we try ReLU (R), LeakyReLU (LR), Tanh (T), GeLU (G), and Identity (I).

10 Results on FGVC↩︎

Implementation Details. We adopt the ViT-Base model [30] (85.8M parameters) for multi-class and fine-grained image classification. We evaluate GenFT on VTAB-1K [31], which contains 19 sub-datasets spanning natural, specialized, and structured images, and on FGVC, including CUB200-2011 (CUB200) [59], NABirds [60], Flowers102 (Flowers) [61], Stanford Dogs (Dogs) [62], and Stanford Cars (Cars) [63]. We report top-1 accuracy as the primary evaluation metric. Our experimental setup follows V-PETL Bench [32]. We conduct random hyperparameter search over the layer-shared dimension \(a\), layer-specific dimension \(b\), activation functions \(\sigma_1\) and \(\sigma_2\), and initialization schemes for \(A_{\textit{specific}}\) and \(B_{\textit{specific}}\), using the AdamW optimizer with a preset weight decay rate. To ensure reproducibility, we fix all random seeds.

Results on FGVC. Results are presented in Table 13. GenFT ranks second on CUB200, NABirds, Flowers, and Cars, and third on Dogs, while achieving the highest average accuracy (90.38%) across the benchmark. With only 0.29M parameters, GenFT leverages layer-shared and layer-specific information via the rank-decomposition policy and extracts structured information from \(W_0\) through row and column transformations with activation functions. This design yields robust generalization and stable performance across fine-grained classification tasks.

11 Layer-shared Dimension Analysis.↩︎

Motivated by the ablation results showing that removing the layer-shared dimension substantially degrades performance, we evaluate GenFT with varying layer-shared dimensions on VTAB-1K and report accuracy. Fig. 3 shows the performance trends, where bar labels indicate the corresponding dimensions. For specialized images, a shared dimension of 82 is critical for Caltech101, improving accuracy by about 1%, whereas SVHN benefits from a smaller dimension. Overall, performance is less sensitive to the shared dimension on specialized images. In contrast, for structured images, Clevr/Count and Kitti exhibit larger performance variations as the shared dimension changes, suggesting higher sensitivity due to their reliance on structured patterns. GenFT’s rank-decomposition policy helps adapt the layer-shared dimension to such structured image tasks.

12 Row and Column Information↩︎

We visualize row and column features on EuroSAT. Figures 4–6 show the pretrained weights \(W_Q\), \(W_V\), and the adapted weights \(W_0+\Delta W\) from LoRA and GenFT. LoRA makes only slight changes, suggesting a relatively uniform transformation, whereas GenFT captures more salient row and column patterns. Further comparisons in Figures 7 and 8 show that GenFT produces larger variations, indicating a weighted transformation that emphasizes important structures in \(W_0\) and yields a more effective \(\Delta W\).

13 Initialization Method Analysis↩︎

We further analyze how different initialization methods affect the best-reported performance. We select three subsets from the three benchmarks and compare performance when varying the initialization scheme. The results are presented in Table 14. Overall, the choice of initialization method has only a minor impact on the best performance, although an appropriate choice can still improve results.

14 Experiments Results Details↩︎

Here, Table 15 supplements the main comparison on VTAB-1K, presenting comprehensive results across all 19 datasets. Table 16 complements the ablation study on VTAB-1K, fully displaying ablation results for all 19 datasets. Table 17 extends the FGVC datasets, providing complete ablation results.

References↩︎

---

1. Corresponding author.↩︎