1 Introduction↩︎

Color retouching plays a crucial role in visual communication and digital photography, as it shapes the perceived style and atmosphere of an image. However, the strong interdependence among color channels makes precise adjustment difficult for non-experts. While traditional approaches like image filters and Look-Up Tables (LUTs) offer simplified solutions, they typically apply rigid transformations that disregard the intrinsic color distribution of the source image, often leading to suboptimal results. To overcome these limitations, many automated color transfer techniques have been proposed. Given a reference style image, these methods aim to transfer its color style to the content image while maintaining photorealism.

Broadly, existing methods can be categorized into two paradigms: online optimization and offline inference. Optimization-based approaches [1]–[6] are often instance-specific and computationally intensive. Moreover, their reliance on hand-crafted or imperfect optimization objectives frequently results in inaccurate color transfer. Conversely, offline techniques are predominantly learning-based [7]–[18]. Feature-transformation-based methods [7]–[9], [13] match deep statistics between images but often introduce visual artifacts or severe color banding. Mapping-based approaches [10], [11], [15] (, LUT generation [19], [20]) learn direct color transformations, yet their performance is limited by data scarcity and synthetic dataset biases, leading to poor generalization in complex real-world scenes. Furthermore, a recent flow-based method [12] focuses on learning global distribution alignment, often overlooking image content, which degrades visual quality.

Beyond these specific limitations, a more fundamental issue lies in the isolated perspective of existing approaches, in which online and offline paradigms are treated as disjoint. In this paper, we bridge this divide by proposing ColorFM, an optimization-to-learning framework that reformulates color transfer as transporting probability distributions via Flow Matching (FM) [21]–[23]. This formulation provides a unified view of instance-specific optimization (ColorFM-O) and feed-forward inference (ColorFM-L), while enabling consistency to be transferred from optimization to learning. As shown in 1, ColorFM-L achieves precise color transfer across diverse styles while retaining structural details and semantic alignment.

In summary, our contributions include:

(1) We propose ColorFM, an optimization-to-learning framework that formulates color transfer as the transport of pixel distributions in color space via Flow Matching. This framework seamlessly integrates precise instance-specific optimization with efficient feed-forward inference.

(2) We introduce ColorFM-O, an instance-specific optimization scheme. By optimizing the velocity fields with explicit semantic alignment and hierarchical color coupling, it enables precise color transfer. The resulting outputs serve as large-scale pseudo-supervised pairs for offline learning.

(3) We design ColorFM-L, a feed-forward model trained on the generated pseudo pairs. By predicting flow parameters through implicit state modeling, it executes efficient color transfer via bidirectional linearized transport. Experiments demonstrate that ColorFM-L achieves a superior balance between style and content, outperforming state-of-the-art methods in overall visual quality.

2 Related Work↩︎

2.1 Color Transfer↩︎

Color transfer, or photorealistic style transfer, is a technique designed to transfer the color style of a reference image to a content image. Distinct from style transfer tasks [24]–[30], color transfer requires preserving the original structure while maintaining photorealistic quality. Generally, existing approaches can be categorized into two main streams: online optimization and offline inference.

Online Optimization. Optimization-based methods typically require iterative tuning for each input pair. Early works [1], [2], [6] rely on linear mappings under rigid Gaussian assumptions, which fail to capture complex, multi-modal distributions. In the deep learning era, DPST [3] formulates the transfer objective by integrating a Matting Laplacian regularization to optimize the content image. However, its iterative optimization is computationally intensive, which can take several minutes. Similarly, NLUT [4] fine-tunes a network to predict image-adaptive LUTs [19] at test time, often introducing noticeable artifacts. More recently, D-LUT [5] utilizes score matching [31] to generate LUTs derived solely from the style image, thereby overlooking the content information. Regardless of their individual constraints, a common shortcoming across these methods is their inability to achieve precise, high-fidelity color transfer in complex scenarios. To address these limitations, we present ColorFM-O, which constructs velocity fields to enable precise and semantically aligned transfer.

Offline Inference. In contrast to online optimization approaches, offline inference methods [7]–[18] enable efficient color transfer. Early representative works [7]–[9], [13], [17], [18] employ Whitening and Coloring Transforms (WCT) for feature alignment. However, they are computationally prohibitive for high-resolution inputs and prone to visual artifacts. Unlike methods that rely on deep feature statistics, Neural-Preset [10] leverages a large collection of LUTs for self-supervised learning. However, even an extensive set of LUTs struggles to encompass the vast diversity of real-world color distributions, leading to limited stylistic expressiveness. Similarly, SA-LUT [11] generates synthetic training pairs by applying LUTs in the LOG color space. However, this strategy relies on an ideal alignment assumption that often fails with real-world chromatic discrepancies, leading to imperfect supervision and inconsistent stylization. Most recently, ModFlows [12] applies FM via a shared uniform intermediate color distribution. This indirect mapping severs semantic correspondence and restricts transfer to global means, often yielding visual artifacts and color banding. Furthermore, its reliance on numerical ODE integration precludes efficient one-step inference. To overcome these limitations, we propose ColorFM-L, which predicts flow parameters to enable rapid and precise color transfer while preserving structural integrity.

2.2 Flow Matching↩︎

Flow Matching (FM) [21]–[23] is a framework that models the continuous transport map between two distributions. Formally, it transports source samples to a target distribution by solving an ODE defined by a learned velocity field. Since sampling requires numerical integration, inference efficiency is strongly influenced by the curvature of the flow trajectories.

Ideally, straight trajectories enable efficient one-step or few-step inference. To achieve this, Rectified Flow [22], [32] straightens trajectories through iterative self-distillation. While effective, this approach is computationally expensive due to the necessity of multiple retraining rounds. Alternatively, Optimal Transport (OT) theoretically provides an ideal coupling for minimizing transport cost and straightening trajectories. However, computing exact OT plans scales cubically, making it intractable for dense distribution matching tasks. Mini-batch approximations like OT-CFM [33], [34] have been proposed to mitigate this cost, but they struggle to capture global distribution statistics, leading to suboptimal mappings [33], [35] (, color desaturation). In contrast, our hierarchical color coupling strategy effectively constructs a globally consistent coupling that yields quasi-linear flow trajectories for efficient inference.

3 ColorFM↩︎

3.1 Preliminaries: Flow Matching↩︎

FM [21]–[23] learns a continuous flow that transports samples from a source distribution \(\pi_{0}\) to a target \(\pi_{1}\). Given a source-target pair \((x_0, x_1)\) sampled from a joint coupling and a time step \(t \sim \mathcal{U}[0, 1]\), the intermediate state \(x_t\) follows a linear interpolation trajectory: \[\begin{align} x_t = (1-t) \cdot x_0 + t \cdot x_1, \end{align}\] the conditional velocity field \(v_{\theta}(t, x)\), parameterized by a neural network, is then optimized to match the conditional vector field \(u_t(x|x_0, x_1) = x_1 - x_0\). The FM objective is therefore: \[\begin{align} \label{eq:fm95loss} \mathcal{L}_{\textit{FM}} = \mathbb{E}_{t, x_0, x_1} \left[ \| v_\theta(t, x_t) - (x_1 - x_0) \|^2 \right]. \end{align}\tag{1}\] At inference, transport is performed by solving the ODE \(dx/dt = v_\theta(t, x)\) starting from \(x_0\), yielding the transported sample at \(t=1\).

3.2 Problem Formulation↩︎

We reformulate color transfer within the FM framework by learning an RGB-space velocity field from source-target color pairs. Given the content distribution \(\pi_{content}\) and the style distribution \(\pi_{style}\), the velocity field \(v_t\) evolves a sampled content pixel \(x_0=(r_0,g_0,b_0) \sim \pi_{content}\) toward a target color \(x_1 \sim \pi_{style}\).

Guided by this formulation, the core objective of ColorFM is to estimate the velocity field \(v_t\). ColorFM-O learns the field via iterative optimization for given image pairs, while ColorFM-L functions as an offline model that learns to predict the flow parameters under supervision.

3.3 ColorFM-O↩︎

As illustrated in 2 (a), ColorFM-O aims to learn high-fidelity color transport trajectories by directly optimizing the velocity field via FM. To facilitate this, we construct the source-target pairs through explicit semantic alignment and hierarchical color coupling.

Explicit Semantic Alignment. Prior approaches [3], [8], [9], [13] utilizing semantic masks typically process each region independently to ensure semantic alignment. However, this independent treatment often disrupts spatial continuity, resulting in visible seam artifacts or halo effects along object boundaries.

To address this issue, we integrate semantic correspondence into a single unified velocity field, thereby reconciling region-aware stylization with global consistency. Specifically, given semantic masks \(M_c\) and \(M_s\) generated by a pre-trained segmentation model, we extract pixel sets corresponding to each shared label \(l\) to form source-target distribution pairs \((\pi_{content}^{l}, \pi_{style}^{l})\). Meanwhile, pixels belonging to unmatched or background regions are aggregated into a residual global distribution pair. Notably, if no residual style region is available , the remaining content pixels are instead paired with the global style distribution. This process yields a collection of \(N\) distribution pairs \(\{(\pi_{content}^i, \pi_{style}^i)\}_{i=1}^N\). Instead of training separate models for each region, we optimize a unified velocity field \(v_\theta\) to learn the transport dynamics for all pairs simultaneously. This design implicitly imposes spatial regularization, enabling the network to achieve precise semantic alignment while maintaining seamless global coherence.

Hierarchical Color Coupling. After obtaining the semantic distribution pairs \(\{(\pi_{content}^i, \pi_{style}^i)\}_{i=1}^N\), the subsequent challenge lies in establishing a high-quality coupling between the source samples \(\mathcal{X}_0 \sim \pi_{content}^i\) and target samples \(\mathcal{X}_1 \sim \pi_{style}^i\) to construct the training set for the velocity field. Standard random matching [21], [22] often yields geometrically incoherent trajectories, causing severe color banding. However, alternative approximations like mini-batch OT [33], [34] fail to capture global distribution statistics, leading to color desaturation. To balance quality and efficiency, we propose the hierarchical color coupling (HCC) strategy.

Our key insight is that preserving relative locality in color space suffices to eliminate banding and ensure flow coherence. Guided by this, HCC recursively aligns distributions in a coarse-to-fine manner (see 3). Specifically, at each hierarchy level, we center the subsets by subtracting their respective means to align their relative positions, and then partition the space into octants based on the coordinate signs. This recursive process continues until a maximum depth is reached, after which points within corresponding leaf nodes are coupled via random pairing. Finally, by independently applying this strategy to each of the \(N\) semantic distribution pairs defined previously, we aggregate the coupled samples into a unified training dataset \(D = \{(x_0^j, x_1^j)\}_{j=1}^M\). This dataset \(D\) serves as a static, geometrically optimized coupling plan, ready for supervision.

Velocity Field Optimization. Given the constructed dataset \(D\), we optimize the network \(v_\theta\) using the objective in 1 . At inference time, stylization is performed by numerically integrating the learned ODE in a pixel-wise manner to transport the content image \(I_c\) to the stylized output \(I_{out}\).

We then synthesize a diverse collection of triplets \((I_c, I_s, I_{out})\) by executing ColorFM-O on various content-style pairs. These triplets provide the essential, semantically aligned supervision for ColorFM-L.

3.4 ColorFM-L↩︎

Despite the high-fidelity performance of ColorFM-O, its reliance on costly online optimization and per-instance hyperparameter tuning (, iteration steps) limits its scalability. To achieve real-time inference, we propose ColorFM-L (see 2 (b)). Instead of iterative solving, ColorFM-L is trained using the pseudo-ground truth images generated by ColorFM-O. Conceptually, ColorFM-L decouples the color transfer task into two collaborative stages: (1) predicting flow parameters via implicit state modeling, and (2) executing the color transfer via bidirectional linearized transport. We detail these components below.

Implicit State Modeling. Directly predicting the transport between arbitrary content-style pairs is non-trivial due to significant domain gaps. Existing methods [10], [12] typically rely on a fixed, pre-defined intermediate distribution to bridge this gap. However, restricting diverse images into a rigid, hand-crafted prior often limits the model’s expressivity.

To address this, we introduce an implicit state modeling scheme. Rather than adhering to a fixed prior, we hypothesize the existence of an optimal intermediate state within the content-style space that acts as a bridge. Accordingly, we employ a weight-sharing encoder \(E_\phi\) to predict the specific transport parameters that drive the flow toward this state, thereby implicitly modeling it. Specifically, \(E_\phi\) comprises a feature backbone \(E_{\text{feat}}\), a cross-attention block \(E_{\text{attn}}\), and a parameter generator \(E_{\text{param}}\). The parameter prediction process is formulated as follows: \[\begin{gather} f_c = E_{\text{feat}}(\tilde{I}_c), \quad f_s = E_{\text{feat}}(\tilde{I}_s), \\ \hat{f}_c = E_{\text{attn}}(f_c, f_s), \quad \hat{f}_s = E_{\text{attn}}(f_s, f_c), \\ \Theta_c = E_{\text{param}}([f_c, \hat{f}_c]), \quad \Theta_s = E_{\text{param}}([f_s, \hat{f}_s]). \end{gather}\] Here, \(\tilde{I}_c\) and \(\tilde{I}_s\) denote the content and style images resized to a fixed resolution, and \([\cdot, \cdot]\) represents the concatenation operation. \(\hat{f}_c\) and \(\hat{f}_s\) are the features generated via cross-attention using deep semantic features \(f_c\) and \(f_s\) as queries, respectively. Importantly, \(\Theta_c\) and \(\Theta_s\) denote the predicted weights of a globally shared, bias-free pixel-wise MLP. This MLP explicitly defines the velocity field \(v(\cdot; \Theta)\) driving the flow toward the implicit state.

Bidirectional Linearized Transport. Given the parameters \(\Theta\), ColorFM-L executes the color transfer process. To accelerate inference, we leverage the quasi-linearity of ColorFM-O trajectories, which is inherently induced by the HCC strategy. This geometric regularity allows us to approximate the flow path as a straight line, enabling efficient one-step transport without iterative integration.

Formally, we first flatten the spatial dimensions of the content image \(I_c \in \mathbb{R}^{3 \times H \times W}\) to obtain a pixel matrix \(m_c \in \mathbb{R}^{HW \times 3}\). The transfer process is formulated as a composition of a forward flow to the intermediate state and a backward flow to the target style domain: \[\begin{align} z = m_c + v(m_c, 0; \Theta_c), \; m_{out} = z - v(z, 1; \Theta_s), \end{align}\] where \(z\) denotes the intermediate implicit state. Effectively, this formulation corresponds to a bidirectional one-step Euler integration (, \(\Delta t = 1\)). Finally, the stylized feature \(m_{out}\) is reshaped back to \(\mathbb{R}^{3 \times H \times W}\) to yield the final output \(I_{out}\). Furthermore, we can perform the inverse color transfer by directly reusing the predicted parameters and simply reversing the flow (, mapping \(I_s\) forward via \(\Theta_s\) and then backward via \(\Theta_c\)).

Optimization Loss. The entire pipeline is differentiable and trained end-to-end using the following objective: \[\mathcal{L}_{total} = \|I_{out} - I_{gt}\|^2 + \lambda \cdot \mathcal{L}_{\textit{LPIPS}}(I_{out}, I_{gt}),\] where \(I_{gt}\) denotes the pseudo-ground truth synthesized by ColorFM-O, and \(\mathcal{L}_{\textit{LPIPS}}\) [36] represents the perceptual loss. We set \(\lambda = 0.1\) to balance pixel-wise accuracy with perceptual fidelity.

4 Experiments↩︎

4.1 Implementation Details↩︎

Datasets. We use the optimization-based ColorFM-O to construct a large-scale dataset of 237,408 triplets \((I_c, I_s, I_{out})\). Source images are collected from Unsplash and DIV2K [37]. To ensure semantic consistency, Unsplash images are categorized into seven classes (Urban, Portrait, Nature, Indoor, Animal, Still Life, and Random) with intra-category pairing. In contrast, DIV2K images are randomly paired to enhance diversity. This data composition enables ColorFM-L to effectively learn both semantic alignment and precise color transfer.

For the test dataset, we collected 40 high-quality images from Unsplash and performed exhaustive pairwise permutations to create 1,560 content-style pairs, ensuring strictly no overlap with the training set. This dataset covers diverse semantic scenes, enabling a comprehensive evaluation of model performance.

Network Configuration. For ColorFM-O, the velocity field is parameterized by a lightweight two-layer bias-free MLP with 512 hidden units and the Swish activation [38]. We use a pre-trained SegFormer-B5 [39], fine-tuned on ADE20K [40], for semantic mask extraction. Hierarchical color coupling is configured with a maximum tree depth of \(D_{max}=3\).

For ColorFM-L, we adopt ViT-S/16 [41], [42] as \(E_{\text{feat}}\) with input images resized to \(256 \times 256\). \(E_{\text{attn}}\) is implemented as a cross-attention layer following the design of a standard ViT attention block. The parameter generator \(E_{\text{param}}\) then applies global average pooling and multiple MLP heads to project features into \(\Theta\). These parameters instantiate a globally shared pixel-wise MLP with channel dimensions \(4 \!\to\! 16 \!\to\! 16 \!\to\! 16 \!\to\! 3\) and Swish activation, where the input includes the time coordinate.

Training and Inference. For ColorFM-O, we use the Adam [43] optimizer with a learning rate of \(5 \times 10^{-4}\). The optimization is run for 700 steps with a batch size of 4,096 sampled pixels. During inference, the ODE is solved using the midpoint method with 5 integration steps to ensure trajectory precision.

For ColorFM-L, we use the Adam optimizer with a batch size of 48. The model is trained for 50 epochs. The learning rate is initialized to \(1 \times 10^{-5}\) and maintained for the first 20 epochs, after which it is gradually decayed to \(1 \times 10^{-6}\) using a cosine annealing schedule over the remaining 30 epochs. All experiments are conducted on a single NVIDIA RTX 4090 GPU.

4.2 Comparisons↩︎

We compare both ColorFM-O and ColorFM-L against state-of-the-art methods: WCT\(^2\) [8], PhotoWCT\(^2\) [9], Deep Preset [16], NLUT [4], CAP-VST [13], SA-LUT [11], D-LUT [5], Neural Preset [10], and ModFlows [12]. As Neural Preset is only available via a mobile application, which prevents large-scale automated evaluation, we restrict its comparison to qualitative visual results.

5pt

Quantitative Metrics. Following the evaluation protocols established in previous works [5], [8], [10], [12], we evaluate performance along two primary dimensions: similarity and regularity. For similarity, we employ Style Similarity (using the pre-trained discriminator from Neural Preset [5], [10], with range \([0,1]\)) and Content Similarity (computed as SSIM [44] on LDC-extracted edge maps [45]). To quantify the balance, we report the Distance to Ideal [5], [8], [10], [12], defined as the Euclidean distance to the optimal point \((1, 1)\), where a shorter distance indicates a better trade-off between content preservation and style alignment. For regularity, we report the Lipschitz Constant as used in ModFlows [12], where lower values correspond to fewer artifacts and less color banding. Detailed metric formulations are provided in Supplementary Appendix C.

Quantitative Results. ¿tbl:tab:quantitative? presents a comprehensive evaluation of ColorFM-O and ColorFM-L against prior methods. Optimization-based models are marked with gray backgrounds. In terms of similarity metrics, ColorFM-L and ColorFM-O attain the top two results in the “Distance to Ideal” metric, as shown in 5, indicating the optimal balance between content preservation and style alignment. While ModFlows and CAP-VST exhibit slightly higher style similarity, they suffer from structural degradation. Conversely, Deep Preset and D-LUT maintain high content fidelity but fail to effectively transfer complex color styles. Regarding regularity, ColorFM-L achieves the lowest Lipschitz constant among all methods. This result indicates that our model learns a smooth and stable color mapping, effectively suppressing visual artifacts and color banding prevalent in competing approaches. In terms of efficiency, ColorFM-L achieves real-time inference (0.016 s) and maintains exceptional speed, processing 4K resolution images (3840 \(\times\) 2160) in 0.043 s on average. A comprehensive runtime analysis across different resolutions is provided in the Supp. Tab. 1.

Notably, ColorFM-L outperforms ColorFM-O. This phenomenon can be attributed to two primary factors. On the one hand, ColorFM-O employs a fixed optimization setting (, fixed iteration steps) across all test pairs. This may lead to underfitting or overfitting for samples with varying complexity [46]. On the other hand, ColorFM-L leverages neural network inductive biases [47] and large-scale training to capture global priors. This enables it to smooth out the irregularities of instance-specific optimization, resulting in more robust transfer.

Qualitative Results. 4 provides visual comparisons of ColorFM-L with competing methods across diverse content-style pairs. As shown, WCT\(^2\) and CAP-VST often introduce spatial distortions or visual artifacts, while D-LUT and Neural Preset tend to produce under-stylized results with insufficient color transfer. Meanwhile, ModFlows tends toward global color averaging; by neglecting semantic correspondence, it leads to visible color bleeding and artifacts. In contrast, ColorFM-L not only performs effective semantically aligned color transfer but also preserves the original image structure. For instance, in Rows 1 and 2, our method accurately distinguishes and aligns the colors of the sky and ground. In Row 3, ColorFM-L successfully transfers the style without degrading the facial details (see the zoomed-in patches). Furthermore, in Row 4, our method retains fine-grained textures, ensuring that the text on the building remains clearly visible. For a detailed analysis of ColorFM-O and ColorFM-L, please refer to 5.2.

4.3 Ablation Studies↩︎

Effectiveness of Explicit Semantic Alignment. [fig:ab95semantic] visualizes the impact of semantic alignment in ColorFM-O. Without semantic guidance, the model fails to establish correct semantic correspondence, resulting in imprecise color mapping (, the sky color is inconsistently transferred). In contrast, enabling explicit alignment enforces region-aware transfer, ensuring that colors are accurately mapped to semantically corresponding regions.

5pt

Effect of Hierarchical Color Coupling. ¿tbl:tab:HCC95ablation? investigates the impact of the HCC strategy. We randomly select a subset of 20 images (380 pairs) from the test set for evaluation. While 2-Rectified Flow straightens trajectories, it inherits suboptimal style mapping from the initial random coupling in 1-Rectified Flow, resulting in low style similarity. Similarly, Mini-batch OT incurs high computational cost and suffers from severe statistical distortion. In contrast, our HCC strategy achieves a better balance. Increasing the subdivision depth \(D_{max}\) from 0 to 3 reduces the Lipschitz value from 5.811 to 2.937, indicating a smoother learned mapping. Further increasing \(D_{max}\) to 5 raises computational cost and leads to degraded style similarity. Thus, we adopt \(D_{max}=3\) as the default setting.

Number of Sampling Steps. Following the experimental setup in ¿tbl:tab:HCC95ablation?, we evaluate the impact of the number of ODE integration steps. As shown in 6, the performance of ColorFM-O saturates at \(T \geq 5\). We therefore adopt a 5-step sampling strategy for ODE integration. This rapid convergence suggests the quasi-linearity of the learned trajectories. To verify this quantitatively, we compute the path length ratio over 100 sampling steps, yielding an average value of 1.009. This inherent linearity minimizes rectification complexity, serving as a key prerequisite for the efficient one-step inference in ColorFM-L.

Impact of ColorFM-L Components. 1 analyzes the architectural components of ColorFM-L. Estimating the transport parameters independently (Row b) outperforms direct mapping (Row a), highlighting the structural advantage of the bidirectional formulation. We further add a fixed-state baseline following [10] to isolate the effects of static and implicit state modeling. The implicit variants further boost performance: specifically, Cross-Attention (Row e) demonstrates stronger modeling capability than simple concatenation (Row d). We therefore adopt Row e as the default design for ColorFM-L.

5 Discussion↩︎

5.1 Robustness of ColorFM-O to Imperfect Masks↩︎

[fig:comparison95masks] demonstrates ColorFM-O’s robustness to coarse and inaccurate masks. Since the masks only guide semantic distribution pairing instead of defining hard regional transformations, local mask errors do not directly introduce discontinuous mappings. The final transfer is then governed by a globally unified velocity field, which is optimized over the entire image rather than applied independently to each region. The quantitative results further show that ColorFM-O remains stable under moderate mask degradation, where the style similarity decreases only slightly from 0.745 to 0.731/0.722 under 20%/40% semantic misclassification, while a more noticeable drop to 0.695 appears only under severe degradation of 60%. Full quantitative results are provided in Supp. Tab. 3.

5.2 Comparative Analysis of ColorFM-O and ColorFM-L↩︎

As shown in 7, while both methods perform comparably in simple scenarios (Row 1), complex scenes reveal their differences. Specifically, when distinct semantic regions exhibit varying distribution gaps and transport complexities, the fixed setting of ColorFM-O (, a fixed number of training steps) struggles to establish a precise unified velocity field for all regions simultaneously. In contrast, ColorFM-L achieves robust color transfer by leveraging large-scale data supervision. This suggests that large-scale learning can help ColorFM-L smooth instance-specific optimization irregularities, leading to more stable transfer across diverse scenes.

5.3 Implicit State Visualization↩︎

[fig:state] visualizes the intermediate states \(Z_A\) and \(Z_B\), extracted during the \(I_A \to I_B\) and \(I_B \to I_A\) color transfer processes, respectively. Notably, both states preserve their original structures while converging to a shared color distribution, perceptually appearing as an averaged intermediate style that blends the color characteristics of both \(I_A\) and \(I_B\). This observation suggests that the implicit state effectively bridges diverse distributions into a unified color manifold. Moreover, ColorFM-L enables efficient bidirectional transfer (, \(I_A \leftrightarrow I_B\)): by computing the transport parameters once, it reuses them for the reverse path, avoiding redundant network evaluations.

5.4 Extension to Video Color Transfer↩︎

We extend ColorFM-L to video stylization by sampling a representative frame to compute transfer parameters conditioned on the content/style image. These fixed parameters are then applied to all frames via bidirectional linearized transport, ensuring a consistent style-conditioned mapping across the sequence without using video-specific temporal modules or optical-flow-based constraints. As shown in 8, this approach avoids the frame-by-frame prediction variance that typically causes flickering artifacts.

6 Conclusion↩︎

In this paper, we proposed ColorFM, a novel optimization-to-learning framework for color transfer. ColorFM bridges online optimization and offline inference by modeling color transfer as pixel distribution transport via Flow Matching. Specifically, ColorFM-O performs instance-specific optimization through hierarchical color coupling with semantic priors, generating high-quality pseudo-ground truths. ColorFM-L subsequently leverages these data for supervised learning. It predicts flow parameters via implicit state modeling and executes color transfer through bidirectional linearized transport. Extensive experiments demonstrate that ColorFM-L outperforms existing state-of-the-art methods.

Despite these advantages, ColorFM has several limitations. For example, ColorFM-O inherits the sensitivity of optimization-based approaches and may require per-instance hyperparameter tuning to achieve optimal performance. Moreover, ColorFM-L may face generalization challenges for extreme color styles or out-of-distribution semantic layouts.

Acknowledgements↩︎

This work was supported by the National Natural Science Foundation of China (Grant No. 62572234), the Gusu Innovation and Entrepreneurship Leading Talent Program (Grant No. ZXL20254324), and the Suzhou Key Technologies Project (Grant No. SGY2023136).

References↩︎