1 Introduction↩︎

Recent advances in diffusion-based generative models have significantly improved image synthesis, enabling high-quality text-to-image generation and image-to-image translation [1]–[5]. Among their applications, virtual try-on (VTON) aims to synthesize a realistic image of a target person wearing one or more reference garments while preserving identity, pose, body structure, and scene context. This task is increasingly important for e-commerce and digital fashion.

Despite rapid progress, existing diffusion-based VTON methods remain limited in practical scenarios. Mask-based approaches [6]–[12] usually formulate VTON as an inpainting task and rely on segmentation masks, human parsing maps, or other spatial priors to define editable regions. Such preprocessing is error-prone under complex poses, occlusions, loose clothing, and cross-category garment transfer. For example, when replacing a T-shirt with a long coat or composing multiple garments, the editable region estimated from the target person may not match the shape or extent of the reference garments, leading to incomplete synthesis, geometric misalignment, and boundary artifacts. Mask-free methods [13]–[15] reduce the dependence on explicit spatial annotations by learning implicit garment-person alignment. However, they still face several limitations. Many existing methods introduce additional garment encoders or dual-branch architectures, increasing computational cost and reducing deployment efficiency. Full-parameter fine-tuning of large diffusion backbones can further increase training overhead and may weaken the general editing ability inherited from pretrained generative models. Moreover, most current VTON systems focus on single-garment transfer and provide limited support for flexible multi-garment composition.

To address these challenges, we propose TAMF-VTON, a texture-aware, mask-free virtual try-on framework that formulates garment transfer as a task-specialized image editing problem. TAMF-VTON requires no segmentation masks, human parsing, or auxiliary control signals during inference, and supports diverse garment styles, categories, and quantities. Central to our framework is an efficient expert-based adaptation mechanism that provides task-specific capacity for virtual try-on without overwriting the pretrained diffusion model. Together with texture-aware supervision and scalable training data construction, this design enables faithful texture preservation and flexible single- or multi-garment try-on within a unified generative framework.

Our key contributions are summarized as follows:

• We propose TAMF-VTON, a mask-free virtual try-on framework that performs high-fidelity garment transfer without segmentation masks, human parsing, or auxiliary control signals during inference.

• We introduce a lightweight MoE adaptation scheme with dynamic expert routing, which injects task-specific try-on expertise into the diffusion model while preserving the pretrained model’s general image editing capability.

• We design a texture-aware training objective with frequency-domain supervision to improve the preservation of fine-grained garment textures, patterns, and material appearance.

• We develop an adaptive inpainting-based data curation strategy that simulates the inverse virtual try-on process and generates high-quality training pairs with semantically aligned synthetic regions.

• We demonstrate flexible single- and multi-garment virtual try-on with practical inference efficiency, showing the potential of TAMF-VTON for scalable deployment in real-world e-commerce and digital fashion scenarios.

2 Related Work↩︎

2.1 GAN-based Virtual Try-On↩︎

Early image-based virtual try-on methods were mainly built upon generative adversarial networks (GANs) and commonly followed a two-stage pipeline: first deforming the reference garment to match the target pose, and then blending it with the person image [16]–[18]. VITON [19] introduced a coarse-to-fine framework with thin-plate spline (TPS) transformation for garment warping, while PF-AFN [20] improved alignment through appearance flow and knowledge distillation. Subsequent works further incorporated attention mechanisms or refined flow estimation to enhance geometric fidelity [21]. Despite these improvements, GAN-based methods remain sensitive to inaccurate warping and often struggle to synthesize photorealistic results with faithful preservation of fine-grained textures, such as patterns, fabric weaves, and material details.

2.2 Mask-based Virtual Try-On↩︎

Recent advances in diffusion models have catalyzed a new generation of high-fidelity virtual try-on (VTON) methods, most of which cast the task as reference-guided image inpainting. TryonDiffusion [6], StableVITON [7], LaDI-VTON [22], DCI-VTON [23], OOTDiffusion [24], IDM-VTON [25], FitDiT [10], and DreamFit [26] adopt a dual-branch architecture, where a ReferenceNet encodes the appearance and structure of the reference garment into features that are injected into a DenoisingNet via cross-attention. In contrast, concatenation-based approaches such as CatVTON [27], [28] and Voost [29] avoid duplicating the diffusion backbone by directly concatenating the target person and garment images as input to a single denoising network. M&M VTON [11] leverages a DiT backbone to enable multi-garment VTON by concatenating the tokens of the reference garments and target person.

Despite their strong visual quality, masked diffusion-based methods inherit several limitations from the inpainting formulation. First, the inpainting mask is typically derived from the target person’s original clothing, which may not match the category, shape, or spatial extent of the reference garment. This mismatch becomes particularly problematic in cross-category try-on and multi-garment composition. Second, the inpainting mask indiscriminately erases not only the original garment but also underlying body structure and background context, compromising body shape and scene consistency. Third, these methods fine-tune off-the-shelf text-to-image diffusion models. This scheme prioritizes global coherence over local texture-specific fidelity. Furthermore, fine-tuning the diffusion backbone incurs substantial computational overhead, consumes extensive training pairs, and compromises the generality of the base model.

2.3 Mask-Free Virtual Try-On↩︎

To overcome the dependency on explicit masks, recent works have explored mask-free VTON settings. BOOW-VTON [13], MFT-VITON [14], and All Parts Matter [15] learn garment-person alignment directly from image pairs, reducing the need for manually specified inpainting regions. These methods improve input flexibility and alleviate artifacts caused by inaccurate masks. However, existing mask-free methods still face important constraints. Many of them rely on additional garment-specific branches or large-scale backbone fine-tuning, which increases training and inference cost. Moreover, most are designed primarily for single-garment transfer and provide limited support for flexible multi-garment composition. Their training data construction also remains challenging, especially when the synthesized region must be semantically consistent with the target garment category.

In contrast, TAMF-VTON formulates virtual try-on as a task-specialized image editing problem rather than a mask-guided inpainting task. Our framework requires no segmentation masks, human parsing, or auxiliary control signals during inference. Instead of introducing a separate garment encoder or fine-tuning the full diffusion backbone, we propose a lightweight MoE adaptation scheme with dynamic expert routing to inject try-on-specific expertise while preserving the general editing capability of the pretrained model. By using token-wise concatenation, TAMF-VTON naturally supports both single-garment transfer and flexible multi-garment composition. We further introduce an adaptive inpainting-based data curation strategy for semantically aligned pseudo-pair construction, together with frequency-domain supervision to improve high-frequency texture consistency in the synthesized results.

3 Preliminaries↩︎

Flow matching [30] formulates the generative process as learning a continuous velocity field that transports samples from a noise distribution to the data manifold along straight-line trajectories. Given an image \(x_0\), its latent representation \(z_0\) and noisy latent \(z_t\) at time \(t \in [0,1]\) are defined as: \[z_0 = \mathcal{E}(x_0),\] \[z_t = (1 - t) z_0 + t \epsilon,\] where \(\mathcal{E}\) is a pre-trained VAE encoder and \(\epsilon \sim \mathcal{N}(0,I)\) denotes a random noise sample. The velocity \(v_t\) at time \(t\) is given by: \[v_t = \frac{dz_t}{dt} = \epsilon - z_0.\]

The flow model \(v_\theta\) (parameterized by \(\theta\)) takes as input the noisy latent \(z_t\), time \(t\) and conditioning signal \(c\) to predict the velocity \(v_t\). The model is optimized using a weighted mean squared error loss: \[\mathcal{L}_{\mathrm{FM}} = \mathbb{E}_{\epsilon \sim \mathcal{N}(0,I), t \sim \mathcal{U}(0, 1)}[\omega(t)\|v_{\theta}(z_t,t,c) - v_t\|_2^2],\] where \(\omega(t)\) is a positive weighting function, \(\mathcal{N}(0,I)\) denotes the standard isotropic Gaussian distribution, and \(\mathcal{U}(0, 1)\) is the uniform distribution over \([0, 1]\).

4 Method↩︎

As illustrated in Figure 1, TAMF-VTON adopts Qwen-Edit [31] for mask-free virtual try-on. Given a target person image, one or more reference garment images, and a text prompt, Qwen-VL extracts textual tokens, while the VAE encoder converts the target and reference images into latent visual tokens. These tokens are concatenated into a unified multimodal sequence and fed into the MMDiT backbone to predict the flow-matching velocity field.

TAMF-VTON introduces three key designs: a lightweight MoE adaptation module inserted into each linear layer for efficient task-specific specialization, a frequency-domain supervision objective for high-frequency texture preservation, and an adaptive inpainting strategy for scalable pseudo-pair construction. We describe these designs in Sections 4.1, 4.2, and 4.3, respectively.

4.1 Mixture-of-Experts Adaptation for Flexible Garment Try-On↩︎

We formulate mask-free VTON as a task-specialized image editing problem that requires simultaneously handling several strongly coupled objectives, including garment deformation, texture preservation, identity consistency, and background preservation. Unlike generic image editing, these objectives are spatially entangled: garment deformation should not distort body shape or non-target regions, while texture synthesis must remain consistent with the generated geometry. Such coupled constraints are difficult to model with a single static adaptation path, as used in conventional adaptation modules such as LoRA [32], IP-Adapter [33], and Redux [34], which apply identical transformations to all visual contexts.

To address this challenge, we introduce a lightweight Mixture-of-Experts (MoE) adaptation scheme for mask-free VTON. Each expert is implemented as an independent LoRA branch, while a token-wise router dynamically selects the most relevant experts for each token. Different from prior MoE-based editing frameworks such as ICEdit [35], which focuses on instruction-driven image editing, and TT-LoRA [36], which is designed for task-level multi-task adaptation, our method is tailored for spatially heterogeneous visual editing in VTON. Specifically, different spatial regions within the same image (e.g., garments, skin, hair, and background) often require different adaptation behaviors simultaneously. Therefore, instead of relying on predefined tasks, parsing priors, or handcrafted expert assignments, our framework learns token-dependent expert activation patterns directly from the try-on objective in an end-to-end manner.

Given an input token feature \(x \in \mathbb{R}^{d}\), the MoE-augmented linear layer is formulated as: \[y = Wx + \frac{\alpha}{r} \sum_{i=1}^{K} g_i(x)\, B_i A_i x,\] where \(W\) denotes the frozen backbone projection, \(A_i \in \mathbb{R}^{r \times d}\) and \(B_i \in \mathbb{R}^{d \times r}\) are the LoRA parameters of the \(i\)-th expert, \(r\) is the LoRA rank, and \(g_i(x)\) is the routing weight predicted by a lightweight router. We adopt top-\(2\) routing to balance adaptation capacity and computational efficiency.

During training, we freeze the entire pretrained diffusion backbone and optimize only the routers and LoRA experts. This parameter-efficient design preserves the general editing capability of the foundation model while introducing task-specific capacity for flexible single- and multi-garment try-on.

4.2 Frequency-Domain Supervision for Textural Detail Consistency↩︎

Fine-grained garment textural details, such as fabric grain, weave structures, logos, and local patterns, are often attenuated during diffusion-based synthesis. Since these details are primarily reflected in the high-frequency components of the Fourier spectrum, we introduce a frequency-domain supervision mechanism to explicitly encourage spectral consistency between the synthesized result and the ground-truth image. Unlike prior frequency-based losses FFL [37] for image reconstruction and synthesis, our method is designed specifically for diffusion-based mask-free VTON, where preserving globally consistent garment textures is critical under large geometric deformation and cross-category transfer. As illustrated in Figure 2, in the FFT-shifted spectrum, low-frequency components near the center mainly capture coarse structure and color distribution, while peripheral high-frequency regions encode fine textural details.

4.2.1 Spectral Decomposition and High-Frequency Loss↩︎

Given the prediction in latent space, we first reconstruct an estimate of the clean image \(x_0\) to enable spectral analysis. Under the flow matching formulation, the model predicts the velocity field \(\hat{v}_t = v_\theta(z_t,t,c)\). Since the ground-truth velocity is \(v_t = \epsilon - z_0\). Inverting this relation yields an estimate of the data latent: \[\hat{z}_0 = z_t - t\hat{v}_t.\]

The estimated latent is then decoded into pixel space using the frozen VAE decoder: \(\hat{x}_0 = \mathcal{D}(\hat{z}_0)\). Next, we apply the FFT to both \(\hat{x}_0\) and the ground-truth image \(x_0\), obtaining their spectral representations \(\mathcal{F}(\hat{x}_0)\) and \(\mathcal{F}(x_0)\). To focus on texture-related frequency components, we construct a radial high-frequency mask \(M_f\) on the FFT-shifted spectrum: \[M_f(u, v) = \begin{cases} 1, & \text{if } \sqrt{u^2 + v^2} > r, \\ 0, & \text{otherwise}, \end{cases}\] where \(r=(h+w)/32\) denotes a resolution-related cutoff radius and \((u,v)\) denotes the centered frequency coordinate.

Different from masked frequency supervision used in prior VTON method FitDiT  [10], we apply spectral constraints on the reconstructed full image rather than local garment regions. This is motivated by the fact that Fourier coefficients are globally coupled, such that spatial masking in the pixel domain does not correspond to localized frequency decomposition. Consequently, enforcing spectral consistency on masked regions may introduce inconsistent frequency constraints under large garment deformation or cross-category transfer.

Moreover, the reconstructed image \(\hat{x}_0\) becomes less reliable at high-noise timesteps. Directly imposing strong spectral supervision in such stages may therefore destabilize optimization. To address this issue, we introduce a timestep-dependent weighting term: \[\gamma(t) = (1-t)^2,\] which emphasizes frequency supervision at low-noise stage and suppresses it when \(t\) is close to the noise endpoint.

The final high-frequency spectral loss is formulated as: \[\mathcal{L}_{f} = \gamma(t) \cdot \frac{1}{\|M_f\|_1} \left\| M_f \odot \left( \mathcal{F}(\hat{x}_0) - \mathcal{F}(x_0) \right) \right\|_2^2,\] where \(\odot\) denotes element-wise multiplication.

4.2.2 Unified Training Objective↩︎

Our overall objective combines the standard flow matching loss \(\mathcal{L}_{\mathrm{FM}}\) with the proposed frequency-aware term \(\mathcal{L}_f\): \[\mathcal{L} = \mathcal{L}_{\mathrm{FM}} + \lambda\mathcal{L}_f,\] where \(\lambda\) controls the contribution of the spectral term. This diffusion-aware frequency supervision provides an effective inductive bias for preserving fine-grained garment textures while remaining fully compatible with mask-free VTON.

4.3 Adaptive Inpainting Strategy for Mask-Free Data Curation↩︎

Training a mask-free VTON model requires aligned training tuples consisting of a reference-agnostic person image, one or more reference garment images, and the corresponding ground-truth try-on image. However, existing datasets such as VITON-HD [38] and DressCode [39] usually provide only paired person-garment images, without pixel-aligned "before-wear" images. We therefore synthesize mask-free training tuples through an inverse VTON process, as illustrated in Figure 3.

Given a target person image \(I_{\mathrm{target}}\) (Image B) wearing garment \(G_{\mathrm{ref}}\) (Image A), we first construct a reference-agnostic base image \(I_{\mathrm{base}}\) (Image D) by replacing the original garment with a randomly sampled garment (Image C) using a mask-based VTON model. The generated \(I_{\mathrm{base}}\) preserves the target identity, pose, and background while removing garment-specific appearance from \(G_{\mathrm{ref}}\). We then rearrange the samples into a mask-free training triplet \((I_{\mathrm{base}}, G_{\mathrm{ref}}, I_{\mathrm{target}})\), where the model learns to recover the original target image from the agnostic base and the reference garment. The editing instruction for single-garment is: "The model in image 1 is dressed in the top/bottom/dress garment from image 2". This construction naturally extends to multi-garment and subject-to-subject settings by using multiple reference garments or donor-model garments.

4.3.1 Adaptive Inpainting Strategy↩︎

A key limitation of prior mask-free data pipelines [13], [14] is that their inpainting masks are usually built from the union of human parsing masks and the target person’s original garment mask. Such masks are tied to the originally worn clothing and often fail when the reference garment has a different category, length, or silhouette, such as replacing a T-shirt with a long coat. To generate more suitable agnostic bases for cross-category and multi-garment try-on, we propose an adaptive inpainting strategy that estimates the inpainting region according to both reference-garment semantics and target-person pose.

Specifically, we detect the main garment region in \(G_{\mathrm{ref}}\) (red rectangle in Figure 4) using a YOLOv10-based detector [40], obtaining its category and height-to-width ratio as coarse cues for garment type and length. We also extract normalized 2D body keypoints using OpenPose [41] to capture pose-dependent body layout. These cues are fed into a lightweight linear regressor to predict the garment hemline position in the target image, from which we construct an adaptive rectangular inpainting region. The final agnostic mask is the union of this predicted region and the original garment mask, ensuring coverage of both the expected reference-garment footprint and residual textures from the original clothing.

We further protect hands and feet using SAM2 segmentations [42] during inpainting, reducing limb distortion caused by overly large masks. We recompute agnostic masks for VITON-HD [38] and DressCode [39], and retrain FitDiT [10] with the updated masks to obtain a VTON-oriented inpainting prior. Compared with pipelines that rely on fixed parsing-based masks or chained multimodal models, our strategy provides a simple and scalable way to synthesize high-quality aligned training tuples for mask-free single- and multi-garment VTON. Importantly, the synthesized samples are not constrained by style, category, or length similarity between the reference garments and the garments originally worn by the target person, enabling diverse cross-category and cross-silhouette training examples.

5 Experiments↩︎

5.1 Experimental Setup↩︎

We train TAMF-VTON on two complementary datasets. The first is a synthetic benchmark dataset constructed from VITON-HD [38] and DressCode [39] using our data curation pipeline. It contains 20K balanced samples at \(768 \times 1024\) resolution across three categories: upper-body, lower-body, and dresses. The second is a 2K high-resolution dataset at \(1536 \times 2048\), synthesized from in-the-wild commercial fashion images to improve real-world generalization and texture fidelity.

Training follows a two-stage schedule: 10K steps on the 20K set and 5K fine-tuning steps on the 2K high-resolution set. We use a total batch size of 4 on NVIDIA A100 GPUs, FP8 quantization, AdamW8bit with learning rate \(5 \times 10^{-5}\), and a MoE adapter with rank 64, 4 experts, and top-2 routing. At inference, we use the Nunchaku acceleration engine [43]; TAMF-VTON produces one result under 15 seconds on an NVIDIA RTX 4090.

5.2 Qualitative Results↩︎

5.2.1 Standard Benchmarks Evaluation↩︎

We first evaluate TAMF-VTON on VITON-HD [38] and DressCode [39], which mainly involve garment transfer from flat-lay product images to studio-captured person images. We presents both single-garment and multi-garment try-on comparisons against three recent SOTA open-source baselines  [13]–[15]. For multi-garment transfer, the compared methods are applied sequentially to the upper- and lower-body garments.

As shown in the Figure 7, TAMF-VTON generates more realistic results with accurate garment geometry, natural draping, and superior preservation of identity and body structure. Our method also better retains fine-grained textures and material appearance, while existing approaches often produce blurred or hallucinated patterns, modify non-garment regions, or distort body contours near the transferred garments. Notably, mask-based pipelines are highly sensitive to segmentation errors, whose accumulated artifacts often prevent them from achieving product-level visual quality in practical scenarios. The advantage of TAMF-VTON becomes more pronounced in multi-garment settings, where competing methods frequently exhibit inconsistent garment layering, structural artifacts, or degraded texture fidelity after sequential editing.

5.2.2 Challenging Real-World Scenarios↩︎

We further evaluate TAMF-VTON on scenarios beyond standard benchmarks, including multi-garment composition, complex backgrounds, and subject-to-subject transfer. As shown in the second row of Figure [fig1], our method handles these challenging cases robustly, producing coherent garment layering, realistic textural appearance, and strong background preservation. Additional results are shown in Figures 8 and 9. These results demonstrate that TAMF-VTON generalizes well beyond controlled benchmark settings to more practical real-world scenarios.

5.3 Quantitative Results↩︎

We quantitatively evaluate TAMF-VTON on the test sets of VITON-HD [38] and DressCode [39]. As shown in Table 1, our method achieves the best or competitive performance across most metrics, outperforming both mask-based and mask-free state-of-the-art methods. The improved SSIM and LPIPS indicate better structural preservation and perceptual similarity, while lower FID and KID demonstrate stronger photorealism and distributional consistency.

In the unpaired setting, where ground-truth outputs are unavailable and evaluation relies on distributional metrics, TAMF-VTON achieves substantially lower FID and KID than prior methods. This improvement is particularly important for cross-category and cross-style try-on, where the reference garment may differ significantly from the originally worn clothing in length, silhouette, or category. These results suggest that TAMF-VTON better decouples person structure from garment appearance, enabling more realistic synthesis under diverse try-on conditions.

Additional comparisons with recent strong baselines (e.g., Qwen-Edit-2509  [31], Flux2-klein-9b  [44], and Nano Banana Pro  [45]) are provided in the supplementary material.

6 Ablation Study↩︎

6.1 Effect of MoE Adaptation↩︎

To evaluate the effectiveness of the proposed Mixture-of-Experts (MoE) adaptation, we compare it with the LoRA baseline, where the MoE module is replaced by a standard LoRA adapter while keeping the training data, optimizer, training steps, and all other components unchanged. As shown in Figure 5, the MoE variant produces more stable and realistic results across challenging cases. In contrast, the single-LoRA baseline shows limited generalization ability: in the first row, it fails to transfer the bottom garment; in the second row, it distorts the upper-garment silhouette and loses important style cues. Quantitative results in Table 1 further show that MoE adaptation consistently improves performance over the single-LoRA baseline across evaluation metrics. These results indicate that token-wise expert routing provides more effective conditional adaptation than a single shared adapter, which is particularly important for complex cases involving cross-category or multi-garment transfer.

6.2 Effect of Frequency-Domain Supervision↩︎

Figure 6 compares TAMF-VTON with and without the proposed frequency-domain supervision. Without this term, the model tends to generate oversmoothed or distorted local patterns, especially for garments with dense textures or fine fabric structures. In contrast, our full model better preserves high-frequency details, producing textile patterns that more faithfully match the reference garment. Quantitative results in Table 1 further show that frequency-domain supervision brings consistent improvements across evaluation metrics, confirming its effectiveness in enhancing texture fidelity beyond standard flow-matching training.

Additional ablation study on Top-\(k\) expert routing and frequency-domain supervision are presented in the supplementary material.

7 Limitations and Future Work↩︎

TAMF-VTON achieves high-quality mask-free virtual try-on with flexible multi-garment composition, but several challenges remain in extreme real-world scenarios. Under severe illumination variations, such as overexposed or underexposed conditions, the synthesized garment color may slightly deviate from the reference appearance. In addition, when previously occluded body regions become exposed after try-on, the generated skin details may differ from the target person due to the strong generative prior of the base model. These limitations remain common challenges for current virtual try-on systems. Future work will focus on illumination-aware color consistency and more identity-preserving synthesis for unconstrained real-world applications.

8 Conclusion↩︎

We present TAMF-VTON, a texture-aware, mask-free framework that formulates virtual try-on as a task-specialized image editing problem. By integrating a visual-context-aware Mixture-of-Experts adaptation scheme, frequency-domain supervision, and an adaptive data curation strategy, our method addresses the coupled challenges of garment deformation, texture fidelity, and identity preservation without relying on error-prone segmentation masks. Extensive experiments demonstrate that TAMF-VTON achieves superior perceptual quality and competitive quantitative performance compared with state-of-the-art methods, while supporting flexible multi-garment composition with efficient inference.

References↩︎