[Paper Review] Disentangled Cycle Consistency for Highly-realistic Virtual Try-On

Chongjian Ge, Yibing Song+4 more|arXiv (Cornell University)|Mar 17, 2021

Generative Adversarial Networks and Image Synthesis45 references4 citations

TL;DR

This paper proposes Disentangled Cycle-consistency Try-On Network (DCTON), a novel framework that disentangles virtual try-on into three distinct modules—clothes warping, skin synthesis, and image composition—enabling highly realistic, photo-realistic results through self-supervised cycle consistency training. DCTON achieves state-of-the-art performance on benchmark datasets, including 87.68% user preference on VITON and real-time inference at 19 FPS.

ABSTRACT

Image virtual try-on replaces the clothes on a person image with a desired in-shop clothes image. It is challenging because the person and the in-shop clothes are unpaired. Existing methods formulate virtual try-on as either in-painting or cycle consistency. Both of these two formulations encourage the generation networks to reconstruct the input image in a self-supervised manner. However, existing methods do not differentiate clothing and non-clothing regions. A straight-forward generation impedes virtual try-on quality because of the heavily coupled image contents. In this paper, we propose a Disentangled Cycle-consistency Try-On Network (DCTON). The DCTON is able to produce highly-realistic try-on images by disentangling important components of virtual try-on including clothes warping, skin synthesis, and image composition. To this end, DCTON can be naturally trained in a self-supervised manner following cycle consistency learning. Extensive experiments on challenging benchmarks show that DCTON outperforms state-of-the-art approaches favorably.

Motivation & Objective

• To address the limitations of existing virtual try-on methods that either rely on one-way reconstruction or vanilla cycle consistency, both of which suffer from poor texture preservation and coupled image content generation.
• To improve visual realism and structural fidelity in virtual try-on by explicitly disentangling clothing warping, skin synthesis, and image composition into separate, trainable modules.
• To enable effective self-supervised training via cycle consistency while maintaining high-quality generation of fine-grained details such as sleeves, collars, and body parts.
• To achieve end-to-end training with improved performance and computational efficiency compared to prior methods like ACGPN and CP-VTON.

Proposed method

• DCTON decomposes virtual try-on into three sub-networks: a clothes warping module using a spatial transformer network (STN) with a novel regularization term to preserve texture details.
• A skin synthesis module is introduced to generate realistic human skin in occluded regions (e.g., arms, torso), guided by prior features from a dedicated encoder.
• An image composition module fuses warped clothes and synthesized skin into a final try-on image, ensuring global coherence and realistic appearance.
• The entire framework is trained in a cycle-consistent manner: input person image with source clothes is transformed to target clothes, and the cycle is closed by reconstructing the original image, enabling self-supervised learning.
• The method uses a disentangled cycle consistency loss that separates content generation into distinct, controllable components, reducing interference between clothing and non-clothing regions.
• A regularization term is added to the STN to stabilize warping and prevent distortion of fine textures like logos and embroidery.

Figure 1: Comparison of virtual try-on pipelines. The inpainting methods (e.g., CP-VTON [ 35 ] and ACGPN [ 40 ] ) shown in the top row use one in-shop clothes to replace the same input clothes. The vanilla CycleGAN [ 18 ] shown in the middle row introduces two in-shop clothes for cycle consistency a

Experimental results

Research questions

• RQ1 Can disentangling virtual try-on into clothes warping, skin synthesis, and image composition improve the realism and quality of generated try-on images?
• RQ2 Does a cycle-consistent training scheme with explicit disentanglement outperform vanilla cycle consistency or one-way inpainting in terms of visual fidelity and detail preservation?
• RQ3 Can the proposed method maintain high performance while reducing artifacts in complex regions such as arms, sleeves, and logos?
• RQ4 How does the inclusion of a dedicated skin synthesis module affect the realism of human skin in occluded areas?
• RQ5 To what extent does the proposed regularization in the STN improve texture preservation during clothes warping?

Key findings

• DCTON achieves a user preference score of 87.68% on the VITON test set, significantly outperforming CA-GAN (79.29%), CP-VTON (85.84%), and other SOTA methods.
• On the challenging VITON-HD dataset, DCTON demonstrates strong robustness and generalization, showing consistent improvement over baseline methods.
• The ablation study confirms that removing the skin synthesis encoder leads to blurry or discolored skin, highlighting its critical role in realism.
• The regularization term in the STN is essential for preserving fine textures—without it, warping causes visible distortion on logos and embroidered patterns.
• DCTON achieves 19 FPS inference speed on a single V100 GPU, nearly double the speed of ACGPN, while using fewer FLOPs (194G vs. 206G) and slightly more parameters (153M vs. 139M).
• Despite a longer training time (~44h vs. ~40h), DCTON maintains high-quality generation with better visual consistency and fewer artifacts compared to ACGPN and CP-VTON.

Figure 3: The pipeline of our disentangled cycle consistency framework. We show the CNN architecture above where there are clothes warping, skin synthesis, and image composition modules. The encoded features from these modules are concatenated to decode the output image. The cycle consistency is sho