[Paper Review] RMGN: A Regional Mask Guided Network for Parser-free Virtual Try-on

Lin Chao, Zhao Li+7 more|arXiv (Cornell University)|Apr 24, 2022

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper proposes RMGN, a parser-free virtual try-on network that uses an adaptive regional mask to explicitly fuse features from reference persons and target clothes, eliminating residual artifacts from original clothing. By incorporating a multi-level feature extractor and posture-aware loss, RMGN achieves state-of-the-art performance in high-resolution, complex-posture settings, outperforming both parser-based and parser-free methods in fidelity and realism.

ABSTRACT

Virtual try-on(VTON) aims at fitting target clothes to reference person images, which is widely adopted in e-commerce.Existing VTON approaches can be narrowly categorized into Parser-Based(PB) and Parser-Free(PF) by whether relying on the parser information to mask the persons' clothes and synthesize try-on images. Although abandoning parser information has improved the applicability of PF methods, the ability of detail synthesizing has also been sacrificed. As a result, the distraction from original cloth may persistin synthesized images, especially in complicated postures and high resolution applications. To address the aforementioned issue, we propose a novel PF method named Regional Mask Guided Network(RMGN). More specifically, a regional mask is proposed to explicitly fuse the features of target clothes and reference persons so that the persisted distraction can be eliminated. A posture awareness loss and a multi-level feature extractor are further proposed to handle the complicated postures and synthesize high resolution images. Extensive experiments demonstrate that our proposed RMGN outperforms both state-of-the-art PB and PF methods.Ablation studies further verify the effectiveness ofmodules in RMGN.

Motivation & Objective

• To address the persistent issue of residual artifacts from reference clothing in parser-free virtual try-on methods.
• To improve high-resolution image synthesis in virtual try-on without relying on human parsing.
• To enhance robustness to complex human postures in virtual try-on generation.
• To design a generator that explicitly fuses relevant features from person and clothing while suppressing distracting features.
• To achieve performance on par with or better than parser-based methods without requiring parsing annotations.

Proposed method

• A multi-level feature extractor is used to separately extract high-resolution features from reference persons and target clothes.
• A regional mask is adaptively generated from features of both inputs to guide feature fusion, replacing the need for parsing.
• The regional mask enables selective fusion by emphasizing target cloth regions and suppressing original clothing features.
• A posture awareness loss is introduced to focus the model on structural consistency and body margins, improving generalization across postures.
• The generator uses the regional mask to combine features in a way that preserves target cloth patterns and colors.
• The model is trained end-to-end with adversarial and perceptual losses, using cyclic consistency and fake image sampling for enhanced stability.

Experimental results

Research questions

• RQ1 Can a parser-free virtual try-on method achieve high-fidelity results without relying on human parsing?
• RQ2 How can feature fusion be improved in parser-free VTON to eliminate residual artifacts from the reference person's clothing?
• RQ3 Can a learnable regional mask effectively replace segmentation maps for guiding feature fusion in virtual try-on?
• RQ4 To what extent can a posture-aware loss improve generalization across complex human postures in virtual try-on?
• RQ5 Can a parser-free method achieve performance comparable to or better than parser-based methods in high-resolution settings?

Key findings

• RMGN achieves a Fréchet Inception Distance (FID) of 9.93 on the VITON dataset (512×384), outperforming state-of-the-art methods.
• Ablation studies confirm that removing the regional mask leads to visible black residuals from original clothing, confirming its role in artifact suppression.
• The posture awareness loss significantly improves structural consistency, especially in complex postures, as shown in qualitative comparisons.
• Increasing the number of fake images used in the posture awareness loss improves image quality, demonstrating its effectiveness.
• User studies show that RMGN-generated images are perceived as more realistic than those from baselines in A/B testing.
• Visualization of the regional mask confirms its ability to adaptively focus on target cloth regions and suppress original clothing, even under varying sleeve lengths and postures.