[Paper Review] Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping

Huan Fu, Mingming Gong+4 more|arXiv (Cornell University)|Sep 16, 2018

Generative Adversarial Networks and Image SynthesisComputer Science58 references18 citations

TL;DR

This paper proposes Geometry-Consistent GAN (GcGAN), a one-sided unsupervised domain translation method that enforces geometric consistency by ensuring that predefined image transformations (e.g., rotation) are preserved across domain mappings. By co-regularizing the generator on both original and transformed images, GcGAN reduces mode collapse and semantic distortion, achieving state-of-the-art performance on tasks like photo-to-map and Monet-to-photo translation with 18–22% higher pixel accuracy than baseline GANs.

ABSTRACT

Unsupervised domain mapping aims to learn a function to translate domain X to Y by a function GXY in the absence of paired examples. Finding the optimal GXY without paired data is an ill-posed problem, so appropriate constraints are required to obtain reasonable solutions. One of the most prominent constraints is cycle consistency, which enforces the translated image by GXY to be translated back to the input image by an inverse mapping GYX. While cycle consistency requires the simultaneous training of GXY and GY X, recent studies have shown that one-sided domain mapping can be achieved by preserving pairwise distances between images. Although cycle consistency and distance preservation successfully constrain the solution space, they overlook the special properties that simple geometric transformations do not change the semantic structure of images. Based on this special property, we develop a geometry-consistent generative adversarial network (GcGAN), which enables one-sided unsupervised domain mapping. GcGAN takes the original image and its counterpart image transformed by a predefined geometric transformation as inputs and generates two images in the new domain coupled with the corresponding geometry-consistency constraint. The geometry-consistency constraint reduces the space of possible solutions while keep the correct solutions in the search space. Quantitative and qualitative comparisons with the baseline (GAN alone) and the state-of-the-art methods including CycleGAN and DistanceGAN demonstrate the effectiveness of our method.

Motivation & Objective

• To address the ill-posed nature of unsupervised domain mapping, where paired training data is unavailable, by introducing a geometric consistency constraint that stabilizes training and improves translation quality.
• To overcome limitations of cycle consistency and distance preservation by leveraging the invariance of semantic structure under simple geometric transformations such as rotation.
• To enable one-sided domain mapping without requiring simultaneous training of both forward and inverse generators, while maintaining high-quality, semantically consistent translations.
• To reduce semantic distortions and mode collapse in GAN-based domain translation by co-regularizing the generator on transformed image pairs.
• To demonstrate compatibility and superiority of the geometry-consistency constraint over existing methods like CycleGAN and DistanceGAN in diverse image translation tasks.

Proposed method

• The method introduces a geometry-consistency constraint: for a predefined geometric transformation $ f(\cdot) $, the translated image $ G_{XY}(x) $ should satisfy $ f(G_{XY}(x)) \approx G_{\tilde{X}\tilde{Y}}(f(x)) $, where $ \tilde{X}, \tilde{Y} $ are the transformed domains.
• The generator is trained on both original images $ x \in \mathcal{X} $ and their transformed counterparts $ f(x) $, with the constraint that the transformation applied to the output should match the output of the generator on the transformed input.
• The constraint is enforced via a cycle-like loss: $ \| f(G_{XY}(x)) - G_{\tilde{X}\tilde{Y}}(f(x)) \|_1 \approx 0 $, ensuring that geometric structure is preserved across domains.
• The model uses a standard GAN framework with a discriminator, but the generator is trained with an additional geometry-consistency loss that couples the original and transformed image translations.
• The approach is compatible with existing constraints like cycle consistency and distance preservation, and can be combined with them to further improve performance.
• The method is trained in an unsupervised manner using only unpaired data, with no need for paired examples or explicit supervision.

Experimental results

Research questions

• RQ1 Can enforcing geometric consistency on image transformations improve one-sided unsupervised domain translation without requiring paired data?
• RQ2 Does the geometry-consistency constraint reduce mode collapse and semantic distortion in GAN-based domain translation models?
• RQ3 Can a one-sided domain mapping model achieve performance comparable or superior to two-sided models like CycleGAN by leveraging geometric invariance?
• RQ4 How does the geometry-consistency constraint interact with other existing constraints such as cycle consistency or distance preservation?
• RQ5 To what extent does the geometry-consistency constraint improve quantitative metrics like RMSE and pixel accuracy in real-world domain translation tasks?

Key findings

• GcGAN achieves a 21.9% improvement in pixel accuracy (with $ \delta = 5 $) over the baseline GAN alone on the Aerial photo → Map translation task, demonstrating superior detail preservation.
• On the Aerial photo → Map benchmark, GcGAN achieves RMSE of 27.98 with the GcGAN-Mix variant, outperforming CycleGAN (RMSE: 28.15) and GAN alone (RMSE: 33.27).
• In the Monet-to-Photo translation task, GcGAN produces more realistic and semantically consistent outputs than both GAN alone and CycleGAN, as shown in qualitative comparisons.
• The ablation study shows that GcGAN-rot + Cycle loss achieves RMSE of 28.21 and accuracy of 40.6% ($ \delta = 5 $), indicating that geometry consistency complements cycle consistency.
• The GcGAN-rot variant achieves 41.2% pixel accuracy ($ \delta = 5 $), outperforming GAN alone (19.3%) and matching CycleGAN (41.8%) on the same benchmark.
• Qualitative results show that GcGAN effectively mitigates mode collapse and produces more diverse, realistic translations across diverse domains including horse-to-zebra, day-to-night, and synthetic-to-real images.