[Paper Review] Image-to-Image Translation: Methods and Applications

Yingxue Pang, Jianxin Lin+2 more|arXiv (Cornell University)|Jan 21, 2021

Generative Adversarial Networks and Image SynthesisComputer Science248 references19 citations

TL;DR

This paper provides a comprehensive survey of image-to-image translation (I2I) methods, covering generative models like VAEs and GANs, two-domain and multi-domain translation tasks, and diverse applications such as style transfer, image restoration, and medical imaging. It identifies key methodological advances and outlines open challenges in efficiency, resolution, and cross-modality generalization.

ABSTRACT

Image-to-image translation (I2I) aims to transfer images from a source domain to a target domain while preserving the content representations. I2I has drawn increasing attention and made tremendous progress in recent years because of its wide range of applications in many computer vision and image processing problems, such as image synthesis, segmentation, style transfer, restoration, and pose estimation. In this paper, we provide an overview of the I2I works developed in recent years. We will analyze the key techniques of the existing I2I works and clarify the main progress the community has made. Additionally, we will elaborate on the effect of I2I on the research and industry community and point out remaining challenges in related fields.

Motivation & Objective

• To provide a systematic overview of recent advancements in image-to-image translation (I2I) techniques.
• To categorize and analyze I2I methods into two-domain and multi-domain tasks, clarifying their technical distinctions and progress.
• To present a thorough taxonomy of I2I applications across computer vision, graphics, and medical imaging.
• To evaluate the impact of I2I on research and industry, highlighting real-world utility and limitations.
• To identify open challenges in model efficiency, resolution fidelity, and generalization to non-image modalities.

Proposed method

• The paper reviews two core generative models—Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs)—as foundational backbones for I2I.
• It categorizes I2I into two-domain tasks (e.g., photo-to-sketch, photo-to-cartoon) and multi-domain tasks (e.g., translating to multiple artistic styles).
• The survey analyzes supervised, unsupervised, semi-supervised, and few-shot I2I learning paradigms, emphasizing their training objectives and constraints.
• It evaluates I2I methods using standard metrics such as FID, LPIPS, and SSIM, which assess image quality, perceptual similarity, and structural fidelity.
• The paper organizes applications into functional categories: style transfer, image restoration, super-resolution, colorization, and domain adaptation.
• It discusses architectural innovations such as cycle consistency, identity loss, and adversarial training to preserve content while transferring style.

Experimental results

Research questions

• RQ1 What are the core generative models driving modern image-to-image translation, and how do they differ in learning translation mappings?
• RQ2 How do two-domain and multi-domain I2I tasks differ in methodology, and what are the key challenges in each?
• RQ3 What are the most impactful applications of I2I in computer vision and image processing, and how do they leverage the translation capability?
• RQ4 What are the major limitations in current I2I frameworks, particularly regarding resolution, efficiency, and generalization?
• RQ5 How can I2I methods be extended beyond images to other modalities such as text, speech, or 3D data?

Key findings

• I2I has achieved significant success in applications including photo-to-artistic style transfer, image super-resolution, and medical image enhancement.
• GAN-based models, especially those using cycle consistency and identity loss, have demonstrated strong performance in unsupervised I2I with high perceptual quality.
• Supervised I2I methods achieve superior results when paired training data is available, while unsupervised approaches rely on cycle consistency to learn mappings without paired examples.
• The paper identifies a persistent trade-off between model complexity, inference speed, and output fidelity, especially at high resolutions.
• I2I methods have been successfully applied in medical imaging for tasks such as dose calculation and surgical phantom generation, demonstrating clinical relevance.
• Future directions include developing lightweight models for real-time deployment and extending I2I to cross-modality tasks like text-to-image or speech-to-video translation.