[Paper Review] Towards the Automatic Anime Characters Creation with Generative Adversarial Networks

Yanghua Jin, Jiakai Zhang+4 more|arXiv (Cornell University)|Aug 18, 2017

Human Motion and AnimationEngineering22 references135 citations

TL;DR

The paper builds a DRAGAN-based GAN trained on a clean anime-face dataset to automatically generate high-quality anime characters, and deploys a public web interface for interactive character design.

ABSTRACT

Automatic generation of facial images has been well studied after the Generative Adversarial Network (GAN) came out. There exists some attempts applying the GAN model to the problem of generating facial images of anime characters, but none of the existing work gives a promising result. In this work, we explore the training of GAN models specialized on an anime facial image dataset. We address the issue from both the data and the model aspect, by collecting a more clean, well-suited dataset and leverage proper, empirical application of DRAGAN. With quantitative analysis and case studies we demonstrate that our efforts lead to a stable and high-quality model. Moreover, to assist people with anime character design, we build a website (http://make.girls.moe) with our pre-trained model available online, which makes the model easily accessible to general public.

Motivation & Objective

• Create a high-quality, anime-face-generating model using a clean, curated dataset.
• Stabilize GAN training for anime-style images with DRAGAN and a SRResNet-like generator.
• Enable conditional, attribute-driven image generation with an auxiliary classifier.
• Quantitatively evaluate generation quality using FID adapted to anime illustrations.
• Provide accessible online tooling to democratize anime character design.

Proposed method

• Assemble a clean anime-face dataset from Getchu with bounding boxes expanded 1.5x and manually filtered for quality.
• Estimate 34 attribute tags per image using a pre-trained Illustration2Vec model to guide conditioning.
• Train a DRAGAN-based GAN with a SRResNet-like generator and a 10-block discriminator.
• Incorporate conditional generation with a 34-dimension attribute vector and an auxiliary classifier in the discriminator.
• Use gradient penalty and measured losses to stabilize training and enable conditional image synthesis.
• Evaluate using an Illustration2vec-based FID metric and qualitative/precision analyses of attribute control.

Experimental results

Research questions

• RQ1 Can a clean, domain-specific anime-face dataset enable stable, high-quality GAN generation of anime characters?
• RQ2 Does conditioning on estimated attributes enable controllable, realistic anime character generation?
• RQ3 How does the DRAGAN-based approach compare to baseline GANs in this domain under an anime-specific evaluation metric?
• RQ4 What is the practical performance of an online interface that generates anime faces from user-specified attributes?

Key findings

• The DRAGAN-based model trained on a clean Getchu-derived dataset yields stable, higher-quality anime faces.
• Conditional generation with 34 attributes yields controllable outputs with varying precision per attribute.
• The model achieves a lower FID (4607.56) than the baseline (5974.96) when evaluated with Illustration2vec features.
• Color attributes are learned more reliably than complex shapes like hats or glasses, which remain challenging.
• A public website (make.girls.moe) enables client-side generation with 6–7 seconds per image, using a smaller SRResNet-based generator for faster downloads.