[Paper Review] City-GAN: Learning architectural styles using a custom Conditional GAN architecture

Maximilian Bachl, Daniel C. Ferreira|arXiv (Cornell University)|Jul 3, 2019

Generative Adversarial Networks and Image Synthesis3 references13 citations

TL;DR

This paper proposes City-GAN, a custom Conditional GAN architecture that learns and generates realistic architectural facades specific to cities like Vienna, Paris, Amsterdam, and Manhattan. By incorporating city-specific labels and a modified generator-discriminator structure, the model produces higher-quality, stylistically accurate images than standard GANs and enables smooth style interpolation between cities.

ABSTRACT

Generative Adversarial Networks (GANs) are a well-known technique that is trained on samples (e.g. pictures of fruits) and which after training is able to generate realistic new samples. Conditional GANs (CGANs) additionally provide label information for subclasses (e.g. apple, orange, pear) which enables the GAN to learn more easily and increase the quality of its output samples. We use GANs to learn architectural features of major cities and to generate images of buildings which do not exist. We show that currently available GAN and CGAN architectures are unsuited for this task and propose a custom architecture and demonstrate that our architecture has superior performance for this task and verify its capabilities with extensive experiments.

Motivation & Objective

• To develop a GAN architecture capable of learning and generating realistic architectural styles unique to specific cities.
• To address the limitations of standard GANs and CGANs in modeling complex urban facade data with diverse architectural features.
• To enable controlled generation and style interpolation between cities using a continuous label space.
• To create and release curated datasets of city facades for reproducible research.
• To provide open-source code and data to support community replication and extension of the method.

Proposed method

• A custom Conditional GAN architecture was designed with a modified generator and discriminator to better model architectural style variations across cities.
• City labels were embedded as one-hot vectors and concatenated with noise vectors in the generator, while the discriminator used both image features and label information for real/fake classification.
• Data augmentation techniques included horizontal flipping (50% probability) and random cropping to 85% of original size, with input resolutions of 64×64, 128×128, and 256×256 pixels.
• The model was trained on a curated dataset of 4,000 facade images (1,000 per city) and a larger uncurated dataset of 500,000 facades from five cities.
• Style interpolation was achieved by linearly interpolating label vectors (e.g., Amsterdam to Manhattan), enabling smooth transitions between architectural styles.
• Feature subtraction was applied by using negative label weights (e.g., -1 for Manhattan) to remove specific stylistic elements from generated images.

Experimental results

Research questions

• RQ1 Can a custom GAN architecture generate more realistic and stylistically accurate facades than standard GANs and CGANs for urban architectural data?
• RQ2 How do architectural features such as color palette, window patterns, roof structures, and vegetation vary across cities, and can they be disentangled in a generative model?
• RQ3 To what extent can the model interpolate between cities to produce plausible hybrid architectural styles?
• RQ4 Does the use of a continuous label space enable meaningful and interpretable style transfer between cities?
• RQ5 Can the model be trained effectively on both curated and uncurated datasets, and how does data quality affect performance?

Key findings

• The proposed City-GAN architecture produced significantly higher-quality and more realistic facade images than standard GANs and CGANs on the same dataset.
• The model successfully captured and generated distinct architectural characteristics such as Amsterdam’s red tones, Florence’s beige facades, and Washington D.C.’s tree-laden streets.
• Interpolation between cities, such as Amsterdam to Manhattan, demonstrated coherent transitions in building height, window size, and roof structure.
• Feature subtraction experiments confirmed that Manhattan’s label suppresses trees and modern buildings, and removing it from Amsterdam-generated images eliminated these features.
• The model generalized to uncurated datasets, producing plausible results even when trained on 500,000 unfiltered facades, though quality was lower than with curated data.
• The model enabled meaningful style transfer, such as transforming a European church into an American skyscraper through label interpolation.