[Paper Review] DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation

Alexandre Carlier, Martin Danelljan+2 more|arXiv (Cornell University)|Jul 22, 2020

Human Motion and AnimationEngineering23 references30 citations

TL;DR

DeepSVG introduces a hierarchical Transformer-based VAE that non-autoregressively predicts SVG path commands, enabling accurate vector graphic reconstruction and smooth interpolations, with a new large SVG-Icons8 dataset.

ABSTRACT

Scalable Vector Graphics (SVG) are ubiquitous in modern 2D interfaces due to their ability to scale to different resolutions. However, despite the success of deep learning-based models applied to rasterized images, the problem of vector graphics representation learning and generation remains largely unexplored. In this work, we propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and interpolation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. The network directly predicts a set of shapes in a non-autoregressive fashion. We introduce the task of complex SVG icons generation by releasing a new large-scale dataset along with an open-source library for SVG manipulation. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool by performing interpolations and other latent space operations. Our code is available at https://github.com/alexandre01/deepsvg.

Motivation & Objective

• Motivate learning generative models for complex vector graphics rather than raster images.
• Disentangle high-level shapes from low-level SVG draw commands in a hierarchical architecture.
• Enable non-autoregressive, feed-forward prediction of SVG paths to improve reconstruction and interpolation.
• Introduce a large-scale SVG-Icons8 dataset and an open-source toolkit for SVG manipulation.
• Demonstrate applications in interpolation, animation, latent-space manipulation, and font-like glyph generation.

Proposed method

• Represent SVGs as sets of paths, each a sequence of draw commands with fixed-length argument lists.
• Embed SVG commands via command, coordinate, and index embeddings into a continuous space.
• Use a two-stage hierarchical Transformer-based VAE: path encoding (E1) and set encoding (E2) to obtain a latent z, then a two-stage decoder (D2 then D1) to predict path representations and decode them into commands.
• Adopt feed-forward, non-autoregressive prediction of commands and attributes, in contrast to autoregressive baselines.
• Train with a VAE objective plus a permutation-aware assignment strategy (ordered or Hungarian) to map predicted paths to ground-truth paths.
• Construct and utilize a new SVG-Icons8 dataset (100,000 icons in 56 categories) for training and evaluation; also evaluate on SVG-Fonts for glyph generation.

Experimental results

Research questions

• RQ1 Can a hierarchical Transformer-based architecture effectively learn and generate complex SVG vector graphics?
• RQ2 Does non-autoregressive (feed-forward) prediction improve reconstruction and interpolation quality for vector graphics over autoregressive baselines?
• RQ3 How can permutation invariance of SVG paths be leveraged during encoding and generation?
• RQ4 What is the impact of path-to-ground-truth assignment strategies (ordered vs Hungarian) on training and interpolation quality?
• RQ5 Can the learned latent space enable meaningful animations and latent-space manipulations of vector graphics?

Key findings

• The hierarchical DeepSVG model achieves superior interpolation and reconstruction quality compared with autoregressive baselines.
• Feed-forward, non-autoregressive prediction substantially improves reconstruction error and interpolation smoothness over autoregressive models.
• A Hungarian or ordered assignment for ground-truth path matching enables meaningful latent-space interpolations, with ordered assignment yielding the most stable results.
• Latent-space operations enable semantically meaningful vector edits and animations across icons, including global shape changes and path-level manipulations.
• On the SVG-Icons8 dataset, the model demonstrates strong reconstruction, interpolation, and manipulation capabilities; it also enables font-like glyph generation on the SVG-Fonts dataset.