[Paper Review] Vectorization of Raster Manga by Deep Reinforcement Learning

Hao Su, Jianwei Niu+3 more|arXiv (Cornell University)|Oct 10, 2021

Video Analysis and Summarization6 citations

TL;DR

This paper proposes Mang2Vec, a deep reinforcement learning approach for vectorizing raster manga by modeling entire comics as sequences of stroke primitives. By training an agent to generate optimal stroke sequences with a structured reward and pruning mechanism, it achieves state-of-the-art results in vectorization quality and storage efficiency.

ABSTRACT

Manga is a popular Japanese-style comic form that consists of black-and-white lines. Compared with images of real-world scenarios, the simpler textures and fewer color gradients of mangas are the extra natures that can be vectorized. In this paper, we propose Mang2Vec, the first approach for vectorizing raster mangas using Deep Reinforcement Learning (DRL). Unlike existing learning-based works of image vectorization, we present a new view that considers an entire manga as a collection of basic primitives stroke line, and the sequence of strokes lines can be deep decomposed for further vectorization. We train a designed DRL agent to produce the most suitable sequence of lines, which is constrained to follow the visual feature of the target manga. Next, the control parameters of strokes are collected to translated to vector format. To improve our performances on visual quality and storage size, we further propose an SA reward to generate accurate stokes, and a pruning mechanism to avoid producing error and redundant strokes. Quantitative and qualitative experiments demonstrate that our Mang2Vec can produce impressive results and reaches the state-of-the-art level.

Motivation & Objective

• To address the lack of learning-based vectorization methods tailored specifically for manga's unique visual characteristics, such as simplified textures and line-based rendering.
• To overcome limitations of existing image vectorization techniques that do not account for the sequential and structural nature of manga line work.
• To develop a method that produces visually accurate, compact vector outputs by modeling manga as a sequence of stroke primitives.
• To improve vectorization quality and reduce redundancy through a novel shaped-aware (SA) reward and a pruning mechanism.

Proposed method

• The method models a manga as a sequence of basic stroke primitives, treating vectorization as a sequential decision-making problem.
• A deep reinforcement learning agent is trained to generate the optimal sequence of strokes that best match the visual features of the input raster manga.
• The agent uses a shaped-aware (SA) reward function to encourage accurate stroke generation that aligns with the target manga’s visual structure.
• A pruning mechanism is applied during inference to eliminate redundant or erroneous strokes, improving output compactness and quality.
• Control parameters of the generated strokes are collected and translated into standard vector format (e.g., SVG-like output).
• The approach leverages visual feature constraints to ensure the generated strokes conform to the original manga’s line work and topology.

Experimental results

Research questions

• RQ1 Can a deep reinforcement learning agent effectively learn to decompose a raster manga into an optimal sequence of vector strokes?
• RQ2 How does a shaped-aware reward function improve the accuracy and visual fidelity of the generated vector strokes compared to standard rewards?
• RQ3 To what extent can a pruning mechanism reduce redundancy and improve storage efficiency without degrading visual quality?
• RQ4 Does modeling manga as a sequence of strokes outperform conventional image vectorization methods in terms of visual quality and compactness?

Key findings

• Mang2Vec achieves state-of-the-art performance in vectorizing raster manga, outperforming existing learning-based methods in visual quality.
• The shaped-aware (SA) reward significantly improves stroke accuracy, resulting in vector outputs that closely match the original manga’s line structure.
• The pruning mechanism effectively reduces the number of redundant strokes, leading to more compact vector representations with minimal visual degradation.
• Quantitative experiments confirm that Mang2Vec produces vector outputs with superior storage efficiency and visual fidelity compared to baseline methods.
• Qualitative results demonstrate that the method preserves fine details and structural coherence of manga, even in complex line work.