[论文解读] Object Detection for Comics using Manga109 Annotations

Toru Ogawa, Atsushi Otsubo+4 more|arXiv (Cornell University)|Mar 23, 2018

Advanced Image and Video Retrieval Techniques参考文献 20被引用 42

一句话总结

本论文提出 Manga109-annotations，这是一个大规模、手工标注的漫画数据集，以及 SSD300-fork，这是一个针对高度重叠漫画对象的锚点强制检测器，在 Manga109-annotations 上达到最先进的 mAP。

ABSTRACT

With the growth of digitized comics, image understanding techniques are becoming important. In this paper, we focus on object detection, which is a fundamental task of image understanding. Although convolutional neural networks (CNN)-based methods archived good performance in object detection for naturalistic images, there are two problems in applying these methods to the comic object detection task. First, there is no large-scale annotated comics dataset. The CNN-based methods require large-scale annotations for training. Secondly, the objects in comics are highly overlapped compared to naturalistic images. This overlap causes the assignment problem in the existing CNN-based methods. To solve these problems, we proposed a new annotation dataset and a new CNN model. We annotated an existing image dataset of comics and created the largest annotation dataset, named Manga109-annotations. For the assignment problem, we proposed a new CNN-based detector, SSD300-fork. We compared SSD300-fork with other detection methods using Manga109-annotations and confirmed that our model outperformed them based on the mAP score.

研究动机与目标

• Motivate object detection in comics and address the lack of large-scale annotated datasets.
• Create Manga109-annotations with bounding boxes for frames, text, faces, and bodies plus extra annotations (character names, text contents).
• Develop an object detector tailored to overlapped comic objects to improve training and inference performance.

提出的方法

• Annotate Manga109 to produce Manga109-annotations with bounding boxes and category labels (frame, text, face, body).
• Propose SSD300-fork, a forked variant of SSD300 with replicated category-specific detection layers to address heavy object overlap.
• Use a weighted category-wise loss to balance detection across four categories.
• Train with VGG-16 backbone and standard SSD data augmentation; apply hard negative mining.
• Evaluate against Faster R-CNN, SSD300, and YOLOv2 on Manga109-annotations; compare with eBDtheque for cross-dataset analysis.

实验结果

研究问题

• RQ1 Can a large-scale, manually annotated manga dataset improve object detection performance on comic pages?
• RQ2 Does replicating detection layers per category (SSD300-fork) mitigate the assignment/labeling problems caused by highly overlapped comic objects?
• RQ3 How does SSD300-fork compare to existing CNN-based detectors on manga data in terms of mAP across frames, text, faces, and bodies?
• RQ4 How well does a model trained on Manga109-annotations transfer to another dataset (eBDtheque) with different drawing styles?

主要发现

方法	mAP	帧	文本	人脸	身体
Faster R-CNN	49.9	96.1	23.8	15.7	63.9
SSD300	81.3	97.1	82.0	67.1	79.1
YOLOv2	59.7	90.2	64.6	37.1	46.9
SSD300-fork	84.2	96.9	84.1	76.2	79.6

• Manga109-annotations provides 527,685 bounding-box annotations over 10,130 pages with four object categories and additional text/character data.
• SSD300-fork outperforms the baseline SSD300 and other detectors on Manga109-annotations with an overall mAP of 84.2%, and shows notable gains in the face category (76.2% vs 67.1% for SSD300).
• SSD300-fork achieves higher mAP (84.2%) than Faster R-CNN (49.9%), YOLOv2 (59.7%), and SSD300 (81.3%) on the Manga109-annotations benchmark.
• On eBDtheque, SSD300-fork attains competitive frame detection (73.3% recall, 76.4% precision, 74.8% F) and substantially better body detection (42.2% recall, 58.0% precision, 48.8% F) than prior methods.
• The forked architecture enables handling of overlapping objects by assigning each category to its own anchor set, while keeping parameter count and runtime close to SSD300.