[论文解读] Dual Graph Convolutional Network for Semantic Segmentation

Li Zhang, Xiangtai Li+4 more|arXiv (Cornell University)|Sep 13, 2019

Advanced Neural Network Applications参考文献 58被引用 114

一句话总结

本文提出一种用于语义分割的 Dual Graph Convolutional Network，并在 Cityscapes 上实现了 state-of-the-art mean IoU，在 Pascal Context 上具有竞争力的结果，超越了若干基线。

ABSTRACT

Exploiting long-range contextual information is key for pixel-wise prediction tasks such as semantic segmentation. In contrast to previous work that uses multi-scale feature fusion or dilated convolutions, we propose a novel graph-convolutional network (GCN) to address this problem. Our Dual Graph Convolutional Network (DGCNet) models the global context of the input feature by modelling two orthogonal graphs in a single framework. The first component models spatial relationships between pixels in the image, whilst the second models interdependencies along the channel dimensions of the network's feature map. This is done efficiently by projecting the feature into a new, lower-dimensional space where all pairwise interactions can be modelled, before reprojecting into the original space. Our simple method provides substantial benefits over a strong baseline and achieves state-of-the-art results on both Cityscapes (82.0% mean IoU) and Pascal Context (53.7% mean IoU) datasets. Code and models are made available to foster any further research (\url{https://github.com/lxtGH/GALD-DGCNet}).

研究动机与目标

• 通过图基推理提升语义分割的动机。
• 介绍并评估用于分割任务的双图卷积网络架构。
• 在 Cityscapes 和 Pascal Context 上对所提方法与现有基线进行基准对比。

提出的方法

• 提出一种用于语义分割的双图卷积网络架构。
• 在标准数据集上评估该方法并与多种基线进行比较。
• 提供逐类和 Mean IoU 结果以展示改进。

实验结果

研究问题

• RQ1 Dual Graph Convolutional Network 是否在 Cityscapes 和 Pascal Context 上相比现有基线提升了分割精度？
• RQ2 所提出的方法在各个语义类别和总体 Mean IoU 上的表现如何？
• RQ3 与前一方法相比，该方法是否更具一致性且产生更少的伪影？
• RQ4 该方法相对于如 DeepLab-v2、RefineNet 与 DANet 等当代架构的排名如何？

主要发现

方法	Mean IoU	road	sidebar	building	wall	fence	pole	traffic light	traffic sign	vegetation	terrain	sky	person	rider	car	truck	bus	train	motorcycle	bicycle
DeepLab-v2	70.4	97.9	81.3	90.3	48.8	47.4	49.6	57.9	67.3	91.9	69.4	94.2	79.8	59.8	93.7	56.5	67.5	57.5	57.7	68.8
RefineNet	73.6	98.2	83.3	91.3	47.8	50.4	56.1	66.9	71.3	92.3	70.3	94.8	80.9	63.3	94.5	64.6	76.1	64.3	62.2	70.0
GCN	76.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DUC	77.6	98.5	85.5	92.8	58.6	55.5	65	73.5	77.9	93.3	72	95.2	84.8	68.5	95.4	70.9	78.8	68.7	65.9	73.8
ResNet-38	78.4	98.5	85.7	93.1	55.5	59.1	67.1	74.8	78.7	93.7	72.6	95.5	86.6	69.2	95.7	64.5	78.8	74.1	69	76.7
PSPNet	78.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
BiSeNet	78.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PSANet	80.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DenseASPP	80.6	98.7	87.1	93.4	60.7	62.7	65.6	74.6	78.5	93.6	72.5	95.4	86.2	71.9	96.0	78.0	90.3	80.7	69.7	76.8
GloRe	80.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DANet	81.5	98.6	86.1	93.5	56.1	63.3	69.7	77.3	81.3	93.9	72.9	95.7	87.3	72.9	96.2	76.8	89.4	86.5	72.2	78.2
Ours	82.0	98.7	87.4	93.9	62.4	63.4	70.8	78.7	81.3	94.0	73.3	95.8	87.8	73.7	96.4	76.0	91.6	81.6	71.5	78.2

• 在 Cityscapes 测试集上实现 82.0% Mean IoU，在 19 个类别中有 16 个类别的 IoU 最高。
• 在所列表中，在 Mean IoU 和逐类准确率方面超过了若干基线（如 DeepLab-v2、RefineNet、DANet）。
• 在 Cityscapes 的所列类别上展示了具竞争力或更优的逐类性能。