[Paper Review] Mask-Guided Attention Network for Occluded Pedestrian Detection

Yanwei Pang, Jin Xie+4 more|arXiv (Cornell University)|Oct 14, 2019

Advanced Neural Network ApplicationsComputer Science33 references36 citations

TL;DR

The paper introduces MGAN, a mask-guided attention module that highlights visible pedestrian regions to suppress occluded parts, integrated into Faster R-CNN, achieving state-of-the-art results on CityPersons and Caltech using coarse-level segmentation annotations.

ABSTRACT

Pedestrian detection relying on deep convolution neural networks has made significant progress. Though promising results have been achieved on standard pedestrians, the performance on heavily occluded pedestrians remains far from satisfactory. The main culprits are intra-class occlusions involving other pedestrians and inter-class occlusions caused by other objects, such as cars and bicycles. These result in a multitude of occlusion patterns. We propose an approach for occluded pedestrian detection with the following contributions. First, we introduce a novel mask-guided attention network that fits naturally into popular pedestrian detection pipelines. Our attention network emphasizes on visible pedestrian regions while suppressing the occluded ones by modulating full body features. Second, we empirically demonstrate that coarse-level segmentation annotations provide reasonable approximation to their dense pixel-wise counterparts. Experiments are performed on CityPersons and Caltech datasets. Our approach sets a new state-of-the-art on both datasets. Our approach obtains an absolute gain of 9.5% in log-average miss rate, compared to the best reported results on the heavily occluded (HO) pedestrian set of CityPersons test set. Further, on the HO pedestrian set of Caltech dataset, our method achieves an absolute gain of 5.0% in log-average miss rate, compared to the best reported results. Code and models are available at: https://github.com/Leotju/MGAN.

Motivation & Objective

• Motivate robust pedestrian detection under heavy occlusion where occluders degrade full-body features.
• Propose a lightweight Mask-Guided Attention (MGA) branch to emphasize visible regions and suppress occlusions within proposals.
• Enable end-to-end training by integrating MGA into a standard Faster R-CNN-based detector.
• Utilize coarse-level visible-region annotations as a practical supervision signal for the MGA branch.

Proposed method

• Introduce a two-branch architecture: a Standard Pedestrian Detector (SPD) branch and a Mask-Guided Attention (MGA) branch.
• MGA generates a pixel-wise spatial attention map from RoI Align features to modulate full-body features via channel-wise reweighting.
• The MGA mask is produced from a small CNN that outputs a per-pixel probability map which multiplies RoI features channel-wise.
• Train with a combined loss L = L0 + alpha Lmask + beta Locc to jointly optimize detection and occlusion-aware supervision.
• Lmask uses coarse-level (weak) pixel-wise supervision via visible-region bounding boxes with binary cross-entropy.
• Locc weights the RCNN classification loss by occlusion level to emphasize hard samples.

Experimental results

Research questions

• RQ1 Can a mask-guided spatial attention branch improve detection of occluded pedestrians within a standard detector?
• RQ2 Does coarse-level visible-region annotation suffice to supervise pixel-wise attention masks without dense pixel-wise labeling?
• RQ3 What is the impact of introducing an occlusion-sensitive loss term on detection under varying occlusion levels?

Key findings

• MGAN improves over a Faster R-CNN baseline, reducing log-average miss rate on CityPersons heavy-occlusion set from 57.0 to 51.7 (absolute 5.3% gain).
• Using MGA alone reduces the HO miss rate to 52.7, and combining Lmask with Locc yields 51.7 on HO and 11.5 on the R set.
• Coarse-level segmentation annotations perform comparably to dense pixel-wise labeling for MGA supervision, offering a cost-effective alternative.
• MGAN outperforms several state-of-the-art occlusion-focused methods on CityPersons validation and Caltech datasets under various occlusion settings.
• On CityPersons test set, MGAN achieves state-of-the-art results with R=9.29 and HO=40.97 (lower MR is better).
• MGAN demonstrates strong performance across small, medium, and large pedestrians under heavy occlusion.