[Paper Review] From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer

Haipeng Xiong, Hao Lü+4 more|arXiv (Cornell University)|Aug 15, 2019

Video Surveillance and Tracking Methods22 references32 citations

TL;DR

S-DCNet converts open-set counting into a closed-set problem by spatially dividing feature maps to sum local counts, achieving state-of-the-art results across multiple datasets.

ABSTRACT

Visual counting, a task that predicts the number of objects from an image/video, is an open-set problem by nature, i.e., the number of population can vary in $[0,+\infty)$ in theory. However, the collected images and labeled count values are limited in reality, which means only a small closed set is observed. Existing methods typically model this task in a regression manner, while they are likely to suffer from an unseen scene with counts out of the scope of the closed set. In fact, counting is decomposable. A dense region can always be divided until sub-region counts are within the previously observed closed set. Inspired by this idea, we propose a simple but effective approach, Spatial Divide-and- Conquer Network (S-DCNet). S-DCNet only learns from a closed set but can generalize well to open-set scenarios via S-DC. S-DCNet is also efficient. To avoid repeatedly computing sub-region convolutional features, S-DC is executed on the feature map instead of on the input image. S-DCNet achieves the state-of-the-art performance on three crowd counting datasets (ShanghaiTech, UCF_CC_50 and UCF-QNRF), a vehicle counting dataset (TRANCOS) and a plant counting dataset (MTC). Compared to the previous best methods, S-DCNet brings a 20.2% relative improvement on the ShanghaiTech Part B, 20.9% on the UCF-QNRF, 22.5% on the TRANCOS and 15.1% on the MTC. Code has been made available at: https://github. com/xhp-hust-2018-2011/S-DCNet.

Motivation & Objective

• Motivate counting as an open-set problem and address data scarcity by leveraging a closed set.
• Propose a spatial divide-and-conquer strategy to decompose counting into local, count-interval predictions.
• Develop a CNN architecture that performs S-DC on feature maps to avoid redundant computations.
• Demonstrate robustness and generalization of closed-set learning to open-set counting across diverse datasets.

Proposed method

• Use a VGG16-based encoder with a UNet-like decoder to generate multi-resolution feature maps.
• Discretize local counts into intervals and formulate counting as a classification problem over these intervals.
• Introduce a division decider to produce soft division masks and merge sub-region counts into a global image count.
• Apply single-stage and multi-stage S-DC to iteratively divide feature maps and refine local counts.
• Train with multi-task losses comprising cross-entropy losses for multiple classification outputs and an L1 loss for final division output.

Experimental results

Research questions

• RQ1 Can a counting model learned from a closed set generalize to open-set counting via spatial divide-and-conquer?
• RQ2 How many division stages are needed for accurate open-set counting without excessive computation?
• RQ3 Does reformulating counting as interval classification improve performance over direct regression?
• RQ4 What is the impact of Cmax and partition strategy on S-DCNet performance?
• RQ5 How does S-DCNet perform on diverse counting datasets (crowd, vehicles, plants) compared to state-of-the-art?

Key findings

• S-DCNet achieves state-of-the-art MAE/MSE on ShanghaiTech Part_A and Part_B, UCF-QNRF, TRANCOS, and MTC datasets.
• Two-stage S-DC suffices to realize strong performance, with incremental gains diminishing beyond two stages.
• Discretizing local counts into intervals and using a shared classifier improves robustness over direct regression or plain classification.
• The model trained on a closed set generalizes effectively to counts beyond the training maximum (open-set) due to S-DC.
• S-DCNet shows substantial relative improvements over prior methods across multiple datasets (e.g., ShanghaiTech Part_B, UCF-QNRF, TRANCOS, MTC).