[Paper Review] Detecting Small Signs from Large Images

Zibo Meng, Xiaochuan Fan+3 more|arXiv (Cornell University)|Jun 26, 2017

Advanced Neural Network Applications6 citations

TL;DR

This paper proposes a patch-based, scale-invariant object detection framework using a Small-Object-Sensitive CNN (SOS-CNN) derived from SSD with VGG-16 to detect small traffic signs in large images (e.g., 2048×2048). By dividing large images into fixed-size patches and applying detection across an image pyramid, the method reduces GPU memory usage and significantly improves detection accuracy and recall for small signs compared to state-of-the-art methods.

ABSTRACT

In the past decade, Convolutional Neural Networks (CNNs) have been demonstrated successful for object detections. However, the size of network input is limited by the amount of memory available on GPUs. Moreover, performance degrades when detecting small objects. To alleviate the memory usage and improve the performance of detecting small traffic signs, we proposed an approach for detecting small traffic signs from large images under real world conditions. In particular, large images are broken into small patches as input to a Small-Object-Sensitive-CNN (SOS-CNN) modified from a Single Shot Multibox Detector (SSD) framework with a VGG-16 network as the base network to produce patch-level object detection results. Scale invariance is achieved by applying the SOS-CNN on an image pyramid. Then, image-level object detection is obtained by projecting all the patch-level detection results to the image at the original scale. Experimental results on a real-world conditioned traffic sign dataset have demonstrated the effectiveness of the proposed method in terms of detection accuracy and recall, especially for those with small sizes.

Motivation & Objective

• Address the challenge of detecting small objects in large images due to GPU memory limitations in deep learning models.
• Improve detection performance for small traffic signs that are underrepresented and hard to detect in real-world, high-resolution images.
• Reduce memory consumption of CNNs by processing large images as smaller patches instead of full-resolution inputs.
• Achieve scale invariance by applying the detector across multiple image resolutions using an image pyramid.
• Develop a framework that maintains high detection accuracy for small signs while being computationally feasible on standard GPU hardware.

Proposed method

• Divide large input images (e.g., 2048×2048) into fixed-size patches (e.g., 200×200) to reduce GPU memory usage.
• Use a Small-Object-Sensitive CNN (SOS-CNN) based on a truncated SSD framework with only the first four VGG-16 convolutional stages retained.
• Apply the SOS-CNN to each patch to generate patch-level object detection results with confidence scores and bounding box coordinates.
• Construct an image pyramid by down-sampling the original image to multiple resolutions (e.g., 1×, 0.5×, 0.25×, 0.0625×) to ensure scale invariance.
• Project all patch-level detection results back to the original image scale and apply non-maximum suppression (NMS) to generate final image-level detections.
• Use default boxes on feature maps to predict object locations and classes, with 3×3 convolutional layers for detection head output.

Experimental results

Research questions

• RQ1 Can patch-based processing reduce GPU memory usage while maintaining high detection accuracy for small objects in large images?
• RQ2 How does the proposed SOS-CNN framework perform on small traffic signs compared to existing methods like Fast R-CNN and Zhu et al.'s approach?
• RQ3 To what extent does using an image pyramid improve detection performance across different object scales?
• RQ4 Does the method achieve scale invariance in real-world conditions where signs are sparse and occupy small image regions?
• RQ5 Can the framework detect small signs more effectively than standard SSD or VGG-16-based detectors when input resolution exceeds GPU memory limits?

Key findings

• The proposed method outperforms Fast R-CNN and Zhu et al.'s method in terms of both accuracy and recall across all sign size categories, with the most significant improvement observed for small signs (size ≤32 pixels).
• On high-resolution images (2048×2048), detection performance for small signs (≤32 pixels) is superior, as evidenced by the highest accuracy and recall (blue curve in Fig. 6a).
• On low-resolution images (≤512×512), detection performance for large signs improves, with the green curve in Fig. 6(c) showing superior recall and accuracy compared to higher-resolution inputs.
• For medium-sized signs (32–96 pixels), both high and medium resolution inputs yield strong performance, indicating that the method captures signs effectively across multiple scales.
• The combination of patch processing and image pyramids enables scale invariance, allowing the model to detect signs of varying sizes more robustly than baseline methods.
• The method achieves better detection performance for small signs than state-of-the-art approaches, especially in real-world conditions where signs are small and sparsely distributed.