[Paper Review] Improved Person Detection on Omnidirectional Images with Non-maxima Suppression

Roman Seidel, André Apitzsch+1 more|arXiv (Cornell University)|May 22, 2018

Advanced Neural Network Applications19 references18 citations

TL;DR

This paper proposes a person detection method for omnidirectional images using YOLOv2 adapted to fish-eye images via virtual perspective views. By back-projecting detections and applying soft non-maximum suppression with Gaussian smoothing, the method achieves 64.6% average precision on PIROPO and 77.6% on the Flat dataset, significantly improving over baseline YOLOv2 and standard NMS.

ABSTRACT

We propose a person detector on omnidirectional images, an accurate method to generate minimal enclosing rectangles of persons. The basic idea is to adapt the qualitative detection performance of a convolutional neural network based method, namely YOLOv2 to fish-eye images. The design of our approach picks up the idea of a state-of-the-art object detector and highly overlapping areas of images with their regions of interests. This overlap reduces the number of false negatives. Based on the raw bounding boxes of the detector we fine-tuned overlapping bounding boxes by three approaches: non-maximum suppression, soft non-maximum suppression and soft non-maximum suppression with Gaussian smoothing. The evaluation was done on the PIROPO database and an own annotated Flat dataset, supplemented with bounding boxes on omnidirectional images. We achieve an average precision of 64.4 % with YOLOv2 for the class person on PIROPO and 77.6 % on Flat. For this purpose we fine-tuned the soft non-maximum suppression with Gaussian smoothing.

Motivation & Objective

• To address the challenge of accurate person detection in omnidirectional (180°) fish-eye images using deep learning.
• To overcome the limitations of standard object detectors trained on perspective images when applied directly to distorted omnidirectional views.
• To reduce false negatives and improve bounding box quality through refined post-processing of overlapping detections.
• To evaluate the effectiveness of different non-maximum suppression variants on omnidirectional image detection.
• To create and utilize a new annotated dataset (Flat) and use the PIROPO database for benchmarking.

Proposed method

• Generate highly overlapping virtual perspective views from omnidirectional images by sampling azimuth and elevation angles with small step sizes.
• Apply YOLOv2 to each virtual perspective view to obtain raw detection bounding boxes.
• Back-project the detected bounding boxes from perspective to the original omnidirectional image coordinate system.
• Apply three post-processing methods: standard NMS, soft NMS, and soft NMS with Gaussian smoothing to refine overlapping detections.
• Use intersection over union (IoU) with a threshold of 0.5 for evaluation, following PASCAL VOC standards.
• Fine-tune the suppression methods based on confidence scores and IoU overlap to minimize redundant and oversized boxes.

Experimental results

Research questions

• RQ1 How does the performance of YOLOv2 for person detection degrade when applied directly to omnidirectional images without adaptation?
• RQ2 To what extent can virtual perspective views improve detection accuracy on omnidirectional images?
• RQ3 Which variant of non-maximum suppression—standard, soft, or soft with Gaussian smoothing—yields the best detection performance on omnidirectional data?
• RQ4 How do the detection results compare across different datasets, particularly PIROPO and the newly created Flat dataset?
• RQ5 What is the impact of lens distortion and non-preserved axis parallelism on bounding box quality in omnidirectional detection?

Key findings

• Soft non-maximum suppression with Gaussian smoothing achieved the highest average precision of 64.6% on the PIROPO dataset for the person class.
• On the newly created Flat dataset, the same method reached 77.6% average precision, outperforming all other NMS variants.
• The direct application of YOLOv2 to omnidirectional images (NMS Omni) yielded only 41.4% average precision, indicating significant performance degradation without adaptation.
• The PR curve for Soft-NMS with Gaussian smoothing showed superior performance across most recall levels, especially between 0.35 and 0.75 recall, maintaining high precision.
• Standard NMS and Soft-NMS produced nearly identical PR curves, with only minor differences at low precision values (<0.2).
• The method effectively reduces false negatives by leveraging overlapping virtual views, which increases the likelihood of detecting persons across multiple projections.