[Paper Review] RAPiD: Rotation-Aware People Detection in Overhead Fisheye Images

Zhihao Duan, M. Ozan Tezcan+3 more|ArXiv.org|May 23, 2020

Video Surveillance and Tracking Methods32 references4 citations

TL;DR

RAPiD is an end-to-end, rotation-aware people detection method for overhead fisheye images that uses a fully convolutional neural network to regress arbitrarily-oriented bounding boxes with a novel periodic loss function for angle prediction. It outperforms state-of-the-art methods on three datasets, including a new challenging dataset (CEPDOF), with higher accuracy and no added computational cost over standard YOLO.

ABSTRACT

Recent methods for people detection in overhead, fisheye images either use radially-aligned bounding boxes to represent people, assuming people always appear along image radius or require significant pre-/post-processing which radically increases computational complexity. In this work, we develop an end-to-end rotation-aware people detection method, named RAPiD, that detects people using arbitrarily-oriented bounding boxes. Our fully-convolutional neural network directly regresses the angle of each bounding box using a periodic loss function, which accounts for angle periodicities. We have also created a new dataset with spatio-temporal annotations of rotated bounding boxes, for people detection as well as other vision tasks in overhead fisheye videos. We show that our simple, yet effective method outperforms state-of-the-art results on three fisheye-image datasets. Code and dataset are available at http://vip.bu.edu/rapid .

Motivation & Objective

• Address the poor performance of standard object detection models on overhead fisheye images due to radial distortion and non-upright human poses.
• Overcome limitations of existing methods that rely on radial alignment or complex pre/post-processing to handle rotated people.
• Develop an end-to-end deep learning model that predicts arbitrarily-oriented bounding boxes without assumptions on pose or orientation.
• Introduce a new, challenging dataset (CEPDOF) with diverse real-world scenarios such as occlusions, hats, and low-light conditions to better evaluate detection robustness.
• Enable high-accuracy, real-time people detection in smart buildings using fisheye cameras for applications like occupancy sensing and security.

Proposed method

• Extend YOLOv3 to predict rotated bounding boxes by regressing center, width, height, and angle of each person.
• Introduce a periodic loss function that accounts for the circular nature of angle space, using sine and cosine representations to avoid discontinuities at ±π.
• Use a bounded angle range of [−π, π) with a symmetry-aware head that forces predicted width < height to ensure unique box representation.
• Train the network end-to-end using a single forward pass per image, avoiding multi-step inference or image rotation as in prior methods.
• Fine-tune the model on fisheye-specific data to improve performance over standard-image pre-trained weights.
• Apply data augmentation and class-agnostic head for single-class person detection to improve inference speed and focus on person-specific features.

Experimental results

Research questions

• RQ1 Can a single-stage, end-to-end deep learning model effectively detect people in overhead fisheye images using arbitrarily-oriented bounding boxes?
• RQ2 Does a periodic loss function for angle regression outperform standard L1/L2 regression in terms of accuracy and robustness to angle discontinuities?
• RQ3 How does the proposed method perform on challenging real-world scenarios such as occlusions, unusual poses, and low-light conditions compared to existing methods?
• RQ4 To what extent does fine-tuning a standard-object-detection model on fisheye data improve performance without increasing inference complexity?
• RQ5 Can a new, diverse dataset with spatio-temporal annotations of rotated bounding boxes enable better evaluation and benchmarking of people detection in fisheye environments?

Key findings

• RAPiD achieves 88.9% AP50 on the HABBOF dataset using periodic L1 loss, outperforming the next best method by 1.9 percentage points.
• On the CEPDOF dataset, RAPiD achieves 97.6% AP50 at 1024×1024 resolution for normal-light videos, with F-measure above 0.96 in high-activity and lunch meeting scenarios.
• Performance drops significantly in low-light conditions (e.g., 52.8% AP50 in All-off video), but improves to 65.6% with IR illumination, indicating sensitivity to visibility.
• The network consistently predicts width < height (mean aspect ratio >1), validating the symmetry assumption and ensuring unique box representation.
• The periodic loss function reduces angle prediction error and enables accurate detection of non-radial and non-upright poses, as shown in qualitative comparisons.
• RAPiD maintains inference speed comparable to standard YOLO, with only a single forward pass per image, avoiding the 24-fold YOLO application required by prior methods.