[Paper Review] Pan-Tilt Camera and PIR Sensor Fusion Based Moving Object Detection for Mobile Security Robots

YongChol Sin, MyongSong Choe+1 more|arXiv (Cornell University)|Oct 26, 2015

Video Surveillance and Tracking Methods8 references3 citations

TL;DR

This paper proposes a real-time moving object detection system for mobile security robots by fusing data from a pan-tilt camera and a passive infrared (PIR) sensor. Using consecutive frame differencing and PIR-triggered camera activation, the method achieves efficient, low-computation detection of humans in complex indoor environments with dynamic backgrounds and varying lighting.

ABSTRACT

One of fundamental issues for security robots is to detect and track people in the surroundings. The main problems of this task are real-time constraints, a changing background, varying illumination conditions and a non-rigid shape of the person to be tracked. In this paper, we propose a solution for tracking with a pan-tilt camera and a passive infrared range (PIR) sensor to detect the moving object based on consecutive frame difference. The proposed method is excellent in real-time performance because it requires only a little memory and computation. Experiment results show that this method can detect the moving object such as human efficiently and accurately in non-stationary and complex indoor environment.

Motivation & Objective

• Address the challenge of real-time, robust moving object detection in mobile security robots operating in complex indoor environments.
• Overcome limitations of traditional vision-based methods, such as sensitivity to lighting changes and background variations.
• Reduce computational and memory load to enable deployment on resource-constrained mobile platforms.
• Leverage the complementary strengths of visual and thermal motion sensing to improve detection reliability.
• Enable accurate and efficient human detection in non-stationary, cluttered indoor scenes with minimal processing overhead.

Proposed method

• Utilizes consecutive frame differencing on video streams from a pan-tilt camera to detect motion changes between frames.
• Employs a passive infrared (PIR) sensor to detect thermal motion and trigger the camera only when motion is sensed.
• Applies a fusion strategy where PIR detection activates the camera for frame differencing, reducing unnecessary processing.
• Implements a lightweight algorithm that requires minimal memory and computation, suitable for embedded robotic systems.
• Uses pan-tilt mechanism to direct the camera toward detected motion regions, improving tracking efficiency.
• Applies thresholding and morphological operations to refine motion regions and reduce false positives.

Experimental results

Research questions

• RQ1 Can PIR sensor-triggered camera activation reduce computational load while maintaining effective motion detection?
• RQ2 How does the fusion of PIR and frame differencing improve detection accuracy in dynamic indoor environments?
• RQ3 To what extent does the proposed method maintain real-time performance under varying lighting and background conditions?
• RQ4 How effective is the system in detecting human-sized moving objects in complex, non-stationary indoor scenes?
• RQ5 Can the fusion approach reduce false positives compared to standalone vision or PIR-based detection?

Key findings

• The proposed method achieves real-time performance with low memory and computational requirements, suitable for embedded robotic systems.
• Fusion of PIR and frame differencing significantly improves detection accuracy in complex indoor environments with dynamic backgrounds.
• The system effectively detects human-sized moving objects even under varying illumination conditions.
• PIR-triggered camera activation reduces processing load by avoiding continuous full-frame analysis.
• The method demonstrates robustness in non-stationary environments, maintaining consistent detection performance.
• Experimental results confirm the system's ability to detect and localize moving humans efficiently and accurately.