[Paper Review] COVID-Robot: Monitoring Social Distancing Constraints in Crowded Scenarios

Adarsh Jagan Sathyamoorthy, Utsav Patel+3 more|arXiv (Cornell University)|Aug 14, 2020

Video Surveillance and Tracking Methods21 references55 citations

TL;DR

The paper presents a vision-guided mobile robot system that detects social distancing breaches, navigates to the largest non-compliant group, and encourages distancing using alerts, with RGB-D, CCTV, and thermal sensors and a CCTV-robot fusion strategy.

ABSTRACT

Maintaining social distancing norms between humans has become an indispensable precaution to slow down the transmission of COVID-19. We present a novel method to automatically detect pairs of humans in a crowded scenario who are not adhering to the social distance constraint, i.e. about 6 feet of space between them. Our approach makes no assumption about the crowd density or pedestrian walking directions. We use a mobile robot with commodity sensors, namely an RGB-D camera and a 2-D lidar to perform collision-free navigation in a crowd and estimate the distance between all detected individuals in the camera's field of view. In addition, we also equip the robot with a thermal camera that wirelessly transmits thermal images to a security/healthcare personnel who monitors if any individual exhibits a higher than normal temperature. In indoor scenarios, our mobile robot can also be combined with static mounted CCTV cameras to further improve the performance in terms of number of social distancing breaches detected, accurately pursuing walking pedestrians etc. We highlight the performance benefits of our approach in different static and dynamic indoor scenarios.

Motivation & Objective

• Develop a mobile robot system that detects social distancing breaches (≤6 feet for ≥5 seconds) without requiring wearables.
• Enable autonomous, collision-free navigation toward non-compliant groups in various indoor crowds.
• Enhance breach detection with a hybrid setup combining a mobile robot and static CCTV cameras.
• Provide a privacy-conscious thermal imaging capability to assist health/security personnel.

Proposed method

• Pedestrian detection and tracking via Yolov3 on RGB images with stable IDs for tracking.
• Depth-enabled distance estimation from RGB-D data to compute inter-person distances.
• Homography-based ground-plane transformation to estimate distances from CCTV footage when using static cameras.
• Group classification of non-compliant pedestrians to target the largest cluster (Algorithm 1).
• A DRL-based collision avoidance policy, refined with Frozone to avoid the Freezing Robot Problem, guiding the robot to a locked pedestrian.
• Computation of goal positions relative to robot (RGB-D) or map coordinates (CCTV) to drive DRL policy.
• Thermal camera integration with wireless transmission to security/healthcare personnel while preserving privacy.

Experimental results

Research questions

• RQ1 How accurately can the system detect social distancing breaches in indoor environments without wearable devices?
• RQ2 Can a mobile robot, possibly fused with CCTV data, reliably navigate to and address the largest non-compliant group in crowds?
• RQ3 What is the impact of incorporating CCTV data and homography-based distance estimation on breach detection and enforcement performance?
• RQ4 Does the thermal imaging pathway improve monitoring while preserving privacy?

Key findings

• The system detects social distancing breaches (≤6 feet for at least 5 seconds) using RGB-D data and optionally CCTV data.
• Distance estimation between pedestrians achieves an average error of about 0.3 feet in indoor environments when using RGB-D data.
• Hybrid CCTV-robot operation can improve breach detection and enforcement by up to 100% in indoor scenarios compared to robot-only or CCTV-only setups.
• The robot autonomously prioritizes the largest non-compliant group, navigates to a locked pedestrian, and displays alerts to encourage distancing.
• A privacy-preserving thermal camera capability transmits temperature signatures to security/healthcare personnel without facial recognition or recording of temperatures themselves.