[Paper Review] A Vision-based Social Distancing and Critical Density Detection System for COVID-19

Dongfang Yang, Ekim Yurtsever+3 more|arXiv (Cornell University)|Jul 7, 2020

COVID-19 epidemiological studies39 references40 citations

TL;DR

The paper presents a real-time, privacy-preserving vision system using monocular cameras to detect social distancing violations and regulate crowd inflow via a defined critical density, with open-source code available.

ABSTRACT

Social distancing has been proven as an effective measure against the spread of the infectious COronaVIrus Disease 2019 (COVID-19). However, individuals are not used to tracking the required 6-feet (2-meters) distance between themselves and their surroundings. An active surveillance system capable of detecting distances between individuals and warning them can slow down the spread of the deadly disease. Furthermore, measuring social density in a region of interest (ROI) and modulating inflow can decrease social distancing violation occurrence chance. On the other hand, recording data and labeling individuals who do not follow the measures will breach individuals' rights in free-societies. Here we propose an Artificial Intelligence (AI) based real-time social distancing detection and warning system considering four important ethical factors: (1) the system should never record/cache data, (2) the warnings should not target the individuals, (3) no human supervisor should be in the detection/warning loop, and (4) the code should be open-source and accessible to the public. Against this backdrop, we propose using a monocular camera and deep learning-based real-time object detectors to measure social distancing. If a violation is detected, a non-intrusive audio-visual warning signal is emitted without targeting the individual who breached the social distancing measure. Also, if the social density is over a critical value, the system sends a control signal to modulate inflow into the ROI. We tested the proposed method across real-world datasets to measure its generality and performance. The proposed method is ready for deployment, and our code is open-sourced.

Motivation & Objective

• Motivate and address privacy and non-intrusiveness concerns in social distancing monitoring.
• Develop a real-time AI-based system to detect inter-person distances without recording data.
• Introduce a novel critical social density metric to regulate inflow and prevent overcrowding.
• Provide an open-source solution validated on real-world crowd datasets.

Proposed method

• Detect pedestrians in image space with pre-trained deep CNN detectors.
• Map image coordinates to real-world BEV via inverse homography to obtain ground-plane positions.
• Compute inter-pedestrian distances and count violations against a threshold d_c.
• Define and estimate a critical social density rho_c using linear regression of violations v versus density rho, using the 95% prediction interval at v=0.
• Emit non-intrusive audio-visual warnings when breaches occur and modulate ROI inflow when rho > rho_c.

Experimental results

Research questions

• RQ1 Can a fixed monocular camera with AI detectors accurately measure inter-pedestrian distances in real time without data recording?
• RQ2 How can a critical social density be defined and estimated to keep social-distancing violations below a target probability?
• RQ3 Do pre-trained detectors generalize well across urban, indoor, and subway-like crowd scenarios for social distancing monitoring?

Key findings

• Faster R-CNN and YOLOv4 achieve real-time pedestrian detection on the tested datasets.
• A BEV mapping via inverse homography enables distance measurement between pedestrians in real world coordinates.
• Social density rho correlates positively with the number of violations v, enabling linear regression-based rho_c estimation.
• The calculated critical densities rho_c are similar across Oxford Town Center, Mall, and Train Station datasets.
• The system supports non-intrusive warnings and inflow modulation without recording data or targeting individuals.