[Paper Review] Single Image Human Proxemics Estimation for Visual Social Distancing

Maya Aghaei, Matteo Bustreo+4 more|arXiv (Cornell University)|Nov 3, 2020

Video Surveillance and Tracking Methods62 references4 citations

TL;DR

This paper proposes a semi-automatic method for estimating social distancing from single uncalibrated images by estimating a homography between the ground plane and image plane using intuitive auxiliary ratios, then leveraging body joint detections (especially torso length) to infer metric inter-personal distances. The approach outperforms baselines on adapted public datasets and enables real-time violation detection in surveillance settings, with key contributions including proxemics estimation, robust metric reference selection, and deployment in real-world environments.

ABSTRACT

In this work, we address the problem of estimating the so-called "Social Distancing" given a single uncalibrated image in unconstrained scenarios. Our approach proposes a semi-automatic solution to approximate the homography matrix between the scene ground and image plane. With the estimated homography, we then leverage an off-the-shelf pose detector to detect body poses on the image and to reason upon their inter-personal distances using the length of their body-parts. Inter-personal distances are further locally inspected to detect possible violations of the social distancing rules. We validate our proposed method quantitatively and qualitatively against baselines on public domain datasets for which we provided groundtruth on inter-personal distances. Besides, we demonstrate the application of our method deployed in a real testing scenario where statistics on the inter-personal distances are currently used to improve the safety in a critical environment.

Motivation & Objective

• To address the challenge of estimating metric inter-personal distances in unconstrained, uncalibrated surveillance scenes for social distancing compliance.
• To develop a practical, semi-automatic solution for homography estimation between ground plane and image plane using intuitive user-defined ratios.
• To identify the most reliable human body parts (e.g., torso) as metric references for distance inference in uncalibrated settings.
• To validate the method quantitatively and qualitatively on adapted public datasets with ground-truth inter-personal distances.
• To demonstrate real-world deployment of the system for 24/7 monitoring and statistical analysis in critical environments like airports and offices.

Proposed method

• Estimates the homography matrix between the ground plane and image plane using two manually selected auxiliary perspective ratios, enabling metric inference without full camera calibration.
• Employs an off-the-shelf 2D pose estimator (e.g., OpenPose) to detect human body joints and extract body part lengths as metric references.
• Uses torso length as the most robust metric reference due to its structural stability and minimal deformation under normal movement.
• Computes inter-personal distances in the image using the estimated homography and body joint coordinates, then checks for violations of social distancing rules (e.g., ≤2m).
• Applies a local reference frame per person to infer pairwise distances and detect overlapping safe spaces (proxemics) indicating violations.
• Deploys the system in real-world surveillance scenarios with up to 18 cameras at 1 fps, using statistical analysis to guide space redesign and alerting.

Figure 1: An example of the output of our proposed pipeline. The pedestrians are modeled as skeletal figures and each disc on the ground plane represents the minimum allowed inter-personal distance. The violation of social distancing is shown in red.

Experimental results

Research questions

• RQ1 Can a semi-automatic homography estimation method using intuitive user-defined ratios achieve accurate metric inference in uncalibrated surveillance images?
• RQ2 Which human body part (torso, arm, leg) provides the most reliable metric reference for inter-personal distance estimation in uncalibrated settings?
• RQ3 How does the proposed method compare to automatic homography estimation (e.g., AutoRect) and baseline methods using bounding boxes in terms of accuracy on uncalibrated data?
• RQ4 To what extent can the system detect social distancing violations in real-world, unconstrained environments with real-time deployment?
• RQ5 What are the key challenges and limitations in deploying such a system in real-world surveillance settings, particularly regarding temporal consistency and family group detection?

Key findings

• The proposed semi-automatic homography estimation outperforms automatic methods like AutoRect in inter-personal distance estimation, with user-selected ratios showing strong alignment with ground-truth homography.
• Using torso length as a metric reference yields the most consistent and accurate distance estimates, outperforming arm and leg lengths due to reduced deformation and occlusion.
• Bounding box-based metric references perform poorly, highlighting the necessity of using detailed body joint information for reliable metric inference.
• The method achieves favorable performance compared to state-of-the-art baselines, including Monoloco, especially in sequences where Monoloco’s performance is biased by its training data.
• The system has been successfully deployed in real-world settings (e.g., airports, offices), processing over 100,000 observations per hour and enabling real-time alerts and statistical analysis for space redesign.
• Limitations include dependency on pose detector accuracy and reduced performance in scenes with multiple homographies or non-uniform camera projections.

Figure 2: A schematic representation of our proposed pipeline for estimation of safe space and detection of its violation, comprised of three main computational stages: body-joint detection, body height estimation for metric inference, and ground-plane geometry estimation.