[Paper Review] Computer-Vision-Enabled Worker Video Analysis for Motion Amount Quantification

Hari Iyer, Neel Macwan+2 more|arXiv (Cornell University)|May 22, 2024

Industrial Vision Systems and Defect Detection4 citations

TL;DR

This paper proposes a computer vision framework that quantifies worker upper and lower limb motion using posture estimation and Hotelling’s $T^2$ statistic to detect ergonomic risks in real time. It demonstrates a 35% higher correlation between motion amount and $T^2$ for micro tasks than macro tasks, validating its sensitivity to fine-grained motion differences.

ABSTRACT

The performance of physical workers is significantly influenced by the extent of their motions. However, monitoring and assessing these motions remains a challenge. Recent advancements have enabled in-situ video analysis for real-time observation of worker behaviors. This paper introduces a novel framework for tracking and quantifying upper and lower limb motions, issuing alerts when critical thresholds are reached. Using joint position data from posture estimation, the framework employs Hotelling's $T^2$ statistic to quantify and monitor motion amounts. A significant positive correlation was noted between motion warnings and the overall NASA Task Load Index (TLX) workload rating ( extit{r} = 0.218, extit{p} = 0.0024). A supervised Random Forest model trained on the collected motion data was benchmarked against multiple datasets including UCF Sports Action and UCF50, and was found to effectively generalize across environments, identifying ergonomic risk patterns with accuracies up to 94\%.

Motivation & Objective

• To address the challenge of objectively quantifying physical worker motion in industrial settings for safety and productivity.
• To develop a non-invasive, real-time method for monitoring motion amount using in-situ video data.
• To identify motion patterns associated with ergonomic risks, particularly in tasks involving repetitive or awkward postures.
• To evaluate the effectiveness of Hotelling’s $T^2$ statistic in capturing motion variability across different task types.
• To support proactive ergonomic interventions by detecting motion thresholds that signal potential strain or injury risk.

Proposed method

• Utilizes single-person pose estimation to extract 3D joint positions from in-situ video footage of workers performing tasks.
• Applies Hotelling’s $T^{2}$ statistic to multivariate joint position data to quantify overall motion amount across upper and lower limbs.
• Processes video data in real time to compute motion statistics and issue alerts when motion exceeds predefined critical thresholds.
• Uses a dual-task setup—macro tasks (lifting large boxes) and micro tasks (moving small wooden cubes)—to compare motion patterns.
• Analyzes mean positions, standard deviations, and ranges of joint movements in all three spatial coordinates (x, y, z) to assess ergonomic demands.
• Employs a pipeline integrating video capture, pose estimation, and statistical motion quantification for continuous monitoring.

Experimental results

Research questions

• RQ1 How accurately can computer vision-based posture estimation capture and quantify worker motion amounts in real time?
• RQ2 To what extent does Hotelling’s $T^{2}$ statistic effectively represent motion variability across different task types?
• RQ3 How do motion patterns differ between macro tasks (large-scale movements) and micro tasks (fine, repetitive motions)?
• RQ4 Can the framework detect critical motion thresholds that correlate with ergonomic risk or physical strain?
• RQ5 How does the correlation between joint motion amount and $T^{2}$ statistic vary between high-variability and low-variability tasks?

Key findings

• The correlation between joint motion amount and Hotelling’s $T^{2}$ statistic was approximately 35% higher for micro tasks compared to macro tasks, indicating greater sensitivity to fine-grained motion differences.
• The framework successfully quantified motion amounts in real time using only video data and posture estimation, enabling proactive risk detection.
• Tasks with higher variability and broader movement ranges (e.g., large unguided tasks) exhibited greater motion demands and higher ergonomic risk potential.
• Spatial characteristics such as mean positions, standard deviations, and movement ranges varied significantly between task types, highlighting distinct ergonomic profiles.
• The consistency of motion trajectories derived from landmark points validated the reliability of the CV-based motion quantification approach.
• The system demonstrated potential for use in virtual workspace simulation and preemptive ergonomic design by identifying high-risk movement patterns before physical implementation.