[Paper Review] Real-Time Posture Monitoring and Risk Assessment for Manual Lifting Tasks Using MediaPipe and LSTM

Ereena Bagga, Ang Yang|arXiv (Cornell University)|Aug 23, 2024

Ergonomics and Musculoskeletal DisordersPsychology3 citations

TL;DR

This paper proposes a real-time posture monitoring and risk assessment system for manual lifting using MediaPipe for pose estimation and an LSTM-based model for sequential posture analysis. The system achieves 93.75% accuracy in classifying lifting postures and provides immediate feedback via a web interface, significantly outperforming existing methods in accuracy and real-time responsiveness.

ABSTRACT

This research focuses on developing a real-time posture monitoring and risk assessment system for manual lifting tasks using advanced AI and computer vision technologies. Musculoskeletal disorders (MSDs) are a significant concern for workers involved in manual lifting, and traditional methods for posture correction are often inadequate due to delayed feedback and lack of personalized assessment. Our proposed solution integrates AI-driven posture detection, detailed keypoint analysis, risk level determination, and real-time feedback delivered through a user-friendly web interface. The system aims to improve posture, reduce the risk of MSDs, and enhance user engagement. The research involves comprehensive data collection, model training, and iterative development to ensure high accuracy and user satisfaction. The solution's effectiveness is evaluated against existing methodologies, demonstrating significant improvements in real-time feedback and risk assessment. This study contributes to the field by offering a novel approach to posture correction that addresses existing gaps and provides practical, immediate benefits to users.

Motivation & Objective

• To address the limitations of delayed and non-personalized feedback in traditional posture correction methods for manual lifting tasks.
• To reduce the risk of work-related musculoskeletal disorders (WMSDs) by enabling real-time detection of incorrect lifting postures.
• To develop a system that integrates computer vision, sequence modeling, and real-time feedback for practical workplace use.
• To improve user engagement and adherence to safe lifting practices through an intuitive web-based interface.
• To evaluate the system’s performance against existing models, demonstrating superior accuracy and generalizability.

Proposed method

• Utilizes MediaPipe Pose to detect 33 keypoint landmarks in real-time from webcam video streams, providing 2D and 3D body posture data.
• Employs a custom Long Short-Term Memory (LSTM) network to model temporal sequences of posture keypoints for detecting risky lifting patterns.
• Processes video data from multiple angles and distances to enhance model robustness and generalization across diverse real-world conditions.
• Applies a risk analysis framework that categorizes postures into low, medium, or high risk based on biomechanical criteria derived from keypoint configurations.
• Trains the LSTM model on a diverse, iteratively collected dataset of videos depicting correct and incorrect lifting techniques.
• Deploys the system via a responsive web interface that delivers real-time visual and textual feedback to users during lifting tasks.

Experimental results

Research questions

• RQ1 Can an LSTM-based model effectively classify risky manual lifting postures from sequential keypoint data in real time?
• RQ2 How does the proposed system compare in accuracy and responsiveness to existing methods such as YOLOv7 with LSTM and LSTM Pose Machines?
• RQ3 To what extent does real-time feedback improve user posture correction and engagement in lifting tasks?
• RQ4 How does dataset diversity and iterative data collection impact model generalizability and performance?
• RQ5 Can a lightweight, web-based system using MediaPipe and LSTM provide practical, scalable solutions for WMSD prevention in occupational settings?

Key findings

• The proposed LSTM-based model achieved a categorical accuracy of 0.9565 and an overall accuracy score of 0.9375, outperforming YOLOv7 with LSTM (0.8615 and 0.8536) and LSTM Pose Machines (0.8348 and 0.8085).
• The system demonstrated superior generalizability and real-time responsiveness due to its streamlined architecture and focus on sequential posture dynamics.
• The integration of a risk analysis framework enabled dynamic categorization of postures into low, medium, or high-risk levels, enhancing practical usability.
• The use of diverse video data collected at varying angles and distances significantly improved model robustness and adaptability to real-world conditions.
• The web-based interface delivered low-latency feedback, increasing user engagement and promoting adherence to safe lifting techniques.
• Comparative evaluation confirmed that the proposed model offers a better balance of accuracy, efficiency, and real-time performance than existing solutions.