[Paper Review] Wearable Vision Detection of Environmental Fall Risks using Convolutional Neural Networks

Mina Nouredanesh, A.C. McCormick+2 more|arXiv (Cornell University)|Nov 2, 2016

Video Surveillance and Tracking Methods1 references3 citations

TL;DR

This paper proposes a wearable first-person vision system using a chest-mounted GoPro and a convolutional neural network (CNN) to detect environmental fall risks in seniors. The system classifies 12 hazard classes—such as stairs, curbs, and uneven surfaces—achieving a mean square error of 8%, demonstrating promising real-time detection for fall prevention in daily living.

ABSTRACT

In this paper, a method to detect environmental hazards related to a fall risk using a mobile vision system is proposed. First-person perspective videos are proposed to provide objective evidence on cause and circumstances of perturbed balance during activities of daily living, targeted to seniors. A classification problem was defined with 12 total classes of potential fall risks, including slope changes (e.g., stairs, curbs, ramps) and surfaces (e.g., gravel, grass, concrete). Data was collected using a chest-mounted GoPro camera. We developed a convolutional neural network for automatic feature extraction, reduction, and classification of frames. Initial results, with a mean square error of 8%, are promising.

Motivation & Objective

• To develop a real-time, wearable vision system that identifies environmental fall risks in older adults during daily activities.
• To use first-person perspective video from a chest-mounted GoPro to capture objective data on balance perturbations.
• To classify 12 distinct environmental hazard classes, including slopes (stairs, ramps) and surfaces (gravel, grass, concrete).
• To leverage deep learning for automatic feature extraction and classification without manual feature engineering.
• To evaluate the system's performance in detecting fall-related environmental conditions with high accuracy.

Proposed method

• A chest-mounted GoPro camera captures first-person video during daily activities to record environmental conditions linked to fall risk.
• A custom convolutional neural network (CNN) is trained to automatically extract, reduce, and classify visual features from individual video frames.
• The CNN is trained on a dataset of 12 predefined hazard classes, including slope changes and surface types.
• A classification loss function minimizes mean square error (MSE) during training to optimize prediction accuracy.
• The system processes frames in real time, enabling immediate hazard detection during mobility.
• Data augmentation and transfer learning techniques are applied to improve generalization with limited real-world data.

Experimental results

Research questions

• RQ1 Can a wearable first-person vision system accurately detect environmental fall risks in real time during daily activities?
• RQ2 How effective is a CNN in classifying diverse environmental hazards such as stairs, curbs, and uneven surfaces from first-person video?
• RQ3 What level of classification accuracy can be achieved using end-to-end deep learning on wearable camera data?
• RQ4 How does the system perform across different types of fall-related environmental hazards?
• RQ5 Can the system detect subtle environmental changes that may lead to balance perturbations?

Key findings

• The proposed system achieved a mean square error (MSE) of 8% in classifying environmental fall risks, indicating strong predictive performance.
• The CNN effectively learned to distinguish between 12 distinct hazard classes, including both slope-related (e.g., stairs, ramps) and surface-related (e.g., gravel, grass) risks.
• First-person video from a chest-mounted GoPro provided reliable visual evidence of environmental conditions during balance perturbations.
• The end-to-end deep learning approach eliminated the need for manual feature engineering, improving robustness and scalability.
• The system demonstrated feasibility for real-time fall risk detection in naturalistic, daily living environments.
• The results suggest that wearable vision systems with CNNs can support proactive fall prevention in older adults.