[Paper Review] Immersive Virtual Reality and Robotics for Upper Extremity Rehabilitation

Vuthea Chheang, Rakshith Lokesh+8 more|arXiv (Cornell University)|Apr 21, 2023

Stroke Rehabilitation and RecoveryMedicine3 citations

TL;DR

This paper presents an immersive VR and robotics framework for upper extremity rehabilitation that integrates a wearable nanocomposite sleeve sensor to monitor elbow joint movements alongside a KinArm robotic system. The pilot study (n=16) showed significant differences in movement accuracy and elbow resistance changes between Circle and Diamond tasks, with strong usability, low task load, and high presence, demonstrating the system’s potential for personalized, home-based therapy.

ABSTRACT

Stroke patients often experience upper limb impairments that restrict their mobility and daily activities. Physical therapy (PT) is the most effective method to improve impairments, but low patient adherence and participation in PT exercises pose significant challenges. To overcome these barriers, a combination of virtual reality (VR) and robotics in PT is promising. However, few systems effectively integrate VR with robotics, especially for upper limb rehabilitation. This work introduces a new virtual rehabilitation solution that combines VR with robotics and a wearable sensor to analyze elbow joint movements. The framework also enhances the capabilities of a traditional robotic device (KinArm) used for motor dysfunction assessment and rehabilitation. A pilot user study (n = 16) was conducted to evaluate the effectiveness and usability of the proposed VR framework. We used a two-way repeated measures experimental design where participants performed two tasks (Circle and Diamond) with two conditions (VR and VR KinArm). We observed no significant differences in the main effect of conditions for task completion time. However, there were significant differences in both the normalized number of mistakes and recorded elbow joint angles (captured as resistance change values from the wearable sleeve sensor) between the Circle and Diamond tasks. Additionally, we report the system usability, task load, and presence in the proposed VR framework. This system demonstrates the potential advantages of an immersive, multi-sensory approach and provides future avenues for research in developing more cost-effective, tailored, and personalized upper limb solutions for home therapy applications.

Motivation & Objective

• To address low adherence and participation in upper limb physical therapy post-stroke.
• To develop an integrated system combining immersive VR, end-point robotics (KinArm), and wearable sensors for holistic upper limb assessment.
• To evaluate the effectiveness, usability, task load, and sense of presence of the VR-robotic framework in a pilot user study.
• To enable more personalized, cost-effective, and home-friendly rehabilitation solutions through multi-sensory feedback and real-time movement tracking.

Proposed method

• Developed a VR environment with two reaching tasks—Circle and Diamond—using inverse kinematics for virtual avatar and robot control.
• Integrated a wearable, knitted nanocomposite sensor into a one-size-fits-all sleeve to capture real-time elbow joint angle changes via resistance variation.
• Synchronized the VR environment with the KinArm robotic device to enable physical interaction and motion tracking in a shared virtual space.
• Used a two-way repeated measures design with two conditions: VR-only and VR-KinArm hybrid, across two tasks.
• Employed standardized questionnaires (SUS, NASA-TLX, IPQ) to assess usability, task load, and sense of presence.
• Collected objective performance data including task completion time, normalized number of mistakes, and resistance change values from the wearable sensor.

Figure 1 . Overview of the therapeutic system for upper extremity rehabilitation using immersive VR and end-point robotics (KinArm): (a) technical setup for study conditions, (b) the VR environment with virtual avatar and virtual robotics with inverse kinematics, (c) the first-person view in VR, and

Experimental results

Research questions

• RQ1 How does the integration of immersive VR with end-point robotics affect user performance in upper extremity rehabilitation tasks?
• RQ2 To what extent does the wearable sleeve sensor provide reliable and distinct measurements of elbow joint movement during different task types?
• RQ3 How do users rate the usability, task load, and sense of presence in the proposed VR-robotic rehabilitation framework?
• RQ4 Can the system distinguish between different movement patterns (e.g., curved vs. angular trajectories) based on sensor and performance data?

Key findings

• There was no significant difference in task completion time between the VR and VR-KinArm conditions, indicating comparable efficiency.
• Significant differences were found in the normalized number of mistakes and resistance change values from the wearable sensor between the Circle and Diamond tasks, indicating task shape influences movement accuracy and joint loading.
• The system demonstrated high usability (SUS scores) and low task load (NASA-TLX), with no significant differences between conditions, suggesting both are feasible for clinical use.
• Participants reported high levels of perceived realism and general presence, with 'experienced realism' rated highest on the IPQ, despite 'spatial presence' being lowest—likely due to physical constraints of the robot.
• The wearable sensor successfully captured distinct resistance patterns corresponding to different movement trajectories, enabling non-line-of-sight, continuous monitoring of elbow joint motion.
• Nine of 16 participants had no prior VR experience, yet still reported strong immersion and usability, indicating accessibility for novice users.

Figure 2 . Technical setup of the VR upper rehabilitation and robotics.