[Paper Review] See, Learn, Assist: Safe and Self-Paced Robotic Rehabilitation via Video-Based Learning from Demonstration

Ali Alabbas, Camillo Murgia+2 more|arXiv (Cornell University)|Mar 15, 2026

Stroke Rehabilitation and Recovery0 citations

TL;DR

A video-based imitation learning framework encodes therapist demonstrations into body-centric 6-DoF Cartesian Dynamic Movement Primitives to safely and adaptively drive cobot-assisted rehabilitation across passive, active-assisted, and active-resistive modes, with on-the-fly safety via GMR-based force monitoring and reversible trajectories.

ABSTRACT

In this paper, we propose a novel framework that allows therapists to teach robot-assisted rehabilitation exercises remotely via RGB-D video. Our system encodes demonstrations as 6-DoF body-centric trajectories using Cartesian Dynamic Movement Primitives (DMPs), ensuring accurate posture-independent spatial generalization across diverse patient anatomies. Crucially, we execute these trajectories through a decoupled hybrid control architecture that constructs a spatially compliant virtual tunnel, paired with an effort-based temporal dilation mechanism. This architecture is applied to three distinct rehabilitation modalities: Passive, Active-Assisted, and Active-Resistive, by dynamically linking the exercise's execution phase to the patient's tangential force contribution. To guarantee safety, a Gaussian Mixture Regression (GMR) model is learned on-the-fly from the patient's own limb. This allows the detection of abnormal interaction forces and, if necessary, reverses the trajectory to prevent injury. Experimental validation demonstrates the system's precision, achieving an average trajectory reproduction error of 3.7cm and a range of motion (ROM) error of 5.5 degrees. Furthermore, dynamic interaction trials confirm that the controller successfully enforces effort-based progression while maintaining strict spatial path adherence against human disturbances.

Motivation & Objective

• Enable remote teaching of robot-assisted rehabilitation exercises from RGB-D video by therapists.
• Achieve posture-independent spatial generalization across diverse patient anatomies via body-centric trajectory encoding.
• Provide a multi-modal hybrid control framework that enforces spatial path adherence while adapting execution speed to patient effort.
• Develop a on-the-fly safety mechanism using Gaussian Mixture Regression to detect abnormal interaction forces and reversibly stop trajectories.

Proposed method

• Perception module uses YOLOv8-pose and Mediapipe Hand to extract 3D 6-DoF poses from RGB-D, constructing a body-centric frame anchored at the shoulder.
• Learning module encodes 6-DoF trajectories with Cartesian Dynamic Movement Primitives for scalable, posture-invariant motion.
• Execution module implements a decoupled spatial tunnel with an effort-based temporal dilation to couple progression to tangential patient effort.
• A Gaussian Mixture Regression model is learned on-the-fly from the patient’s limb to form a dynamic safety corridor and trigger trajectory reversals when forces exceed safe bounds.

Experimental results

Research questions

• RQ1 Can therapist demonstrations captured in RGB-D be encoded into body-centric 6-DoF Cartesian DMPs that generalize across different patient anatomies?
• RQ2 Does the decoupled spatial-temporal control with an effort-based dilation ensure safe, self-paced execution across Passive, Active-Assisted, and Active-Resistive rehabilitation modes?
• RQ3 Can a patient-specific GMR model effectively detect abnormal interaction forces and safely reverses trajectories to prevent injury?

Key findings

• Average trajectory reproduction error of 3.7 cm across exercises and subjects.
• ROM error around 5.5 degrees on validated movements.
• Virtual tunnel with orthogonal admittance maintains spatial adherence under significant orthogonal forces (max spatial deviation 3.7 cm in baseline test).
• Force-based dilation enables Passive, Active-Assisted, and Active-Resistive modalities with self-paced progression.
• GMR-based safety corridor triggers smooth trajectory reversals within ~0.3 s of force violations, ensuring safe operation.