[Paper Review] Self-learning and adaptation in a sensorimotor framework

Ali Ghadirzadeh, Judith Bütepage+2 more|arXiv (Cornell University)|Jan 5, 2016

Gaussian Processes and Bayesian Inference15 references4 citations

TL;DR

This paper proposes a self-learning sensorimotor framework that enables a robot to autonomously learn and adapt its motor commands using Gaussian processes with automatic relevance determination for dimensionality reduction. By combining RRT* for long-term planning and gradient-based optimization for real-time action selection, the system achieves fast, data-efficient learning (under 12 seconds) and rapid adaptation to environmental changes such as added load, demonstrating real-time performance in high-dimensional joint space control.

ABSTRACT

We present a general framework to autonomously achieve a task, where autonomy is acquired by learning sensorimotor patterns of a robot, while it is interacting with its environment. To accomplish the task, using the learned sensorimotor contingencies, our approach predicts a sequence of actions that will lead to the desirable observations. Gaussian processes (GP) with automatic relevance determination is used to learn the sensorimotor mapping. In this way, relevant sensory and motor components can be systematically found in high-dimensional sensory and motor spaces. We propose an incremental GP learning strategy, which discerns between situations, when an update or an adaptation must be implemented. RRT* is exploited to enable long-term planning and generating a sequence of states that lead to a given goal; while a gradient-based search finds the optimum action to steer to a neighbouring state in a single time step. Our experimental results prove the successfulness of the proposed framework to learn a joint space controller with high data dimensions (10$ imes$15). It demonstrates short training phase (less than 12 seconds), real-time performance and rapid adaptations capabilities.

Motivation & Objective

• To enable autonomous learning of sensorimotor tasks without prior expert knowledge or robot model assumptions.
• To address the challenge of high-dimensional sensory and motor spaces under uncertainty and dynamic environmental changes.
• To develop a system capable of detecting and adapting to external perturbations, such as changes in load or task conditions.
• To integrate long-term planning with real-time action optimization for constrained, non-convex control tasks.
• To achieve data-efficient, incremental learning while automatically identifying relevant sensory and motor dimensions.

Proposed method

• Uses Gaussian processes with automatic relevance determination (ARD) to learn forward sensorimotor mappings and identify relevant input dimensions in high-dimensional spaces.
• Employs an incremental learning strategy that distinguishes between model updates and system adaptations based on prediction errors.
• Applies RRT* with goal-biased sampling to generate action sequences that minimize overshoot and satisfy task constraints.
• Uses gradient-based optimization at each time step to compute the optimal action for transitioning to a neighboring state.
• Detects mismatches between predicted and actual sensory outcomes (e.g., joint positions) to trigger adaptation when prediction error exceeds a threshold.
• Replaces outdated training data pairs with new measurements collected under changed conditions to enable rapid adaptation.

Experimental results

Research questions

• RQ1 How can a robot autonomously learn optimal motor commands using only sensorimotor feedback without prior models or expert demonstrations?
• RQ2 How can a system detect and adapt to changes in the environment, such as added load, during task execution?
• RQ3 To what extent can RRT* planning improve trajectory quality by minimizing overshoot in high-dimensional control tasks?
• RQ4 How effective is automatic relevance determination in reducing computational cost by identifying relevant sensory and motor components?
• RQ5 Can the framework maintain real-time performance and achieve fast convergence in complex, dynamic environments?

Key findings

• The framework achieved successful learning of a joint space controller in less than 12 seconds for a 10×15 dimensional task space.
• The system demonstrated real-time performance with rapid adaptation to external perturbations, such as a 300g load added to the robot’s gripper.
• With dimensionality reduction, the system reduced overshoot and stabilized at the goal state, as shown in Fig. 6 (left), where planning began after ~32 seconds.
• Adaptation was triggered by prediction errors exceeding a threshold, enabling the system to detect and correct for mismatches in applied torque within a few iterations.
• RRT* planning successfully generated action sequences that avoided overshoot, as illustrated in Fig. 7, by favoring goal-proximate samples and minimizing cost.
• The method enabled active detection of changes and replacement of outdated training data with new measurements under altered conditions, ensuring robustness to dynamic environments.