[Paper Review] Empowerment -- an Introduction

Christoph Salge, Cornelius Glackin+1 more|arXiv (Cornell University)|Oct 7, 2013

Cognitive Science and Mapping4 citations

TL;DR

This paper introduces empowerment as an information-theoretic, task-independent utility function that quantifies an agent's ability to influence its environment through sensorimotor control. By modeling control as channel capacity between actions and sensory inputs, the framework enables intrinsic motivation in agents, with applications across diverse sensorimotor configurations and a proposed continuous-domain approximation for scalability.

ABSTRACT

This book chapter is an introduction to and an overview of the information-theoretic, task independent utility function "Empowerment", which is defined as the channel capacity between an agent's actions and an agent's sensors. It quantifies how much influence and control an agent has over the world it can perceive. This book chapter discusses the general idea behind empowerment as an intrinsic motivation and showcases several previous applications of empowerment to demonstrate how empowerment can be applied to different sensor-motor configuration, and how the same formalism can lead to different observed behaviors. Furthermore, we also present a fast approximation for empowerment in the continuous domain.

Motivation & Objective

• To formalize empowerment as a task-independent utility function grounded in information theory.
• To demonstrate how empowerment can serve as an intrinsic motivator for autonomous agents without external rewards.
• To show the versatility of the empowerment framework across different sensorimotor architectures and environments.
• To present a fast approximation method for computing empowerment in continuous state and action spaces.
• To provide a comprehensive overview of empowerment's theoretical foundations and practical applications in artificial intelligence.

Proposed method

• Define empowerment as the channel capacity between an agent's actions and its sensory inputs, using information theory.
• Model the agent's sensorimotor system as a stochastic channel, where actions probabilistically influence sensory outcomes.
• Formulate empowerment as the mutual information between action and sensory state, maximizing over action policies.
• Apply the framework to discrete and continuous domains, using variational inference and Gaussian process approximations for continuous cases.
• Demonstrate the method through simulations and case studies in robotic and artificial agent environments.
• Use the principle of maximizing empowerment to guide exploration and behavior selection without external reward signals.

Experimental results

Research questions

• RQ1 How can empowerment be formalized as a task-independent measure of an agent's control over its environment?
• RQ2 In what ways can empowerment serve as a viable intrinsic motivation for autonomous agents in diverse sensorimotor configurations?
• RQ3 What are the computational challenges of applying empowerment in continuous state and action spaces, and how can they be addressed?
• RQ4 How does empowerment lead to emergent behaviors such as exploration, navigation, and self-preservation in agent systems?
• RQ5 What are the theoretical and practical implications of using channel capacity as a proxy for agency and autonomy?

Key findings

• Empowerment successfully quantifies an agent's effective control over its environment through information-theoretic channel capacity.
• The framework enables intrinsic motivation in agents, leading to behaviors such as exploration and environmental interaction without external rewards.
• Empowerment can be applied across various sensorimotor configurations, including robots and simulated agents, with consistent behavioral outcomes.
• A fast approximation method for continuous domains was developed, enabling practical deployment in high-dimensional systems.
• Empirical results show that agents maximizing empowerment naturally explore their environment and avoid getting trapped in low-control states.
• The formalism supports both discrete and continuous settings, with strong theoretical grounding in information theory and stochastic control.