[Paper Review] Brain-Swarm Interface (BSI): Controlling a Swarm of Robots with Brain and Eye Signals from an EEG Headset

Aamodh Suresh, Mac Schwager|arXiv (Cornell University)|Dec 24, 2016

EEG and Brain-Computer Interfaces3 references3 citations

TL;DR

This paper introduces a Brain-Swarm Interface (BSI) that enables a human user to control a swarm of robots using only thoughts and eye movements, recorded via an off-the-shelf EEG headset. By combining Hidden Markov Models for thought decoding and multi-step signal processing for eye movement detection, the system modulates potential field-based swarm controllers to achieve real-time control of swarm dispersion, aggregation, and direction in both simulation (128 robots) and hardware experiments (3 M3pi robots).

ABSTRACT

This work presents a novel marriage of Swarm Robotics and Brain Computer Interface technology to produce an interface which connects a user to a swarm of robots. The proposed interface enables the user to control the swarm's size and motion employing just thoughts and eye movements. The thoughts and eye movements are recorded as electrical signals from the scalp by an off-the-shelf Electroencephalogram (EEG) headset. Signal processing techniques are used to filter out noise and decode the user's eye movements from raw signals, while a Hidden Markov Model technique is employed to decipher the user's thoughts from filtered signals. The dynamics of the robots are controlled using a swarm controller based on potential fields. The shape and motion parameters of the potential fields are modulated by the human user through the brain-swarm interface to move the robots. The method is demonstrated experimentally with a human controlling a swarm of three M3pi robots in a laboratory environment, as well as controlling a swarm of 128 robots in a computer simulation.

Motivation & Objective

• To develop an intuitive, non-invasive human-machine interface for controlling large-scale robot swarms using only brain and eye signals.
• To address the lack of BCI applications in swarm robotics by integrating neuroscience, signal processing, and control theory.
• To enable users with motor impairments to manipulate complex environments through collective robot action.
• To demonstrate real-time, online control of swarm behavior—specifically aggregation, dispersion, and directional motion—using EEG-based inputs.
• To validate the system in both high-fidelity simulation and physical hardware experiments with real robots.

Proposed method

• Uses an off-the-shelf Emotiv Epoc EEG headset to non-invasively record scalp electrical signals from brain activity and eye movements.
• Applies multi-step signal processing to filter and decode horizontal and vertical eye movement signals from raw EEG data.
• Employs a Hidden Markov Model (HMM) to classify filtered brain signals into discrete thought states representing swarm aggregation or dispersion.
• Maps decoded thought states and eye movement directions to dynamic parameters (e.g., potential field shape and scale) in a potential field-based swarm controller.
• Transmits control commands wirelessly via Zigbee at 30 Hz to individual M3pi robots, which use a proportional point-offset controller to generate motor speeds.
• Integrates a motion capture system (Optitrack) for real-time robot pose feedback and visual monitoring via live video feed.

Experimental results

Research questions

• RQ1 Can a hybrid BCI system effectively decode both thought-based intentions and eye movement signals from off-the-shelf EEG hardware for swarm control?
• RQ2 Can a potential field-based swarm controller be dynamically modulated in real time using decoded brain and eye signals to achieve desired swarm behaviors?
• RQ3 Is it feasible to control a physical swarm of robots (3 M3pi robots) and a simulated swarm (128 robots) using only non-invasive EEG signals in an online, real-time setting?
• RQ4 How reliable and accurate are the HMM-based thought state estimation and eye movement detection in a complex, multi-component system with human cognitive load?
• RQ5 Can such a system serve as a viable, intuitive interface for individuals with motor impairments to control multiple robots simultaneously?

Key findings

• The system successfully demonstrated real-time control of a swarm of 3 M3pi robots in a physical lab environment using only EEG signals.
• The HMM-based thought state estimation achieved high confidence in distinguishing between aggregation and dispersion intentions during the experiment.
• Eye movement detection accurately steered the swarm along a predefined path with four sequential sectors, confirming directional control capability.
• In simulation, the swarm of 128 robots followed a complex path with correct switching between dispersion and aggregation states as intended.
• The system maintained stable performance across both hardware and simulation setups, with consistent visual feedback and control response.
• Despite challenges in human cognitive load and hardware complexity, the system achieved reliable operation over multiple experimental trials.