Controlling Robots with

Non-Invasive Brain-Computer Interfaces

Elliott Forney

Colorado State UniversityBrain-Computer Interfaces Group

February 21, 2013

1 / 11

Brain-Computer Interfaces

Brain-Computer Interface (BCI)

Direct channel of between brain and machine

Voluntary changes in mental state

Bypasses innate motor-based means of communication

2 / 11

Brain-Computer Interfaces

Brain-Computer Interface (BCI)

Direct channel of between brain and machine

Voluntary changes in mental state

Bypasses innate motor-based means of communication

2 / 11

Uses for BCI

BCI have many potential uses

Reestablish communication with people who are Locked-in

ALS, stroke, traumatic brain injury

Assistive technology

electric wheelchairs, computers, telephones

Everyday devices

video games, monitoring emotional states

Driving kewl robots around the lab with your mind!

3 / 11

Uses for BCI

BCI have many potential uses

Reestablish communication with people who are Locked-in

ALS, stroke, traumatic brain injury

Assistive technology

electric wheelchairs, computers, telephones

Everyday devices

video games, monitoring emotional states

Driving kewl robots around the lab with your mind!

3 / 11

Uses for BCI

BCI have many potential uses

Reestablish communication with people who are Locked-in

ALS, stroke, traumatic brain injury

Assistive technology

electric wheelchairs, computers, telephones

Everyday devices

video games, monitoring emotional states

Driving kewl robots around the lab with your mind!

3 / 11

Uses for BCI

BCI have many potential uses

Reestablish communication with people who are Locked-in

ALS, stroke, traumatic brain injury

Assistive technology

electric wheelchairs, computers, telephones

Everyday devices

video games, monitoring emotional states

Driving kewl robots around the lab with your mind!

3 / 11

Uses for BCI

BCI have many potential uses

Reestablish communication with people who are Locked-in

ALS, stroke, traumatic brain injury

Assistive technology

electric wheelchairs, computers, telephones

Everyday devices

video games, monitoring emotional states

Driving kewl robots around the lab with your mind!

3 / 11

Electroencephalography

Electroencephalography (EEG) to measure brain activity

Non-invasive, portable, relatively inexpensive

Superficial & noisy signals

Time (s)

FP1

FP2

F3

F4

C3

C4

P3

P4

O1

O2

F7

T3

T4

T5

T6

CZ

FZ

PZ

0 1 2 3 4 5

4 / 11

Electroencephalography

Electroencephalography (EEG) to measure brain activity

Non-invasive, portable, relatively inexpensive

Superficial & noisy signals

Time (s)

FP1

FP2

F3

F4

C3

C4

P3

P4

O1

O2

F7

T3

T4

T5

T6

CZ

FZ

PZ

0 1 2 3 4 5

4 / 11

Electroencephalography

Electroencephalography (EEG) to measure brain activity

Non-invasive, portable, relatively inexpensive

Superficial & noisy signals

Time (s)

FP1

FP2

F3

F4

C3

C4

P3

P4

O1

O2

F7

T3

T4

T5

T6

CZ

FZ

PZ

0 1 2 3 4 5

4 / 11

Machine Learning & Pattern Analysis

Machine Learning algorithms identify patterns in EEG

This is extremely difficult because

different for each person

change over time

noise & artifacts

the brain is complex!

6 7 8 9 10

Time (s)

Sig

na

l

EEG

RNN

−3

−1

13

5 / 11

Machine Learning & Pattern Analysis

Machine Learning algorithms identify patterns in EEG

This is extremely difficult because

different for each person

change over time

noise & artifacts

the brain is complex!

6 7 8 9 10

Time (s)

Sig

na

l

EEG

RNN

−3

−1

13

5 / 11

Synchronous BCI

Some BCI use patterns associated with external stimuli

P300 speller is an example

User looks for a specific character in a series or grid of

flashing characters

6 / 11

Synchronous BCI

Some BCI use patterns associated with external stimuli

P300 speller is an example

User looks for a specific character in a series or grid of

flashing characters

6 / 11

Asynchronous BCI

Other BCI do not require

external stimuli

Mental Tasks is an

example

Imagine left arm moving

moves to the left while

silently singing a song

moves to the right

7 / 11

Asynchronous BCI

Other BCI do not require

external stimuli

Mental Tasks is an

example

Imagine left arm moving

moves to the left while

silently singing a song

moves to the right

7 / 11

Robots

Controlling robots is a first step toward real-world

applications

Drive a robot before an electric wheelchair

Demonstrates that user has control

Just plain fun!

8 / 11

Robots

Controlling robots is a first step toward real-world

applications

Drive a robot before an electric wheelchair

Demonstrates that user has control

Just plain fun!

8 / 11

Ideas for Robot Club, I

Current robot module is hacked together but better

systems to come.

Some ideas for robot club to start on

Streaming video through sockets, to python

Better user interfaces

Fluid control

How far to move

Control gripper

9 / 11

Ideas for Robot Club, I

Current robot module is hacked together but better

systems to come.

Some ideas for robot club to start on

Streaming video through sockets, to python

Better user interfaces

Fluid control

How far to move

Control gripper

9 / 11

Ideas for Robot Club, I

Current robot module is hacked together but better

systems to come.

Some ideas for robot club to start on

Streaming video through sockets, to python

Better user interfaces

Fluid control

How far to move

Control gripper

9 / 11

Ideas for Robot Club, I

Current robot module is hacked together but better

systems to come.

Some ideas for robot club to start on

Streaming video through sockets, to python

Better user interfaces

Fluid control

How far to move

Control gripper

9 / 11

Ideas for Robot Club, II

Safety systems

Don’t hit a wall

Don’t go down stairs

Hybrid control

Turn away from walls

Move toward objects

10 / 11

Ideas for Robot Club, II

Safety systems

Don’t hit a wall

Don’t go down stairs

Hybrid control

Turn away from walls

Move toward objects

10 / 11

Thanks!

11 / 11