Introduction to Brain-Computer Interfaces

Saim Rasheed

Dipartimento Di Informatica e Comunicazione

Università Degli Studi Di Milano

2

On the Way…• What is BCI?• Different Brain’s Regions• Motor Map and Motor Imagery• Neuronal Structure• Why EEG?• Brain Rhythms• Electrode Placement System• Different kinds of Artefacts• BCI Example• Generic Model of BCI• Feedback types and their Effects on classification• BCI System Setup• Conclusion

3

• Imagine being alert and aware of your environment, but unable to move, speak or express yourself!

• This condition is called locked in syndrome.

• Patients are truly locked in to their bodies.

Background and Motivation

4

• The term locked-in refers to a state in which individuals are conscious and alert, but unable to use their muscles and therefore cannot communicate their needs, wishes, and emotions: the healthy brain is locked into a paralyzed body.

• This condition motivates us to restore their communication of those individuals who have lost the ability to communicate by speech or other muscular activities.

Background and Motivation

5

Introduction - What is BCI ?• The promise of Brain-Computer Interfaces (BCI) technology is

to augment human capabilities by enabling people to interact with a computer through their brainwaves after a short training period.

• BCI translate your brain’s electrical activity into messages or commands.

• A BCI is like a closed loop system where information is visually fed back to the user.

6

• For example, Healthy participants may move forward within a virtual environment (VE) by the imagination of foot movement

• It is achieved by using a brain-computer interface (BCI) that transforms thought-modulated electroencephalogram (EEG) recordings into a control signal.

• A BCI establishes a communication channel between the human brain and the computer.

Introduction - What is BCI ?

7

Introduction – What is BCI ?

• The basic principle of the BCI is the detection and classification of motor-imagery-related EEG pattern.

• Remember that BCI cannot:– Write to the brain– Read your thoughts...

8

Human Brain

9

Responsibilities of each part

• Frontal Lobe- associated with reasoning, planning, parts of speech, movement, emotions, and problem solving

• Parietal Lobe- associated with movement, orientation, recognition, perception of stimuli

• Occipital Lobe- associated with visual processing

• Temporal Lobe- associated with perception and recognition of auditory stimuli, memory, and speech

10

Activation sequence for motor areas

11

• The basic function of the brain is to produce behaviours (set of movements produced by the muscles). Planning for any given movement is done in the forward portion of the frontal lobe. This part of the cortex receives information about the individual's current position from several other parts. Then it issues its commands, to Area 6. It decides which set of muscles to contract to achieve the required movement. which issues the corresponding orders to the Primary motor cortex. This area in turn activates specific muscles or groups of muscles via the motor neurons in the spinal cord.

Activation sequence for motor areas

12

Motor Map

13

Motor Map

• A Penfield homunculus is a physical representation of the primary motor cortex, i.e., the portion of the human brain directly responsible for the movement and exchange of sense and motor information (namely touch: sensitivity, cold, heat, pain etc.) of the rest of the body.

• The most striking aspect of this map is that the areas

assigned to various body parts on the cortex are proportional not to their size, but rather to the complexity of the movements that they can perform.

14

• Hence, the areas for the hand and face are especially large compared with those for the rest of the body

• This is no surprise, because the speed and dexterity of human hand and mouth movements are precisely what give us two of our most distinctly human faculties: the ability to use tools and the ability to speak.

Motor Map

15

Motor Imagery

• It is broadly accepted that mental imagination of movements involves similar brain regions that are used in programming and preparing such movements.

• Motor Imagery defined as mental simulation of a movement – an efficient mental strategy to operate a BCI.

16

• The imagination of different types of movements, for example, right hand, left hand, foot, or tongue movement, results in a characteristic change of the brain signals over the sensorimotor cortex.

Motor Imagery

17

Brain’s Unit

18

• Cell body - This main part has all of the necessary components of the cell, such as the nucleus (contains DNA), endoplasmic reticulum and ribosomes (for building proteins) and mitochondria (for making energy). If the cell body dies, the neuron dies.

• Axon - This long, cable-like projection of the cell carries the electrochemical message (nerve impulse or action potential) along the length of the cell.

• Depending upon the type of neuron, axons can be covered with a thin layer of myelin, like an insulated electrical wire. Myelin is made of fat, and it helps to speed transmission of a nerve impulse down a long axon. Myelinated neurons are typically found in the peripheral nerves (sensory and motor neurons), while non-myelinated neurons are found in the brain and spinal cord.

• Dendrites or nerve endings - These small, branch-like projections of the cell make connections to other cells as well as to other dendrite and axons.

Brain’s Unit

19

Brain’s Unit Synapses

20

Neurons also vary with respect to their functions: • Sensory neurons carry signals from the outer parts of your

body (periphery) into the central nervous system. • Motor neurons (motorneurons) carry signals from the central

nervous system to the outer parts (muscles, skin, glands) of your body.

• Receptors sense the environment (chemicals, light, sound, touch) and encode this information into electrochemical messages that are transmitted by sensory neurons.

• Interneurons connect various neurons within the brain and spinal cord.

Brain’s Unit

21

• When the doctor taps the right spot on your knee with a rubber hammer, receptors send a signal into the spinal cord through a sensory neuron. The sensory neuron passes the message to a motor neuron that controls your leg muscles. Nerve impulses travel down the motor neuron and stimulate the appropriate leg muscle to contract

Knee Experiment Reflex Action

22

Along an Axon-Action Potential

23

Action Potential• A stimulus is received by the dendrites of a nerve cell. This

causes the Na+ channels to open. If the opening is sufficient to drive the interior potential from -70 mV up to -55 mV, the process continues.

• Having reached the action threshold, more Na+ channels (sometimes called voltage-gated channels) open. The Na+ influx drives the interior of the cell membrane up to about +30 mV. The process to this point is called depolarization.

• The Na+ channels close and the K+ channels open. Since the K+ channels are much slower to open, the depolarization has time to be completed. Having both Na+ and K+ channels open at the same time would drive the system toward neutrality and prevent the creation of the action potential.

24

Action Potential

25

Action Potential• With the K+ channels open, the membrance begins to

repolarize back toward its rest potential.

• The repolarization typically overshoots the rest potential to about -90 mV. This is called hyperpolarization and would seem to be counterproductive, but it is actually important in the transmission of information. Hyperpolarization prevents the neuron from receiving another stimulus during this time, or at least raises the threshold for any new stimulus. Part of the importance of hyperpolarization is in preventing any stimulus already sent up an axon from triggering another action potential in the opposite direction. In other words , hyperpolarization assures that the signal is proceeding in one direction.

• After hyperpolarization, the Na+/K+ pump eventually brings the membrane back to its resting state of -70 mV .

26

Neuro Transmitters - Acetylcholine

•The chemical compound acetylcholine is one of the neurotransmitters in both the peripheral nervous system (PNS) and central nervous system (CNS) in many organisms including humans.

27

• average skull thickness.

– men : 6.5 millimeters,

– women : 7.1 mm

• average front-to-back

– 176 mm for men

– 171 mm for women

• average width

– 145 mm for men

– 140 mm for women

Average Thickness and Resistivities

28

Electroencephalogram• Electroencephalogram (EEG) are the electrical signals that

can be recorded from the brain, either directly or through the scalp by hooking up the electrodes.

• On the scalp the amplitudes commonly lie within 10–100 μV

• A useful way of observing human brain activity – A new communication channel.

• An Austrian psychiatrist named Hans Berger was the first to record this activity in humans, in the late 1920s.

29

Technology RemarksElectro-corticography (ECoG) Highly invasive, surgery

Magneto-encephalography (MEG) Very expensive

Computed Tomography (CT) Only anatomical data

Single Photon Emission Computerized Tomography (SPECT)

Radiation Exposure

Positron Emission Tomography (PET) Radiation Exposure

Magnetic Resonance Imaging (MRI) Only anatomical data

Functional Magnetic Resonance Imaging (fMRI)

Extremely Expensive

Electroencephalogram (EEG),

Electromyogram (EMG),Electroocculogram (EOG)

Non-invasive, relatively cheaper, No exposure to radiation, non-anatomical data, easily portable

Why do all BCIs use EEGs?

30

Why do all BCIs use EEGs?●Other approaches are not portable.

●They also require millions of Euro’s of equipment and highly trained technician and are less universal.

31

Brain Rhythms

• EEG of human brain contains different types of oscillatory activities. The oscillations in the alpha and beta band are particularly important to use in discriminating between different brain states.

• DELTA 0.5 – 4 HZ– Deep dreamless sleep, non-REM sleep, unconscious state

• THETA 4 – 8 Hz– Deep relaxation and meditation, mental imagery, dream, fantasy – amplitude usually greater than 20 μ V.

• ALPHA 8 – 13 Hz– Usually found over occipital region of brain, indicate relaxed

awareness without any attention or concentration, not thinking– Amplitude is normally less then 30 - 50 µv.

32

Brain Rhythms

• BETA 13-30 Hz– Appear in frontal and parietal region, Associated with active

thinking, active attention, focus on the outside world or solving concrete problems or simply normal alert consciousness.

• GAMMA 30-50 Hz– In some cases, detection of gamma waves can be used for

confirmation of certain brain diseases.– Recent studies proposed that activity in gamma may be related to

higher mental activity (i.e. processing of multi-sensorial stimuli) • Mu-Rhytm

– associated with motor activities and maximally recorded over motor cortex. Mu wave is in the same frequency band as in the alpha wave

33

Brain Rhythms

• Alpha and Beta bands are often subdivided in Low and High sub-bands

• Relations between different bands is still under analysis

– Ratio between beta and alpha has been proposed as can an index of the level of attention or arousal of a user during a task-based experiment

34

EEG Recording and Measurement

• The first electrical neural activities were registered using simple galvanometers.

• More recently EEG systems consist of a number of delicate electrodes.

• the effective bandwidth for EEG signals is limited to approximately 100 Hz.

• a minimum frequency of 200 samples/s is often enough for sampling the EEG signals

• In some applications where a higher resolution is required for representation of brain activities in the frequency domain, sampling frequencies of up to 2000 sample/s may be used.

35

• Representation of each signal sample with up to 16 bits is very popular for the EEG recording systems.

• A simple calculation shows that for a one hour recording from 128-electrode EEG signals sampled at 500 samples/s a memory size of 128 × 60 × 60 × 500 × 16 ≈ 3.68 Gbits ≈0.45 Gbyte is required.

• To enable a satisfactory recording the electrode impedances should read less than 5 k Ω and be balanced to within 1 kΩ of each other.

EEG Recording and Measurement

36

Types of electrodes

• disposable (gel-less, and pre-gelled types);

• reusable disc electrodes (gold, silver, stainless steel, or tin);

• headbands and electrode caps;

• saline-based electrodes;

• needle electrodes.

37

EEG 10 - 20 Electrode Placement

38

EEG Electrode Placement• Relationship between location of an electrode and the

underlying area of cerebral cortex.• the actual distances between adjacent electrodes are either

10% or 20% of the total front-back or right-left distance of the skull.

• The letters F, T, C, P and O stand for Frontal, Temporal, Central, Parietal and Occipital respectively.

• A "z" (zero) refers to an electrode placed on the midline.• Even numbers refers to right hemisphere and odd numbers

refers to left hemisphere.• Nasion: point between the forehead and nose.• Inion: Bump at back of skull.

39

21 Electrode Placement

40

75 Electrode Placement

41

Maudsley electrode positioning system

• In Maudsley electrode positioning system the outer electrodes are slightly lowered to enable better capturing of the required signals from epileptic foci in epileptic seizure recordings.

42

Artefacts

• Physiological Artefacts

– patient-related or internal artefacts are body movement-related, EMG, ECG (and pulsation), EOG , ballistocardiogram, and sweating

• System Artefacts

– The system artefacts are 50/60 Hz power supply interference, impedance fluctuation, cable defects, electrical noise from the electronic components, and unbalanced impedances of the electrodes.

43

Artefact Waveforms

44

Event-related (de)synchronization

• ERD/ERS is the task-related or event-related change in the amplitude of the oscillatory behavior of specific cortical areas within various frequency bands. An amplitude (or power) increase is defined as event-related synchronization while an amplitude (or power) decrease is defined as event-related desynchronization.

45

Visual evoked potential

• The visual evoked potential (VEP) is the electrical response of the brain's primary visual cortex to a visual stimulus.

46

Steady state Visual evoked potentials

• SSVEPs occur when sensory stimuli are repetitively delivered at high enough rates so that the relevant neuronal structures are prevented to return to their resting states

47

P300 Potential

• The P300 is a positive evoked potential that appears 300 ms after presentation of an attended stimulus embedded in a sequence of irrelevant stimuli.

• Event-related potential (ERP) P300 auditory evoked potential measure reflects the speed of neural events related to attention and short term memory.

48

BCI Example

• A user views a matrix containing letters and other characters. This is an example of a matrix:

49

BCI Example

• One row or column is briefly flashed, then another, and so on until all rows and columns are flashed. The user counts each time a target (such as the letter K) flashes while ignoring other flashes.

50

BCI Example

• Your brain produced different EEGs in response to ignored vs. counted flashes.

EEGs from a P300 BCI

-4

-2

0

2

4

6

0 100 200 300 400 500 600 700

time after flash begins (ms)

volta

ge (u

V) Nontargets

TargetsK

K

51

Earlier VR-BCI Research

• Based on two types of visually evoked responses.

– steady-state visual evoked potential

– event-related P300 potential

• In contrast, different BCI paradigm for VR control based on motor imagery.

52

Why Virtual Reality ?

• If it is possible to show that people are able to control their movements through space within a VE, it would justify the much bigger expense of building physical devices like a robot arm controlled by a BCI.

53

• It is a safe environment.

• It can be used to control the experience and reduce distractions.

• It can be highly motivating.

• VR as a feedback enhances the classification accuracy and reduces the time needed for BCI training sessions.

Why Virtual Reality ?

54

BCI Goals/Research Problems

• Analysis of the influence of different feedback types on performing the same task.

• Demonstration of how to move through the Virtual Environment, using imagination of foot/hand movement. No muscular activity should be involved.

55

Two Learning Systems• What Machine should learn?

– discrimination between different patterns of brain activity as accurate as possible

• What user of BCI should learn?

– performing different mental tasks in order to produce distinct brain signals.

– If the subjects are well trained (say, classification accuracies above 90%) they can produce the same/distinct EEG patterns over a long/short period for the same/distinct tasks.

56

Basic Components ofBrain-Computer Interface

• A BCI system is, in general, composed of the following components:

– signal acquisition,

– preprocessing,

– feature extraction,

– classification (detection),

– application interface

57

Generic Model for BCI

58

Signal acquisition

• The signal acquisition component is responsible for recording the electrophysiological signals and providing the input to the BCI.

59

Pre-Processing

• Preprocessing includes artifact reduction (electrooculogram, EOG, and electromyogram, EMG), application of signal processing methods, that is, low-pass or highpass filter, methods to remove the influence of the line frequency and in the case of multichannel data the use of spatial filters (bipolar, Laplacian, common average reference)

60

Pre-ProcessingEye blink Removal

• An eye blink or an eye movement is generally detected by a voltage increase above 100µV.

• Rejection method consist of discarding contaminated EEG, based on either automatic or visual detection. Their success crucially depends on the quality of the detection, and its use depends also on the specific application for which it is used

61

Pre-ProcessingArtefact Removal

• Discarding segments of EEG data with artifact can greatly decrease the amount of data available for analysis.

62

Pre-ProcessingEye blink Removal

• Subtraction methods are based on the assumption that the measured EEG is a linear combination of an original EEG and a signal caused by eye movement, called EOG (electrooculogram). The EOG is a potential produced by movement of the eye or eyelid.

• The original EEG is recovered by subtracting separately recorded EOG from the measured EEG

63

Pre-ProcessingArtefact Removal

• Other approaches include:

– Principal component analysis,

– Independent component analysis,

– signal fraction analysis

– canonical correlation analysis.

– Common Spatial patterns

64

Feature Extraction

• The goal of this component is to find a suitable representation (signal features) of the electrophysiological data that simplifies the subsequent classification or detection of specific brain patterns.

• There are a variety of feature extraction methods used in BCI systems. A less than exhaustive list of these methods includes amplitude measures, band power, autoregressive parameters, and wavelets.

65

Classification

• The task of the classifier component is to use the signal features provided by the feature extractor to assign the recorded samples of the signal to a category of brain patterns.

• The classifier output, which can be a simple on-off signal or a signal that encodes a number of different classes, is transformed into an appropriate signal that can then be used to control a VR system.

66

• Without Feedback

– To acquire subject specific data for the used imaginations

• With Feedback

– Wrong feedback can elicit frustration, a response likely to be associated with a widespread EEG desynchronization.

– Correct feedback reinforces the desired EEG patterns.

BCI-Two Adapting Systems

67

Possibilities of Feedback Types

• Through conventional monitor e.g. simple moving bar graph.

• Through VE experiments with HMD.

• Through highly immersive projection environment e.g. CAVE.

68

• BCI2000 (www.bci2000org)

– Free general-purpose system for BCI research and development. It can also be used for data acquisition, stimulus presentation, or brain observation applications

• OpenViBe (openvibe.inria.fr)

– Open-source

– node-based software

BCI application development

69

g.MOBIlab+

• The portable device g.MOBIlab+ supports 8 EEG channels and sends the data via a serial cable or via Bluetooth to the computer system. The wireless transmission allows the subject to move freely in a range of about 30 meters. This enables the easy usage of the system over 36 hours with one battery set. The whole system easily fits into a notebook bag for travelling to different measurement sites.

70

NeuroSky Mindset

• Very affordable (less than 200 euro)• Single dry sensor• Bluetooth connection• Detects activity in the frontal lobe• eSenseTM proprietary Attention/Meditation• Used in videogames but also in research applications

71

References• Leeb R, Scherer R, Keinrath C, Guger C, Pfurtscheller G. October 2006. Walking by Thinking: The

Brainwaves are Crusial, Not the Muscles. Presence Vol 15, No 5 500 – 514 Massachusetts Institute of Technology USA.

• Leeb R, Scherer R, Keinrath C, Guger C, Pfurtscheller G. Exploring virtual environments with an EEG-based BCI through motor imagery. Biomedical Technology Berlin 50(4), 86 – 91.

• E. C. Lalor, S. P. Kelly, C. Finucane, R. Burke, R. Smith, R. B. Reilly, and G.McDarby. 2005Steady-State VEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment. EURASIP, Journal on Applied Signal Processing, 19, 3156 – 3164.

• Bayliss, J.D. 2003. Use of the evoked potential P3 component for control in a virtual apartment. Neural Systems and Rehabilitation Engineering, IEEE Transactions. 11(2), 113 – 116.

• Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. 2002. Brain-computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767 – 791.

• Birbaumer N; Kübler A; Ghanayim N; Hinterberger T; Perelmouter J; Kaiser J; Iversen I; 2000. The thought translation device (TTD) for completely paralyzed patients. IEEE transactions on rehabilitation engineering; 8(2):190-3.

• http://www.pbs.org/wnet/brain/scanning/eeg.html• http://hothardware.com/Articles/OCZ-NIA-BrainComputer-Interface/?page=2• http://neurosky.com/menu/main/technology/product_summary/• http://www.emotiv.com/• http://www.bci-info.tugraz.at/Members/graimann/definitions• http://www.gtec.at• http://health.howstuffworks.com/brain2.htm• http://hyperphysics.phy-astr.gsu.edu/Hbase/biology/nervecell.html• http://health.howstuffworks.com/brain.htm• http://en.wikipedia.org/wiki/10-20_system_(EEG)• http://serendip.brynmawr.edu/bb/kinser/Structure1.html• http://thebrain.mcgill.ca/flash/i/i_06/i_06_cr/i_06_cr_mou/i_06_cr_mou.html• http://ohsonline.com/Articles/2008/01/Study-Womens-Skulls-Thicker-Mens-Wider-Might-Affect-Protection-

Design.aspx