Brain Computer Interface & It's Applications | NeuroSky Minwave | Raspberry Pi

A

PROJECT REPORT

ON

“Brain Computer Interface & its Applications”

Guide- Submitted by-

Mrs. Preeti Gupta Vishal Aditya

Assistant Professor (ASET) B.Tech (CSE, VIII Semester)

AMITY SCHOOL OF ENGINEERING & TECHNOLOGY

AMITY UNIVERSITY RAJASTHAN

ii

ABSTRACT

Brain Computer Interface is a leading technology in today’s world. In simple words, it

creates an interface between the brain and the machine, allowing the machine to get input from

human brain. This technology has been up in the airs since decades and a lot of work has already

been done, but a lot is also remaining. The main use of this technology is in the field of medical

science. Works are being done in the field of military and skill improvement domains as well.

This technology is growing day by day and is proving to be one of the most important

technologies in the current scenario.

This project shows how Brain Computer Interfaces (BCI) works and how it can be used

in certain applications. EEG (Electroencephalography) was used to read brain waves and then

those waves were analyzed to fetch fruitful results which are used in certain applications. The

applications which have been demonstrated in this project are Brain Controlled Game Simulator,

Brain Controlled Keyboard (for handicapped).

iii

D E C L A R A T I O N

I hereby declare that the project entitled “Brain Computer Interface & its

Applications” submitted for the partial fulfillment of B. Tech Degree is my

original work and the project has not formed the basis for the award of any degree,

associate ship, fellowship or any other similar titles.

Signature of the Student:

Vishal Aditya

Programme: B. Tech (CSE, VIII Semester)

Place: Jaipur

Date:

iv

ACKNOWLEDGEMENT

It is indeed a great pleasure and matter of immense satisfaction for us to express our deep

sense profound gratitude towards all the people who have helped, inspired us in our project

work.

First I would like to give my gratitude to Mrs. Preeti Gupta (Assistant Professor, ASET)

for the effort taken by her right from the selection of the project to its completion. She spent her

precious time whenever we were in need of guidance.

Moreover I would like to thank Prof. (Dr.) D.D.Shukla (Director, ASET) and Prof. (Dr.)

Tarun Kumar Sharma (Head Of Department, CSE/IT), who were always there whenever I

needed any support and were a constant source of inspiration for accomplishment of this project.

Vishal Aditya

v

Table Of Contents

Title page i

Abstract ii

Declaration iii

Acknowledgment iv

Table of Contents v

List of Figures vii

1 Introduction 1

1.1 Introduction 2

1.2 Aims & Objective 2

1.3 Brain Computer Interface 3

1.3.1 Invasive BCI 3

1.3.2 Non Invasive BCI 4

1.4 Electroencephalography (EEG) 5

2 Reading Brain Waves 7

2.1 Types Of Waves 8

2.1.1 Gamma Waves (31-50 Hz) 8

2.1.2 Beta Waves (13-30 Hz) 8

2.1.3 Alpha Waves (8-12 Hz) 8

2.1.4 Theta Waves (4-7 Hz) 8

2.1.5 Delta Waves (1-3 Hz) 9

2.2 Comparison Between EEG Bands 9

2.3 BCI System 10

2.3.1 Signal Acquisition 11

2.3.2 Feature Extraction 11

2.3.3. Feature Translation 11

vi

2.3.4 Output Device 11

3 Requirement & Feasibility Analysis 12

3.1 Requirement Analysis 13

3.1.1 Hardware Requirement 13

3.1.2 Software Requirement 13

3.2 Feasibility Analysis 13

3.2.1 Technical Feasibility 13

3.2.2 Economic Feasibility 14

4 Research Methodology 15

4.1 Literature Review 16

4.2 Objective 18

4.3 Algorithm 18

4.3.1 Attention 18

4.3.2 Meditation 18

4.3.3 Blink Detection 19

4.3.4 Mental Effort 19

4.3.5 Familiarity 19

4.3.6 Appreciation 19

4.3.7 Emotional Spectrum 20

4.3.8 Cognitive Preparedness 20

4.3.9 Creativity 20

4.3.10 Alertness 20

5 Working & Applications 21

5.1 EEG & Mindwave 22

5.1.1 The Origin 22

vii

5.1.2 EEG Signals 24

5.1.3 Clinical EEG 25

5.1.4 Emotiv Epoch EEG 26

5.1.5 NeuroSky Mindwave 27

5.2 How Does The Technology Work ? 30

5.3 Applications Of Our Project 31

5.4 How to Setup/Configure 31

5.4.1 Mind Controlled Robot 32

5.4.1.1 How to Do ? 33

5.4.1.2 What is RaspberryPi ? 33

5.4.1.3 Circuit Connection 34

5.4.1.4 Motor Direction Logic 35

5.4.1.5 Software Setup 35

5.4.2 Brain Keyboard 37

5.4.2.1 How to Setup/Configure 38

5.4.3 Data Visualization 38

5.4.3.1 Python 38

5.4.3.2 Open Vibe 39

5.4.3.3 MATLAB 41

5.4.3.4 Visual Studio 43

6 Conclusion 45

6.1 Conclusion 46

6.1.1 Medical Uses of EEG 46

6.1.2 EEG Technology & Research 47

6.1.3 EEG-Enabled Prosthetics 47

6.1.4 EEG-Enabled Communication Devices 48

viii

6.1.5 EEG & Education 49

6.1.6 EEG Entertainment 49

7 Future Scope 50

7.1 BCI – Advantages & Disadvantages 51

7.2 EEG – Future Scope 51

7.3 BCI – Future Scope 51

References 53

vii

List Of Figures

Sr. no. Figure No. Figure Title Page No

1 Figure 1.2.1 Invasive BCI 4

2 Figure 1.2.2 Non Invasive BCI 5

3 Figure 2.2 EEG Bands of Frequencies 9

4 Figure 2.3 BCI System 10

6 Figure 5.1 Brain Game Simulator Setup 22

6 Figure 5.1.1.1 Human Brain Lobes 23

7 Figure 5.1.1.2 Different layers of the brain cortex 23

8 Figure 5.1.2 Example of EEG Bands 24

9 Figure 5.1.3 Common Areas of Bipolar EEG Sensory 26

10 Figure 5.1.4 Emotiv Epoch EEG NeuroHeadset 26

11 Figure 5.1.5 Mindwave EEG NeuroHeadset 27

12 Figure 5.4 Brain Controlled Car Racing Game 32

13 Figure 5.4.1 Mind Controlled Robot 32

14 Figure 5.4.1.1 Mind Controlled Robot – Hardware Setup 33

15 Figure 5.4.1.2 RaspberryPi 33

16 Figure 5.4.1.3 Circuit Connection 34

17 Figure 5.4.1.4 Booting Raspbian/Noobs 35

18 Figure 5.4.1.5 Installing XRDP 36

19 Figure 5.4.2 Brain Keyboard 37

20 Figure 5.4.3.1 Screenshot of Plot in Python 39

21 Figure 5.4.3.2 Acquisition Server & Clients in Open Vibe 40

22 Figure 5.4.3.2.1 Running & Setting-up Acquisition Server 40

23 Figure 5.4.3.2.2 Acquisition Client & Signal Display Connection 41

24 Figure 5.4.3.3 NeuroSky Mindwave and MATLAB 41

24 Figure 5.4.3.4 Raw Value using Visual Studio 44

25 Figure 6.1.1 EEG Testing 46

25 Figure 6.1.4 Stephen Hawking 48

26 Figure 7.1 Future Scope (Robotic Assistance) 52

viii

27 Figure 7.2 Future Scope (Google Glass & BCI) 53

ASET

Brain Computer Interface & its Applications 1

Chapter 1

Introduction

ASET


1.1 INTRODUCTION

The project titled ‘Brain Computer Interface & its Applications’ is processed and designed

under multiple softwares, MATLAB and Visual Studio. Core programming languages used are

Python and C#. The project aims to deal with creating an interface between human brain and a

computer using Electroencephalography (EEG) technique, a technique to read brain waves using

sensors that read delta, alpha, beta, gamma & theta waves from the brain, amplifies it and gives

the output in form of numerical values to the machine. The users just need to think of movement

in order to drive the system. Several brain scanning techniques like fMRI, PET, or EEG are

capable to analyze what you are thinking, dreaming, or seeing.

EEG device that is used to read the brain waves is NeuroSky Mindwave Headset (MW001).

It comprises of a Wi-Fi dongle and two lobes, frontal lobe for forehead and another lobe that

connects to left ear. In all, the device can detect and give three values which can be used to

develop mind-controlled applications:

Raw Values

Attention Values

Meditation Values

Eye Blink Strength

1.2 AIMS AND OBJECTIVE

The core objectives which have been designated as fundamental to the project are:

• To acquire EEG signal from BCI technique using NeuroSky Mindwave Headset.

Read the brain waves using the EEG device.

• Filter those Raw Values and understand the basic functionalities of how those brain waves are

giving the particular output as per the reaction of the brain.

Using the raw values obtained gathered as a base, we then used it to fetch Attention, Meditation

and Eye Blink Strengths values.

. • Analyze the values and use them with certain applications to control them by mind, such as

Game Control, Keyboard Control for the disabled and show the use of BCI in Military.

After the Raw Values are fetched and analyzed, we now need to use them in our applications

enabling mind controlled applications.

ASET


1.3 BRAIN COMPUTER INTERFACE

A Brain Computer Interface (BCI) often called a Mind-Machine Interface (MMI) or

sometimes called a Direct Neural Interface (DNI), Synthetic Telepathy Interface (STI) or a Brain

Machine Interface (BMI) is a direct communication pathway between the brain and an external

device. BCIs are often directed at assisting, augmenting or repairing human cognitive or sensory-

motor functions.

Humans’ brain is filled with neurons, individual nerve cells connected to one another by

dendrites and axons. Every action like think, move, feel or remember something make neurons

are at work. That work is carried out by small electric signals that zip from neuron to neuron as

fast as 250 mph. The signals are generated based on the differences in electric potential carried

by ions on the membrane of each neuron. The signals then can be detected, interpreted to what

they mean and use them to direct a device of some purpose. Therefore, BCI is a system that

provides direct interface between the human brain and the computer. In order to develop the BCI

system, the feasible technique should be studied. BCI systems are broadly classified into invasive

and non-invasive techniques.

1.3.1 INVASIVE BCI:

Invasive BCI are Neuroprosthetics where electrode arrays heads are buried within the

brain during neurosurgery and left there on a permanent basis. Invasive devices produce the

highest quality signals of BCI device because they lie in the grey matter of brain. They have by

far the best signal to noise ratio and accuracy of any BCI method. Unfortunately invasive BCI is

costly and require complex surgery to implant. They are require a permanent hole in the skull,

build-up prone to scar-tissue, causing the signal to become weaker or even non-existent, as the

body reacts to a foreign object in the brain. Electrocorticography (ECoG) is one of the invasive

BCI. It also known as partially invasive as the device is implanted inside the skull but rest

outside the brain rather than within the grey matter. ECoG is a very promising intermediate BCI

modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency

range, and less training requirements than scalp-recorded EEG, and at the same time has lower

technical difficulty, lower clinical risk, and probably superior long-term stability than

intracortical single-neuron recording. This feature profile shows potential for real world

application for people with motor disabilities. Unfortunately ECoG is also costly and required

ASET


dangerous nature of surgeries for such system.

Figure 1.3.1 Invasive BCI

1.3.2 NON INVASIVE BCI:

Non-invasive BCI is the most popular technique where the electrodes need to be placed

outside of the skull or on the scalp. Non-invasive methods are limited in that they are often

susceptible to noise, have worse signal resolution due to distance from the brain, and have

difficulty recording the inner workings of the brain. However they have the advantages that can

combat these difficulties by lower cost, greater portability and the fact that they do not require

any special surgery. Most non-invasive BCI systems use electroencephalogram (EEG) signals.

EEG is the first non-invasive neuron imaging technique discovered which is used for measuring

the electrical activity of the brain. Besides electrical activity, neural activity also produces other

types of signals such as magnetic and metabolic that could be used in a BCI. Magnetic fields can

be recorded by using magnetoencephalography (MEG), while brain metabolic activity which is

reflected in changes in blood flow can be observed by using positron emission tomography

ASET


(PET), functional magnetic resonance imaging (fMRI), and optical imaging [1].

Unfortunately, such alternative techniques require sophisticated devices that can be operated only

in special facilities. Moreover, techniques for measuring blood flow have long latencies and thus

are less appropriate for interaction.

Figure 1.3.2 Non-Invasive BCI

1.4 ELECTROENCEPHALOGRAPHY (EEG):

EEG is the first non-invasive neuron imaging technique discovered which is used for

measuring the electrical activity of the brain. EEG signals are detected from the scalp and contain

noise as a result of electrical interference and movement of electrodes. Applying a large number of

EEG channels may include noisy and redundant signals that degrade the BCI performance and also

involve a prolonged preparation time that directly impacts the convenience in the use of the BCI.

Therefore, selecting the least number of channels that yield the best or required accuracy can

balance both needs for performance and convenience. Due to its ease of use, cost and high

temporal resolution this method is the most widely used one in BCIs nowadays. The advantages of

using EEG technique are:

a) Hardware costs are significantly lower than those of most other techniques.

ASET


b) EEG sensors can be used in more places than fMRI, SPECT, PET, MRS, or MEG, as these

techniques require bulky and immobile equipment. For example, MEG requires equipment

consisting of liquid helium-cooled detectors that can be used only in magnetically shielded rooms,

altogether costing upwards of several million dollars and fMRI requires the use of a 1-ton magnet

in, again, a shielded room.

c) EEG has very high temporal resolution, on the order of milliseconds rather than seconds. EEG is

commonly recorded at sampling rates between 250 and 2000 Hz in clinical and research settings,

but modern EEG data collection systems are capable of recording at sampling rates above 20,000

Hz if desired. MEG and EROS are the only other non-invasive cognitive neuroscience techniques

that acquire data at this level of temporal resolution.

d) EEG is silent, which allows for better study of the responses to auditory stimuli.

e) EEG does not aggravate claustrophobia, unlike fMRI, PET, MRS, SPECT, and sometimes MEG.

f) EEG does not involve exposure to high-intensity (>1 Tesla) magnetic fields, as in some of the

other techniques, especially MRI and MRS. These can cause a variety of undesirable issues with

the data, and also prohibit use of these techniques with participants that have metal implants in

their body, such as metal-containing pacemakers.

g) Extremely non-invasive, unlike ECoG which actually requires electrodes to be placed on the

surface of the brain.

The characteristics of EEG that compare favourably with behavioural testing:

a) EEG can detect covert processing (i.e., processing that does not require a response)

b) EEG can be used in subjects who are incapable of making a motor response.

c) EEG is a powerful tool for tracking brain changes during different phases of life. EEG sleep

analysis can indicate significant aspects of the timing of brain development, including evaluating

adolescent brain maturation.

d) In EEG there is a better understanding of what signal is measured as compared to other research

techniques, i.e. the BOLD response in MRI.

ASET


Chapter 2

Reading Brain Waves

ASET


2.1 TYPES OF WAVES:

You have around 86 billion neurons in your brain which control your whole body and generate

thoughts in your head. These neurons fire with certain frequencies. The EEG headset reads these

frequencies by reading the electrical activity that the neurons produce. Scientists have agreed to

split these frequencies in different frequency bands. All the frequency bands have been studied

intensively and are associated with several states of mind [2]:

Delta waves (1-3 Hz): Deepest meditation and deep sleep

Theta waves (4-7 Hz): Normal sleeping and normal meditation

Alpha waves (8-12 Hz): Relaxation/reflection

Beta waves (13-30 Hz): Active thinking, focus, hi alert, anxiety

Gamma waves (31-50 Hz): Conscious perception

2.1.1 GAMMA WAVES (31-50 Hz):

Too much: Anxiety, high arousal, stress

Too little: ADHD, depression, learning disabilities

Optimal: Binding senses, cognition, information processing, learning, perception, REM

sleep

2.1.2 BETA WAVES (13-30 Hz):

Too much: Adrenaline, anxiety, high arousal, inability to relax, stress

Too little: ADHD, daydreaming, depression, poor cognition

Optimal: Conscious focus, memory, problem solving

2.1.3 ALPHA WAVES (8-12 Hz):

Too much: Daydreaming, inability to focus, too relaxed

Too little: Anxiety, high stress, insomnia, OCD

Optimal: Relaxation

2.1.4 THETA WAVES (4-7 Hz):

Too much: ADHD, depression, hyperactivity, impulsivity, inattentiveness

Too little: Anxiety, poor emotional awareness, stress

ASET


Optimal: Creativity, emotional connection, intuition, relaxation

2.1.5 DELTA WAVES (1-3 Hz):

Too much: Brain injuries, learning problems, inability to think, severe ADHD

Too little: Inability to rejuvenate body, inability to revitalize the brain, poor sleep

Optimal: Immune system, natural healing, restorative / deep sleep

2.2 COMPARISON BETWEEN EEG BANDS:

The following is a comparison between the different bands of frequencies of Alpha, Beta

Gamma, Theta and Delta waves:

Figure 2.2 EEG Bands of Frequencies

ASET


All the frequencies are further divided as follows:

Delta: 1-3Hz

Theta: 4-7Hz

Alpha1: 8-9Hz

Alpha2: 10-12Hz

Beta1: 13-17Hz

Beta2: 18-30Hz

Gamma1: 31-40Hz

Gamma2: 41-50Hz

2.3 BCI SYSTEM:

A BCI system consists of three components: Signal or Data Acquisition, Signal Processing

(Feature Extraction, Feature Translation), and Output Device. These components are controlled by

a protocol which defines the timing for operation, signal processing details, nature of device

commands and the performance.

Figure 2.3 BCI System

ASET


2.3.1 SIGNAL ACQUISITION:

Signal acquisition in a BCI helps in the measurement of brain signals using a sensor

modality. The sensor is basically a device implanted in the brain usually multi-electrode arrays that

records the signals directly related to the movement. The signals can be amplified to levels suitable

for electronic processing. Also, they can be subjected to filtering to remove electrical noise or other

undesirable signals. After amplification and filtering process, the signals can be digitized and

transmitted to a computer.

2.3.3 FEATURE EXTRACTION:

Feature extraction in Brain Computer Interface (BCI) is the process of analyzing the digital

signals to distinguish signal characteristics and represent them in a compact form suitable for

translation into output commands. These features been extracted should have good correlations

with the users intent.

2.3.3 FEATURE TRANSLATION:

Resulting signal features are passed to the feature translation algorithm, which converts the

features into the commands for the output device (i.e., commands that accomplish the users need).

2.3.4 OUTPUT DEVICE:

The commands from the feature translation algorithm operate the external device of the

Brain Computer Interface (BCI), providing functions such as cursor control, letter selection, robotic

arm operation etc. The device operation then provides feedback to the user finally, thus completing

the closed loop of Brain Computer Interface (BCI).

ASET


Chapter 3 Requirement & Feasibility

Analysis

ASET


3.1 REQUIREMENT ANALYSIS

The requirement gathering process is intensified and focused specifically on input. To

understand the nature of the project to be built, the researcher (“analyst”) must understand the

information domain for the research, as well as required function, behaviour, performance and

interface, Requirement for both the system and the research project are documented and reviewed

with the expertise to bias and due to variance [3].

3.1.1 Hardware Requirement

Processor: Pentium 4 1.6GHz

Memory: 1GB

Hard disk: 200MB

Bluetooth: Bundled USB dongle

Keyboard

Mouse

3.1.2 Software Requirement

Operating System: Windows XP, Vista, or later.

Documentation: Any Office Products

Softwares: MATLAB, Visual Studio

Languages: Python, C#.

3.2 FEASIBILITY ANALYSIS

All projects are feasible given unlimited resources and infinite time. Unfortunately the

development of a computer-based system is more likely to be plagued by scarcity of resources and

difficult delivery dates. It is both necessary and prudent to evaluate the feasibility of a project at

the earliest possible time. We concentrate our attention on two primary areas of interest.

3.2.1 Technical Feasibility

A study of function, performance and constraints may affect the ability to achieve an

acceptable system. This is the most difficult area to access at this stage of the system development

process. The consideration that are normally attached with this are:

1. Development risk

2. Resource availability

3. Technology

During technical analysis the analyst evaluates technical merits of the system concept, while

ASET


at the same time collecting information about performance, reliability, maintainability and

productibility.

3.2.2 Economic Feasibility

An evaluation of development cost weight against the ultimate income or benefit derived

from the development system. Development of this Research project is economically feasible and

is achievable within the cost limit.

ASET


Chapter 4

Research Methodology

ASET


4.1 LITERATURE REVIEW

Research work on Brain Computer Interface (BCI) began early in the year 1970 itself. This

research work was started at the University of California Los Angeles (UCLA) under the

permission from the National Science Foundation after a contract from DARPA. Thanks to the

development of brain imaging technologies like EEG, ECOG fMRI, MEG etc. This is because

they were one of the major milestones which helped the research workers to continue their work

on BCI. At first, BCI work was carried on many animals and today it is tested on human beings

too. Today, one of the major goal of BCI research is to develop applications that helps disabled

people (suffering from brainstem stroke, spinal cord injury, blindness etc.) to communicate and

interact with people and external environments.

Recently, brain-computer interfaces (BCIs) based on visual evoked potentials (VEPs) have

been shown to achieve remarkable communication speeds. As they use electroencephalography

(EEG) as non-invasive method for recording neural signals, the application of gel-based EEG is

time-consuming and cumbersome. In order to achieve a more user-friendly system, this work

explores the usability of dry EEG electrodes with a VEP-based BCI. While the results show a high

variability between subjects, they also show that communication speeds of more than 100 bit/min

are possible using dry EEG electrodes. To reduce performance variability and deal with the lower

signal-to-noise ratio of the dry EEG electrodes, an averaging method and a dynamic stopping

method were introduced to the BCI system. Those changes were shown to improve performance

significantly, leading to an average classification accuracy of 76% with an average

communication speed of 46 bit/min, which is equivalent to a writing speed of 8.8 error-free letters

per minute. Although the BCI system works substantially better with gel-based EEG, dry EEG

electrodes are more user-friendly and still allow high-speed BCI communication.

A Brain-Computer Interface (BCI) is a device that allows to control a computer by brain

activity only, without the need for muscle control. Its main purpose is to restore or improve

communication for paralysed patients. While there are different kinds of BCI that either rely on

motor imagery or the detection of event-related potentials like the P300, BCI systems based on

visual evoked potentials (VEPs) are currently the fastest method to establish non-invasive BCI

control. Using code-modulated visual evoked potentials (c-VEPs), it was shown that participants

ASET


could spell more than 20 error-free characters per minute (> 140 bit/min). When using steady state

visual evoked potentials (SSVEPs), the communication speed can further be improved by using

frequency and phase information of the SSVEP to reach communication speeds up to 60

characters per minute (> 300 bit/min).

While these results show that VEP-based BCIs allow for high-speed communication in a

lab environment, there are only few efforts to transfer those systems out of the lab and make them

useable for a broader audience. One of those few approaches is the Tübingen c-VEP BCI, which

was previously shown to reach spelling speeds of more than 20 error-free letters per minute and in

another work was improved to control mouse and keyboard of the Windows®operating system,

thereby enabling a user to completely control a computer and use arbitrary applications by means

of his brain activity only. While this is an important step towards improved useability of a BCI

system, the use of gel-based EEG devices still limits the applicability, as gel-based EEG is time-

consuming to apply, cannot be applied by the user itself, and “few people are ready to wash their

hair each time they wish to use Word or a LaTeX editor”.

To solve this problem, dry EEG electrodes have been developed that are quick to setup and

allow to measure EEG without the need for conductive gel. While there are a variety of different

dry electrodes being developed, there are also commercially available dry EEG electrodes that

deliver good results and have been shown to work for BCI applications using P300, SSVEP and

motor imagery. Further, there are consumer-grade EEG headsets with dry electrodes or easy to

apply wet electrodes that don’t need the application of conductive gel like the Emotiv EPOC.

While such systems have also been used for BCI applications, the signal quality of such EEG

hardware is not comparable to professional devices and their use is not recommended for serious

applications.

A good comparison of different dry EEG electrodes developed and tested can be found in

Baek et al. who review and discuss several publications. They conclude that dry electrodes allow

the acquisition of spontaneous EEG signals with lower signal quality than gel-based EEG

electrodes and that movement artefacts are a larger problem in dry electrodes. Nevertheless, signal

quality is good enough to record P300 and SSVEP response to use them for BCI applications.

While dry electrodes haven’t been tested with a c-VEP BCI system so far, the aim of this study

was to test if commercially available medical-grade dry EEG electrodes can be used with a high-

speed c-VEP BCI and what algorithmic improvements of the system are necessary to deal with the

increased signal-to-noise ratio of dry EEG electrodes [4.1].

ASET


4.2 OBJECTIVE

The objective of this assignment is to extract and then analyze the Raw Values obtained by

EEG.

4.3 ALGORITHM

NeuroSky algorithms provide the foundation of a universe of applications that can be built to

optimize brain health, education, alertness and overall function. General algorithm is as follows:

1. First, for the raw EEG signals, first you need to apply pre-processing tasks like feature

extraction. Various methods are available for the same like VLC methods, PCA or wavelets

etc

2. Then apply classification methods over the transformed data.

3. Generally if you apply classification algorithms directly on the raw data, then it does not

yield good accuracy and good models.

NeuroSky provides inbuilt algorithms that are used to record certain types of activities [4.2].

4.3.1 Attention:

The Attention Meter algorithm indicates the intensity of mental “focus” or “attention.” The

value ranges from 0 to 100. The attention level increases when a user focuses on a single thought or

an external object, and decreases when distracted. Users can observe their ability to concentrate

using the algorithm. In educational settings, attention to lesson plans can be tracked to measure

their effectiveness in engaging students. In gaming, attention has been used to create “push”

control over virtual objects.

4.3.2 Meditation:

The Meditation Meter algorithm indicates the level of mental “calmness” or “relaxation.”

The value ranges from 0 to 100, and increases when users relax the mind and decreases when they

are uneasy or stressed. The Meditation Meter quantifies the ability to find an inner state of

mindfulness, and can thus help users learn how to self correct and find inner balance to overcome

the stresses of everyday life. The algorithm is also used in a variety of game-design controls.

ASET


4.3.3 Blink Detection:

The Blink Detection algorithm signals a user’s blinks. A higher number indicates a

“stronger” blink, while a smaller number indicates a “lighter” or “weaker” blink. The frequency of

blinking is often correlated with nervousness or fatigue. Eye blinks are akin to a standard on/off

binary system and therefore are valuable for controls that require definitive responses. For instance,

in communication applications, one blink means no, two mean yes — giving individuals with a

special needs a simple way to communicate [5].

4.3.4 Mental Effort:

The Mental Effort algorithm measures the mental workload while performing a task. The

harder a user’s brain works on a task, the higher the value. The algorithm works well with both

physical (e.g., drawing) and mental (e.g., reciting) tasks, and can be used for continuous real-time

tracking and between-trial comparisons to measure the effects of multi-tasking, workload

variability, and more. The algorithm can be used to track the effects of diverse cognitive loads on

the ability to learn and provide feedback for user self-improvement.

4.3.5 Familiarity:

The Familiarity algorithm tracks learning processes to measure the relative level of

understanding, learning, or comfort with a task. It is a measure of the subconscious learning of

procedural (motor) and mental tasks. In some cases, it reflects how well a user is doing with the

task. By observing trends, users can better understand and assess their learning process. It can be

applied to tasks that are physical in nature (e.g., drawing) or mental (e.g., recitation), and enable

data-tracking assessments to gauge learning status.

4.3.6 Appreciation:

The Appreciation algorithm provides real-time measurement of the level of enjoyment or

appreciation a subject feels towards an external audiovisual stimulus. The algorithm allows

moment-by-moment detection of emotions. In marketing applications, it can be used to track and

understand a users level of appreciation, providing insights and strategic direction for marketing

efforts.

ASET


4.3.7 Emotional Spectrum:

Emotional spectrum measures the intensity (low-high) and pleasantness (pleasant/unpleasant) of

emotional activity.

Pleasant/Unpleasant identifies whether a user is having a pleasant or unpleasant emotion. Pleasant

emotions could be happy, serene, relaxed, etc. Unpleasant emotions could be anger, disgust,

depressed, etc.

Intensity measures how strong a user’s emotions are in real time. The emotion could be happy,

angry, calm, or any other emotion. 4 eTensity levels Very Weak – Weak – Strong – Very Strong.

4.3.8 Cognitive Preparedness:

Cognitive Preparedness measures the brain’s capacity for optimal cognitive performance on

a relatively complex task or in other words, the brain’s capacity for higher level cognitive

functions.

4.3.9 Creativity:

Creativity measures the brainwave activities underlying creative cognition. The higher the

Creativity value, the stronger the brainwave activities promoting innovative and creative thinking.

4.3.10 Alertness:

Alertness measures user’s alertness or vigilance level at the moment. High alertness value

indicates you are at a state of focus while low value represents a relaxing state of mind.


Chapter 5

Working & Applications


5.1 EEG – ELECTROENCEPHALOGRAPHY AND MINDWAVE

Figure 5.1: Brain Game Simulator Setup

Our brain has always been fascinating and a lot of researchers are still trying to explore what we

can do if we can capture some meaningful data from brain. It's a simple approach used by

NeuroSky to capture raw data (frequency) that can further be analyzed for various applications.

The working principle of our project is based on earlier discussed eSense Algorithms for

capturing Attention, Meditation & Eye Blink Strengths.

5.1.1 The Origins

EEG or electroencephalography are since Hans Berger in 1929 exposed that the activity

of the brain could be measured from electrodes situated in the human skull. With EEG we could

measure in fact, the functional state of the brain and diagnose some future or actual problems.

This is the most common way to measure injuries in the brain and functional brain disturbances,

but the creation of the signal is not well understood. Different regions of the cortex have

different cytoarchitectures and each region has its own morphological patterns, aspects of


intrinsic organization of the cortex are general. Most of the cortical cells are arranged in the

form of columns, in which the neurons are distributed with the main axes of the dendritic trees

parallel to each other and perpendicular to the cortical surface. This radial orientation is an

important condition for the appearance of powerful dipoles. Figures below lists the parts of

human brain cortex and zones of interest.. These layers are places of specialized cell structures

and within places of different functions and different behaviors in electrical response. An area of

very high activity is, for example, layer IV, which neurons function to distribute information

locally to neurons located in the more superficial (or deeper) layers. Neurons in the superficial

layers receive information from other regions of the cortex. Neurons in layers II, III, V, and VI

serve to output the information from the cortex to deeper structures of the brain [6].

Figure 5.1.1.1: Lobes of the human brain, external cerebrum, midbrain areas such as the dience-

phalon and the hindbrain areas such as the cerebellum, medulla, etc.

Figure 5.1.1.2: Different layers (columns) of the brain cortex.

http://cerescontrols.com/wp-content/uploads/2014/06/brain.png


Pyramidal cells in layers III and V are mainly responsible for the generation of the EEG.

5.1.2 EEG Signals

The EEG signal consists of spontaneous potential fluctuations that also appear without a

sensory input. It seems to be a stochastic signal, but it is also composed of quasi- sinusoidal

rhythms. The synchrony of cerebral rhythms may occur from pacemaker centers in deeper

cortical layers like the thalamus or in sub cortical regions, acting through diffuse synaptic

linkages, reverberatory circuits incorporating axonal pathways with extensive ramifications, or

electrical coupling of neuronal elements. The range of amplitudes is normally from 10mV to

150mV, when recorded from electrodes attached to the scalp. The EEG signal consists of a

clinical relevant frequency range of 0.5–50 Hz (10).

The most common frequency bands of EEG are the most common way of analysis. This

information can reveal physiological and statistical evidence but each band could vary on people

and animals with is behaviors and metal sanity, age, etc. The most important patterns of human

EEG are described below [7].

Figure 5.1.2: Example of EEG Bands


The phenomena of alpha de-synchronization channel could be used to get the eyes closed/open

detection.

Delta Waves

The appearance of delta waves are common in neonatal and infant EEGs and during in sleep

stages in adult EEGs. If delta EEGs appears by itself in a adult it means cerebral injury

Theta Waves

In the beginning where part of the delta waves, but scientists discovered the importance activity

of these waves. Its region of interaction is between thalamic region and play dominant part in

childhood and infancy. The normal adult waking of theta waves are a few or small amount of

these frequencies observed in drowsiness and sleep. Large amount of theta waves are associated

between different amount of pathologies.

Alpha Waves

These are originated on the posterior half back of the head and are from occipital an parietal

regions. These waves are observed during conditions of awakeness, physical relaxation and

mental inactivity can be blocked by mental activity or an influx of light when eyes are opened.

Beta Waves

Are presented in a healthy adult and the area of formation are in the frontal and central region of

the cortex. Typical voltage of beta waves are less than 30uV. Beta activity increase when the

organism is added with barbiturates, some non barbiturates sedatives and minor tranquilizers. It

also appears during mental activity and tension.

5.1.3 Clinical EEG

The most common EEG uses up to 30 landmarks on the skull using bipolar derivation (two

electrodes on the skull and the difference is the gradient of potential). Unipolar derivation is

done with an electrode or group of electrodes with the active part (activity) and the inactive part

(usually nose or ear). The advantage of unipolar derivation are that the amplitude of each

deflection is proportional to the magnitude of the potential change that causes it and the


demonstration of small time differences between the occurrence of a widespread discharge at

several electrode.

Figure 5.1.3: Common areas of bipolar EEG sensory.

5.1.4 Emotiv Epoch EEG

Based on the latest developments in neuro-technology, Emotiv presents a revolutionary

personal interface for human computer interaction. Emotiv EPOC is a high resolution, multi-

channel, wireless NeuroHeadset. The EPOC uses a set of 14 sensors plus 2 references to tune

into electric signals produced by the brain to detect the user’s thoughts, feelings and expressions

in real time. The EPOC connects wirelessly to PCs running Windows or MAC OS X.

Experience the fantasy of controlling and influencing the virtual environment with your mind.

Access applications and play games developed specifically for the EPOC, or use the EmoKey to

connect to current PC games and experience them in a completely new way [8].

If you or any of your 3rd party applications require access to raw EEG data, you will need to

purchase the Emotiv EEG NeuroHeadset.

Figure 5.1.4: Emotiv Epoch EEG NeuroHeadset

http://cerescontrols.com/wp-content/uploads/2014/06/eeg_areas.png

http://cerescontrols.com/wp-content/uploads/2014/06/emotiv.jpg


5.1.5 NeuroSky Mindwave

Measuring Electroencephalogram (EEG) activity has historically required complex, intimidating

and immovable equipment costing thousands of dollars. NeuroSky is unlocking a new world of

solutions for education and entertainment with our research-grade, mobile, embeddable EEG

biosensor solutions. Precisely accurate, portable and noise-filtering, our EEG biosensors translate

brain activity into action.

Our EEG solution digitizes analog electrical brainwaves to power the user-interface of games,

education and research applications. We amplify and process raw brain signals to deliver concise

input to the device. Our brainwave algorithms, developed by NeuroSky researcher and our

partner universities and research institutions are uncovering new ways to interact with our world.

Figure 5.1.5: Mindwave EEG NeuroHeadset

Both EEGs are good. NeuroSky Mindwave is cheap, easy to hack and useful for develop simple

applications and filters for study brainwave signals. One thing to note is that NeuroSky EEG is

only for develop games and must not be used to study the signals as a medical device, this is not

the goal of this company. The cost of the NeuroSky Mindwave is about 100 USD.

Emotiv EEG is good at a clinical level because gives you a lot of information of regions from F1

to F15, has an gyroscope to sense orientation of the head and a SDK for developer or researcher

in Linux and Windows. The cost of the EEG rounds about 300 USD. But if you need to get raw

http://cerescontrols.com/wp-content/uploads/2014/06/neurosky.jpg


data for better analysis you need to buy the complete package of EEG NeuroHeadset, and

software, it’s around 750 USD.

There are two frames that are outputted at variable rates by the device:

AA 04 80 02 00 56 27 AA AA 04 80 02 00 53 2A AA AA 20 02 38 83 18 02 43 EA 00 03 90 00

00 89 00 00 47 00 00 1E 00 00 28 00 00 3B 00 00 27 04 00 05 00 E7 AA AA 04 80 02 00 53 2A

AA AA 04 80 02 00 55 28 AA AA 04 80 02 00 54 29 AA AA 04 80 02 00 54 29 AA

Purple frame is outputted every 512 Hz and is not exchangeable the frequency time of output.

Green frame is outputted every 1 Hz and is not exchangeable the frequency time of output.

The frames contain useful information about raw values and calculated values. The below table

lists the different frames that are output of every frame.

For the 512 Hz frame the information output is:

byte: value // Explanation

[ 0]: 0xAA // [SYNC]

[ 1]: 0xAA // [SYNC]

[ 2]: 0x04 // [PLENGTH] (payload length) of 8 bytes

[ 3]: 0x80 // [RAW_WAVE_VALUE] 16-bit two's-compliment signed value (high-order byte

followed by low-order byte) (-32768 to 32767)

[ 4]: 0x02 // [VLENGHT] (payload variable length) of 'n' bytes

[ 5]: 0x00 // [RAW_HIGH] high order byte of raw data two's compliment signed value

[ 6]: 0x53 // [RAW_LOW] low order byte of raw data two's compliment signed value

[ 7]: 0x2A // [CHKSUM] (1's comp inverse of 8-bit Payload sum)


For the 1 Hz frame the information output is:

byte: value // Explanation

[ 0]: 0xAA // [SYNC]

[ 1]: 0xAA // [SYNC]

[ 2]: 0x20 // [PLENGTH] (payload length) of 32 bytes

[ 3]: 0x02 // [POOR_SIGNAL_QUALITY] (0 to 255)

[ 4]: 0x38 // 56 of 255

[ 5]: 0x83 // [ASIC_EEG_POWER] eight big-endian 3-byte unsigned integer values

representing delta, theta, low-alpha, high-alpha, low-beta, high-beta, low-gamma, and mid-

gamma EEG band power values

[ 6]: 0x18 // upper byte of EEG_POWER_DELTA

[ 7]: 0x02 // middle byte of EEG_POWER_DELTA

[ 8]: 0x43 // lower byte of EEG_POWER_DELTA

[ 9]: 0xEA // upper byte of EEG_POWER_THETA

[10]: 0x00 // middle byte of EEG_POWER_THETA

[11]: 0x03 // lower byte of EEG_POWER_THETA

[12]: 0x90 // upper byte of EEG_POWER_LOW_ALPHA

[13]: 0x00 // middle byte of EEG_POWER_LOW_ALPHA

[14]: 0x00 // lower byte of EEG_POWER_LOW_ALPHA

[15]: 0x89 // upper byte of EEG_POWER_HIGH_ALPHA

[16]: 0x00 // middle byte of EEG_POWER_HIGH_ALPHA

[17]: 0x00 // lower byte of EEG_POWER_HIGH_ALPHA


[18]: 0x47 // upper byte of EEG_POWER_LOW_BETA

[19]: 0x00 // middle byte of EEG_POWER_LOW_BETA

[20]: 0x00 // lower byte of EEG_POWER_LOW_BETA

[21]: 0x1E // upper byte of EEG_POWER_HIGH_BETA

[22]: 0x00 // middle byte of EEG_POWER_HIGH_BETA

[23]: 0x00 // lower byte of EEG_POWER_HIGH_BETA

[24]: 0x28 // upper byte of EEG_POWER_LOW_GAMMA

[25]: 0x00 // middle byte of EEG_POWER_LOW_GAMMA

[26]: 0x3B // lower byte of EEG_POWER_LOW_GAMMA

[27]: 0x00 // upper byte of EEG_POWER_MID_GAMMA

[28]: 0x00 // middle byte of EEG_POWER_MID_GAMMA

[29]: 0x27 // lower byte of EEG_POWER_MID_GAMMA

[31]: 0x04 // [ATTENTION] eSense (0 to 100)

[32]: 0x00 // Attention level

[33]: 0x05 // [MEDITATION] eSense (0 to 100)

[34]: 0x00 // Meditation level

[35]: 0xE7 // [CHKSUM] (1's comp inverse of 8-bit Payload sum)

5.2 HOW DOES THE TECHNOLOGY WORK ?

Brainwaves are tiny electrical impulses released when a neuron fires in the brain.

NeuroSky’s brain-computer interface (BCI) technology works by monitoring these electrical

impulses with a forehead sensor. The neural signals are input into our ThinkGear chip, and

interpreted with our patented Attention and Meditation algorithms. The measured electrical

signals and calculated interpretations are then output as digital messages to the computer, toy, or


mobile device, allowing you to see your brainwaves on the screen, or use your brainwaves to

affect the device’s behavior.

5.3 APPLICATIONS OF OUR PROJECT:

Brain Game Simulator: We are trying to automate the windows games first racing

games controls using attention & meditation level values that we have captured from the frontal

lobe of brain using NeuroSky Mindwave Headset.

# Python Direct X game controller server

# DirectX works with Scancodes for Direct Input

5.4 HOW TO SETUP/CONFIGURE?

Install python2.7 64-bit/32-bit (recommended) or later.

Install dependency packages for python

o PyWin32: To automate windows keys using sendkeys & directX programming

o Mindwave: To interface with ThinkGear socket connection & capture data from

NeuroSky Mindwave Headset.

Check your port number from Device Manager & Edit the line from

“braingamesimulator.py” file:

headset = mindwave.Headset('COM13', 'CC0E')

Run file “braingamesimulator.py”

Calibration for individual person can be done by changing threshold values of if-elif

conditions.

Start any racing game preferred Asphalt8.


Figure 5.4: A brain controlled car racing game

5.4.1 Mind Controlled Robot:

We are trying to control a robot using BCI through the threshold values of attention &

meditation. Further, the data is sent to Raspberry Pi micro-controller through socket connection

over Wireless network & GPIO triggers are sent to drive motors of Robot.

Figure 5.4.1: Mind Controlled Robot


5.4.1.1 How to do?

Hardware Setup:

Figure 5.4.1.1: Mind Controlled Robot – Hardware Setup

5.4.1.2 What is RaspberryPi?

Figure 5.4.1.2: Raspberry Pi


The Raspberry Pi is a low cost, credit-card sized computer that plugs into a computer monitor

or TV, and uses a standard keyboard and mouse. It is a capable little device that enables people

of all ages to explore computing, and to learn how to program in languages like Scratch and

Python. It’s capable of doing everything you’d expect a desktop computer to do, from browsing

the internet and playing high-definition video, to making spreadsheets, word-processing, and

playing games.

Using Python for RaspberryPi.

Python is a wonderful and powerful programming language that's

easy to use (easy to read and write) and with Raspberry Pi lets to

connect projects to the real world. Python syntax is very clean,

with an emphasis on readability and uses standard English

keywords. Start by opening IDLE from the desktop. The easiest

introduction to Python is through IDLE, a Python development environment.

5.4.1.3 Circuit Connection with MOTOR Driver L298n Dual H-Bridge

Figure 5.4.1.3: Circuit Connection


5.4.1.4 Motor Direction Logic

5.4.1.5 Software Setup:

Boot Raspbian/Noobs in Raspberry Pi.

Figure 5.4.1.4: Booting Raspbian/Noobs


Install XRDP to enable remote desktop connection.

Figure 5.4.1.5: Installing XRDP

Install Dependencies: Python2.7, Mindwave, GPIO, PyQt4.

Start Hotspot from laptop using Connectify or CMD.(for wireless communication)

Connect to Remote Desktop Connection with the IP scanned for wlan0 of RPi.

Run the program “robotcontrol.py” to start connecting to NeuroSky Headset & send

signals to drive motors of Robot.

Calibration: “Try to focus on Robot & increase your attention levels for FORWARD

control & distract yourself to increase your meditation level for REVERSE control.


5.4.2 Brain Keyboard:

This application is an attempt to develop a keyboard interface for physically

handicapped: Lost his/her hand & they can type using virtual keyboard through eye blinks.

Figure 5.4.2: Brain Keyboard

People who have severe movement disorders like Amyotrophic Lateral

Sclerosis (ALS) or Locked in Syndrome, is a condition where a patient is awake and

aware of its surrounding but unable to communicate or perform any action due to

paralysis of almost every voluntary muscles in the body (with the exception of eye

movements and blinking). The people who are suffering from severe Cerebral Palsy

disorder are not able to speak or not understandable enough so they could not

communicate and interact with others.

These people also have severe movement disorder.Imagine these patients having a

fully functional brain trapped within a non-functioning body. The brain of the

patient would be fully conscious and aware of its surroundings, it could think and

process stimuli, but unable to translate thought into action. This is caused by a

gradual degeneration of the nerve cells in the central nervous system that controls

voluntary muscle movement. The application has 26 alphabetical keys with a space

and a delete key. Here, a square selection box scans all the keys one by one i n a

sequence. There is a time delay of about 100ms to move the square selection box

from one key to another key. The application takes the input when user blinks his


eye. The eye blinks triggers the current selected key (as shown in the picture) and

the letter gets print on the text box

5.4.2.1 How to setup/configure?

Install python2.7 64-bit/32-bit (recommended) or later.

Install dependency packages for python

o PyWin32: To automate windows keys using sendkeys & directX programming

o Mindwave: To interface with ThinkGear socket connection & capture data from

NeuroSky Mindwave Headset.

o Check your port number from Device Manager & Edit the line from

brainkeyboard.py:

headset = mindwave.Headset('COM13', 'CC0E')

Run & start training your mind & eye blink strength for better controls

5.4.3. Data Visualization:

Capture & Plot RAW Data from NeuroSky Mindwave Headset

Our brain has always been fascinating & lot of researchers are still trying to explore what we can

do if we can capture some meaningful data from brain. It's a simple approach used by NeuroSky

to capture raw data (frequency) that can further be analyzed for various applications.

5.4.3.1 Python:

Dependencies

Python2.7 or later version

Packages: PyQtGraph, Matplotlib, PyQt4

Drivers: ThinkGear Connector

http://www.pyqtgraph.org/

http://developer.neurosky.com/docs/doku.php?id=thinkgear_connector_tgc


GitHub: https://github.com/vsltech/neuroskymindwavecapture/

Figure 5.4.3.1: Screenshot of Plot in Python

5.4.3.2 OpenVibe:

The purpose of OpenViBE is to get data from the acquisition device through the Acquisition

Server and then send it to one or more clients. This client is usually, for now, the OpenViBE

designer. The Acquisition Server and the clients (Designers) can be either on the same machine

or different machines on the same network, or any combination of these.

The following diagram explains the possibilities:

https://github.com/vsltech/neuroskymindwavecapture/

https://2.bp.blogspot.com/-ji_xnLImlB8/WO_afZY46yI/AAAAAAAAGuk/eA7rDeTgK6kpwhknnbBvpI81KIyOnJwNgCLcB/s1600/Screenshot2.png


Figure 5.4.3.2: Acquisition Server and Clients in OpenVibe

Procedure

Run and setup the Acquisition Server

Figure 5.4.3.2.1: Running and setting up the acquisition server in OpenVibe

Run the designer and create your first scenario

Now search for the Signal Display box and drag it into the scenario. This box displays the signal

it gets on the input. Now connect the two boxes. Click on the signal output (pink arrow) on the

Acquisition Client box and drag a connector to the matrix input (green arrow) of the Signal

Display box.

http://openvibe.inria.fr/openvibe/wp-content/uploads/2012/06/connections.png

http://openvibe.inria.fr/openvibe/wp-content/uploads/2012/06/acquisition-server.png


Figure 5.4.3.2.2: Connecting acquisition client with Signal Display

5.4.3.3 MATLAB

Figure 5.4.3.3: NeuroSky Mindwave and MATLAB

MATLAB is a part of the code decode Attention value calculated by e-sense algorithm from

ThinkGear packets

Matlab Code to Read Eyeblink using Mindwave: This part of the code decode eye blink

value calculated by e-sense algorithm from ThinkGear packets.

http://openvibe.inria.fr/openvibe/wp-content/uploads/2012/06/basic-connection.png


Matlab code to read Meditation using Mindwave: This part of the code decode

meditation value calculated by e-sense algorithm from ThinkGear packets.

Mind-Attention Control LED using Arduino.

Blinking led using EEG.

Interfacing Mindwave with Arduino.

MATLAB Code to read attention values [9]:

Matlab code to read attention using Mindwave: TG_DATA_ATTENTION = 2;

%Clear Screen clc;

%Clear Variables clear all;

%Close figures close all;

%Preallocate buffer data_att = zeros(1,256);

%Comport Selection portnum1 = 7;

%COM Port # comPortName1 = sprintf('\\\\.\\COM%d', portnum1);

% Baud rate for use with TG_Connect() and TG_SetBaudrate(). TG_BAUD_115200 = 115200;

% Data format for use with TG_Connect() and TG_SetDataFormat(). TG_STREAM_PACKETS = 0;

% Data type that can be requested from TG_GetValue(). TG_DATA_ATTENTION = 2;

%load thinkgear dll loadlibrary('Thinkgear.dll');

%To display in Command Window fprintf('Thinkgear.dll loaded\n'); %get dll version dllVersion = calllib('Thinkgear', 'TG_GetDriverVersion');

%To display in command window fprintf('ThinkGear DLL version: %d\n', dllVersion );

% Get a connection ID handle to ThinkGear connectionId1 = calllib('Thinkgear', 'TG_GetNewConnectionId');


if ( connectionId1 < 0 ) error( sprintf( 'ERROR: TG_GetNewConnectionId() returned %d.\n', connectionId1 ) ); end;

% Attempt to connect the connection ID handle to serial port "COM3" errCode = calllib('Thinkgear', 'TG_Connect', connectionId1,comPortName1,TG_BAUD_115200, TG_STREAM_PACKETS ); if ( errCode < 0 ) error( sprintf( 'ERROR: TG_Connect() returned %d.\n', errCode ) ); end fprintf( 'Connected. Reading Packets...\n' ); i=0; j=0;

%To display in Command Window disp('Reading Brainwaves'); figure; while i < 20 if (calllib('Thinkgear','TG_ReadPackets',connectionId1,1) == 1) %if a packet was read... if (calllib('Thinkgear','TG_GetValueStatus',connectionId1,TG_DATA_ATTENTION ) ~= 0) j = j + 1; i = i + 1;

%Read attention Valus from thinkgear packets data_att(j) = calllib('Thinkgear','TG_GetValue',connectionId1,TG_DATA_ATTENTION );

%To display in Command Window disp(data_att(j));

%Plot Graph plot(data_att); title('Attention');

%Delay to display graph pause(1); end end end

%To display in Command Window disp('Loop Completed')

%Release the comm port calllib('Thinkgear', 'TG_FreeConnection', connectionId1 );

5.4.3.4 Visual Studio

Similarly, there is the visual studio code as well, which is coded in C#. The same steps

are followed as above but in C#:


Figure 5.4.3.4: Getting Raw Values using C# in Visual Studio

ASET


Chapter 6 Conclusion

ASET


6.1 CONCLUSION:

As we have seen the working of BCI using EEG, it can be used in several modern

applications as follows:

6.1.1 Medical Uses of EEG:

EEG testing is used to diagnose and evaluate various neurological conditions, such as

epilepsy. The monitoring of a patient’s brain waves can give their doctor insight into any

abnormalities in brain function. Other conditions which may be tested for using EEG include

Alzheimer’s disease and narcolepsy.

Figure 6.1.1 EEG Testing

EEG is also used to monitor brain activity when a patient is in a coma. If the coma is natural and

the patient is on complete life support, EEG can detect brain death. If a patient is in a medically

induced coma, EEG is used to determine the appropriate amount of sedation needed to keep the

patient in the coma.

If extensive brain damage is suspected due to physical trauma, drug abuse, or any other factor,

EEG testing can help determine the actual extent of the damage.

EEG cannot measure the performance of the brain beyond its electrical activity, and cannot be used

to determine intelligence or to diagnose most mental health conditions (with the exception of

certain kinds of psychoses which affect electrical activity in the brain) [10].

ASET


Modern EEG tests, unlike the invasive procedures of the 18th and 19th centuries, are completely

painless and safe. No electricity is actually administered to the patient—it is simply measured.

6.1.2 EEG Technology and Research:

EEG technology enables research in a variety of fields, from medicine to marketing. Recent

research into sports-related concussions has used EEG testing to determine the short- and long-term

results of sustaining multiple concussions or other head traumas. EEG has also played an important

role in recent sleep studies, helping researchers learn the true effects of various lengths and

qualities of sleep on the human brain.

EEG testing is also being used in advertising. Researchers who observe eye movement, facial

expressions, and brain waves to determine the effectiveness of various pieces of media are

pioneering a field called neuromarketing, which seeks to use biometrics to better the odds of

advertising success in a world already oversaturated by media.

These are just a few examples of new and growing fields of research enabled by EEG

technology—the measuring of human engagement and brain activity has nearly universal

applications.

6.1.3 EEG-Enabled Prosthetics:

Controlling a prosthetic limb with your mind was the stuff of science fiction until 1999,

when a collaboration between researchers at the MCP Hahnemann School of Medicine in

Philadelphia and Duke University in Durham (North Carolina) reported that they were successful

in getting rats to control a simple robot using the rats’ brain activity alone. The field of

neuroprosthetics is young, but promising.

15 years later, a paraplegic wearing a robotic exoskeleton controlled by his own brain signals

kicked off the 2014 World Cup in Brazil. In 2015, researchers from the University of Houston

developed an algorithm which allowed an amputee to grasp objects using a robotic hand which he

controlled with his own mind.

Early research in the field of neuroprosthetics required more invasive methods of recording brain

signals (much like the early work of Richard Caton and Hans Berger). However, the robotic hand

ASET


at the University of Houston was enabled by EEG alone, signalling a drastic advancement in

neuroprosthetic technology and a new hope for people living with loss of limbs or mobility.

6.1.4 EEG-Enabled Communication Devices:

Most people are familiar with the story of theoretical physicist Stephen Hawking, who lost

the ability to speak after gradually becoming paralyzed due to ALS. His speech deteriorated rapidly

in the late 70s, at which point he required a translator of sorts to communicate clearly with others.

In the 80s, a serious bout of pneumonia necessitated a tracheotomy, which rendered him

completely unable to speak.

Figure 6.1.4 Stephen Hawking

Hawking used a computer program to help him speak for several years, which was fairly high-tech

for the late 1980s and early 90s. In 2005 he began using a program which allowed him to

communicate via computer using only the movement of his cheek muscles, and in 2012 he began

experimenting with brainwave-powered communication devices.

A communication device powered by brain activity alone would restore speech to patients even

with complete paralysis. The technology has yet to be perfected, but new research is promising,

and may one day lead to perfect technology-enabled communication for patients with ALS and

other degenerative illnesses [11].

ASET


6.1.5 EEG and Education:

Learning about how someone’s brain works is an incredible educational tool. The education

system suffers from a lack of resources and understanding, which hurts teachers’ ability to

effectively instruct students who have learning disabilities or simply don’t learn well from

conventional instruction methods.

Some connections have been made between ADHD diagnoses and EEG readings showing

dysregulated cortical electrical activity. While diagnostics are not the responsibility of a teacher,

applications of EEG technology which monitor focus and concentration could lead to a greater

understanding of which educational techniques work best for individual students. Additionally,

educational EEG-enabled games requiring intense focus could help unfocused students train their

brains to concentrate better [12].

6.1.6 EEG Entertainment:

What if you could play video games with your mind? It may not carry the scientific gravity

of neuroprosthetics, but it sure is fun. Neurogaming (are you starting to get the hang of this naming

convention?) is another new tech field—and definitely the most exciting growth area for the

gaming industry—enabled by EEG.

Game development company Crooked Tree Studios used NeuroSky’s EEG technology to

build “Throw Trucks With Your Mind”, the objective of which is probably self-explanatory.

Players focus their thoughts on a single object, and the NeuroSky MindWave Mobile

Headset interprets the electrical activity of their brains to pick up and hurl objects at opponents.

Lat Ware, the creator of “Throw Trucks With Your Mind”, explains that the inspiration for the

game came from his own experience with experimental neurofeedback therapy to treat his

Attention Deficit Disorder. Learning that his brain activity could be fed into and interpreted by a

computer fascinated Ware, eventually leading him to pitch his idea to Johnny Liu, the Director of

NeuroSky’s Developer Program. Ware then raised nearly $50,000 via Kickstarter to get his project

off the ground. The technology to support more applications of neurogaming exists but, until

recently, it was too expensive to appeal to the average consumer of video games. Now that EEG

headsets can be purchased for less than the cost of the newest traditional gaming console,

neurogaming is poised to be the next big thing in video game entertainment [13].

http://developer.neurosky.com/wp-content/uploads/2016/07/Throw_Trucks_With_Your_Mind_Revolutionizing_Entertainment_with_Neurogaming.pdf

http://store.neurosky.com/

http://store.neurosky.com/

ASET


Chapter 7 Future Scope

ASET


7.1 BCI ADVANTAGES & DISADVANTAGES

Some of the good advantages of BCI are direct communication between brain and devices,

better living for paralyzed people etc. While the disadvantages of BCI are; ill effects in the brain

due to viral attacks, requiring excessive training for proper usage, high cost, slow speed, lack of

better sensor modality, invasive BCIs are risky since it requires neurosurgery etc.

7.2 EEG FUTURESCOPE:

As more exciting developments are made with EEG applications and the cost of hardware

decreases, the future of EEG for device manufacturers seems full of promise. Consumers

increasingly turn to mobile apps and wearable devices (such as MyFitnessPal, Clue, MySugr, orthe

LifeBeat™ fitness tracking wristband) to gain insight into their health and wellness metrics, and

help them meet their personal health goals. As EEG data can be interpreted to measure biometrics

such as focus, attention, mental fatigue, stress, and more, it presents a new opportunity to give

consumers insight into their personal wellness metrics.

As mentioned above, with personal EEG devices becoming affordable for the average consumer,

the neurogaming applications are virtually endless. Augmented and virtual reality games which can

be controlled with the mind may seem like science fiction, but are solidly within the reach of

modern game developers. Combined with other biometrics such as heart rate, pupil dilation, and

facial expression analysis, it seems anything is possible–if you can observe, measure, and interpret

it, you can make a game of it.

7.3 BCI FUTURESCOPE:

As the coming step of technological advancement, many research workers and scientists are

trying to bring out wide variety of BCI applications useful for the society. Coming years, we can

make BCI restore and augment human functions thereby improving quality of living. Some

applications like flying an aeroplane just by thinking, a blind driving a vehicle etc. will be coming

to a reality. In the medical sector, research workers are trying to bring out miniaturized equipments

and introduction of wireless BCI. It’s said that in the coming future, we can replace the robotic

ASET


devices and directly bypass the signals to the nerves in the damaged part of the brain, thereby

allowing the paralyzed patient to move their body. Last but not the least, development in BCI can

bring out drastic and attractive changes to the society [14].

Future applications of BCI:

• Bionic eye

• Transfer the hearing impulse to brain.

• Robotic assistance to old and disabled.

• Real time gaming.

• Use a device similar to Google Glass. Include a BCI device within it.

• Use a BCI bar code reader in a product.

• When BCI bar code reader is read by a Google glass device activate the BCI interface in the

device and record and process the signal on a server level.

Figure 7.1: BCI - Future Scope (Robotic assistance to old and disabled)

ASET


Figure 7.2: BCI - Future Scope (Google Glass and BCI)


REFERENCES

[1] A Brain-Computer Interface for Shared Vehicle Control on TORCS Car Racing Game,

2014 10th International Conference on Natural Computation

[2] NeuroSky SDK for .NET: Development Guide and API Reference.

[3] Flappy Brain - Cambridge Jam Raspberry Pi project.

WEB REFERENCES

[4.1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0172400

[4.2] http://neurosky.com/biosensors/eeg-sensor/algorithms/

[5] http://blog.makezine.com/archive/2010/04/tan_le_demos_mind-controller_on_mar.html

[6] https://www.cnet.com/uk/news/moving-objects-with-mattels-brainwave-reading-

mindflex/

[7] http://www.knight-of-pi.org/raspberry-pi-mindcontrol-neurosky-mindwave-as-simple-

eeg-interface/

[8] http://neurosky.com/biosensors/eeg-sensor/algorithms/

[9] https://github.com/BarkleyUS/mindwave-python

[10] http://developer.neurosky.com/docs/doku.php?id=app_notes_and_tutorials

[11] https://www.pantechsolutions.net/blog/list-of-source-codes-available-for-neurosky-

mindwave-mobile/

[12] https://store.neurosky.com/products/pc-developer-tools

[13] http://openvibe.inria.fr/

[14] https://cofindtira.jimdo.com/2017/05/03/drivers-in-labview/

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0172400

http://neurosky.com/biosensors/eeg-sensor/algorithms/

http://blog.makezine.com/archive/2010/04/tan_le_demos_mind-controller_on_mar.html

https://www.cnet.com/uk/news/moving-objects-with-mattels-brainwave-reading-mindflex/

https://www.cnet.com/uk/news/moving-objects-with-mattels-brainwave-reading-mindflex/

http://www.knight-of-pi.org/raspberry-pi-mindcontrol-neurosky-mindwave-as-simple-eeg-interface/

http://www.knight-of-pi.org/raspberry-pi-mindcontrol-neurosky-mindwave-as-simple-eeg-interface/

http://neurosky.com/biosensors/eeg-sensor/algorithms/

https://github.com/BarkleyUS/mindwave-python

http://developer.neurosky.com/docs/doku.php?id=app_notes_and_tutorials

https://www.pantechsolutions.net/blog/list-of-source-codes-available-for-neurosky-mindwave-mobile/

https://www.pantechsolutions.net/blog/list-of-source-codes-available-for-neurosky-mindwave-mobile/

https://store.neurosky.com/products/pc-developer-tools

http://openvibe.inria.fr/

https://cofindtira.jimdo.com/2017/05/03/drivers-in-labview/

Brain Computer Interface & It's Applications | NeuroSky Minwave | Raspberry Pi

Engineering

Transcript of Brain Computer Interface & It's Applications | NeuroSky Minwave | Raspberry Pi