Papavasileiou-1 CSE 5810 Brain Computer Interface in BMI Ioannis Papavasileiou Computer Science &...
-
Upload
claribel-scarlett-rice -
Category
Documents
-
view
218 -
download
0
Transcript of Papavasileiou-1 CSE 5810 Brain Computer Interface in BMI Ioannis Papavasileiou Computer Science &...
Papavasileiou-1
CSE5810
Brain Computer Interface in BMIBrain Computer Interface in BMI
Ioannis Papavasileiou Computer Science & Engineering Department
The University of Connecticut371 Fairfield Road, Unit 4155
Storrs, CT 06269-2155
Papavasileiou-2
CSE5810
What is BCI?What is BCI? BCI is:BCI is:
System that allows direct communication pathway between human brain and computer
It consists of data acquisition devices, and appropriate algorithms
How is it used in BMI:How is it used in BMI: Clinical research Disease-condition detection and treatment Human computer interfaces for
Control Emotions detection Text input - communication
Papavasileiou-3
CSE5810
Research areas involvedResearch areas involved Computer scienceComputer science
Data mining Machine learning Human computer interaction
NeuroscienceNeuroscience Cognitive scienceCognitive science EngineeringEngineering
Papavasileiou-4
CSE5810
Key challengesKey challenges Technology-related:Technology-related:
Sensor quality – low SNR Supervised learning – “curse of dimensionality” System usability Real-time constraints Non-invasive EEG information transfer rate is
approx. 1 order of magn. lower People-relatedPeople-related
People are not always familiar with technology Preparation – training phases are not fun! Concentration, attention consciousness levels Task difficulty
Papavasileiou-5
CSE5810
BCI componentsBCI components Data acquisitionData acquisition
Electroencephalography (EEG) Electrical activity recording Invasive or not
Functional Near Infrared Spectroscopy (fNIRS) Recording of infrared light reflections of the brain
Functional magnetic resonance imaging (FMRI) Detection of changes in blood flow
Data AnalysisData Analysis Data mining & machine learning
Decision makingDecision making Output & ControlOutput & Control
HCI
Papavasileiou-6
CSE5810
Typical BCI architectureTypical BCI architecture
Papavasileiou-7
CSE5810
Electroencephalography (EEG)Electroencephalography (EEG) What is it:What is it:
Recoding of the electrical activity of the brain
Types:Types: Invasive Non-invasive
Properties:Properties: High temporal resolution Low spatial resolution Scalp acts as filter!
Papavasileiou-8
CSE5810
International 10-20 standardInternational 10-20 standard Electrodes located at the scalp at predefined Electrodes located at the scalp at predefined
positionspositions Number of electrodes can varyNumber of electrodes can vary
Papavasileiou-9
CSE5810
The EEG wavesThe EEG waves Alpha – Alpha –
occipitallyoccipitally Beta – frontally Beta – frontally
and parietallyand parietally Theta – children, Theta – children,
sleeping adultssleeping adults Delta – infants, Delta – infants,
sleeping adultssleeping adults
Papavasileiou-10
CSE5810
fMRIfMRI Functional magnetic resonance imagingFunctional magnetic resonance imaging Fact:Fact:
Cerebral blood flow and neuronal activation coupled
Detection of blood flow changesDetection of blood flow changes Use of magnetic fieldsUse of magnetic fields High spatial resolutionHigh spatial resolution Low temporal resolutionLow temporal resolution Clinical use:Clinical use:
Assess risky brain surgery Study brain functions
Normal Diseased Injured
Map functional areas of the brain
Papavasileiou-11
CSE5810
fNIRSfNIRS Functional Near Infrared SpectroscopyFunctional Near Infrared Spectroscopy Project near infrared light into the brain from the scalpProject near infrared light into the brain from the scalp Measure changes in the reflection of the light due toMeasure changes in the reflection of the light due to
Oxygen levels associated with brain activity Result absorption and scattering of the light
photons Used to build maps of brain activity
High spatial resolution High spatial resolution <1 cm
Lower temporal resolutionLower temporal resolution >2-5 seconds
Papavasileiou-12
CSE5810
BMI & clinical applicationsBMI & clinical applications Diagnose:Diagnose:
Epilepsy – seizures Brain-death Alzheimer’s disease Physical or mental problems
Study of:Study of: Problems with loss of consciousness Schizophrenia (reduced Delta waves during sleep)
Find location of:Find location of: Tumor Infection bleeding
Source: http://www.webmd.com/, http://www.nlm.nih.gov
Papavasileiou-13
CSE5810
Sleep disorders & mental tasksSleep disorders & mental tasks Sleep disorders studySleep disorders study
Insomnia Hypersomnia Circadian rhythm disorders Parasomnia (disruptions in slow sleep waves)
Mental tasks monitoringMental tasks monitoring Mathematical operations Counting Etc.
Papavasileiou-14
CSE5810
NeurofeedbackNeurofeedback Applications in
Autistic Spectrum Disorder (ASD) Anxiety Depression Personality Mood Nervous system
Self control
Papavasileiou-15
CSE5810
Feedback EEG-BCI architectureFeedback EEG-BCI architecture
Papavasileiou-16
CSE5810
Typical data analysis processTypical data analysis process Data acquisition and segmentationData acquisition and segmentation
Preprocessing Removal of artifacts Facial muscle activity External sources, like power lines
Feature extractionFeature extraction Typically sliding window Time-frequency features Latency introduced
Papavasileiou-17
CSE5810
Feature extractionFeature extraction Model-based methodsModel-based methods
Require selection of the model order FFT (Fast Fourier Transform) – based methodsFFT (Fast Fourier Transform) – based methods
Apply a smoothing window Features used:Features used:
Specific frequency band power Band-pass filtering and squaring Autoregressive spectral analysis
Many times a feature selection or projection is done to Many times a feature selection or projection is done to reduce the huge feature vectorsreduce the huge feature vectors
Papavasileiou-18
CSE5810
Data ClassificationData Classification Typical classifiers usedTypical classifiers used
Artificial Neural Networks (ANN) Linear Discriminant analysis (LDA) Support Vector Machines (SVM) Bayesian classifier Hidden Markov Models (HMM) K-nearest neighbor (KNN)
Parameters for each classifier can affect the Parameters for each classifier can affect the performanceperformance # of hidden units in ANN # of supporting vectors for SVMs Etc.
Papavasileiou-19
CSE5810
Human computer interactionHuman computer interaction BCIs are considered to be means of communication and BCIs are considered to be means of communication and
control for their userscontrol for their users HCI community defines three types:HCI community defines three types:
Active BCIs Consciously controlled by the user E.g. sensorimotor imagery (multi-valued control signal)
Reactive BCIs Output derived from reaction to external stimulation Like P300 spellers
Passive BCIs Output is related to arbitrary brain activity E.g. memory load, emotional state, surprise, etc.
Used in assistive technologies and rehabilitation Used in assistive technologies and rehabilitation therapiestherapies
Papavasileiou-20
CSE5810
BCI & Assistive TechnologiesBCI & Assistive Technologies Communication systemsCommunication systems
Basic yes/no Character spellers Virtual keyboards
ControlControl Movement imagination
Cursor Wheelchairs Artificial limbs & prosthesis
Automation in smart environments Current BCI systems have at most 10-25 bits/minute Current BCI systems have at most 10-25 bits/minute
maximum information transfer ratesmaximum information transfer rates It can be valuable for those with severe disabilities
Papavasileiou-21
CSE5810
P300 spellersP300 spellers Most typical reactive BCIMost typical reactive BCI 3-4 characters / min with 95% success3-4 characters / min with 95% success
Papavasileiou-22
CSE5810
P300 waveP300 wave Event related potential (ERP)Event related potential (ERP) Elicited in the process of decision makingElicited in the process of decision making Occurs when person reacts to stimulusOccurs when person reacts to stimulus Characteristics:Characteristics:
Positive deflection in voltage Latency 250 to 500 ms
Typically 300 ms Close to the parietal lobe in the brain Averaging over multiple records required
Papavasileiou-23
CSE5810
Other ERP usesOther ERP uses Lie detectionLie detection
Increased legal permissibility Compared to other methods
ERP abnormalities related to conditions such as:ERP abnormalities related to conditions such as: Parkinson’s Stroke Head injuries And others
Typical ERP paradigmsTypical ERP paradigms Event related synchronization (ERS) Event related de-synchronization (ERD)
Papavasileiou-24
CSE5810
Other Control BCI paradigmsOther Control BCI paradigms Lateralized readiness potentialLateralized readiness potential
Game control 1~2 seconds latency Negative shift in EEG develops before actual
movement onset Steady-state visually evoked potentials (SSVEPs)Steady-state visually evoked potentials (SSVEPs) Slow cortical potential (SCP)Slow cortical potential (SCP)
Imaged movements affect mu-rhythms They shift polarity (+ or -) of SCP
Sensorimotor cortex rhythms (SMR)Sensorimotor cortex rhythms (SMR) EMGEMG
Papavasileiou-25
CSE5810
SCP & SMR vs P300SCP & SMR vs P300 Typically SCP and SMR BCIs require significant Typically SCP and SMR BCIs require significant
training to gain sufficient controltraining to gain sufficient control In contrast P300 BCIs require less as they record In contrast P300 BCIs require less as they record
response to stimuliresponse to stimuli However, they require some sort of stimuli like
visual (monitor always in place) or audio Also SCP BCIs have longer response timesAlso SCP BCIs have longer response times
Papavasileiou-26
CSE5810
Binary speller controlBinary speller control User imagines movement of cursorUser imagines movement of cursor
Typically hand movement The goal is to select a characterThe goal is to select a character
Papavasileiou-27
CSE5810
Wheel chair controlWheel chair control All the mentioned BCI All the mentioned BCI
paradigms have been paradigms have been applied to wheelchair applied to wheelchair controlcontrol
Either using a monitor for Either using a monitor for feedbackfeedback
Or active paradigms as Or active paradigms as sensorimotor imagery sensorimotor imagery (SMR)(SMR)
Similar approaches have Similar approaches have been applied to roboticsbeen applied to robotics Artificial limbs etc
Papavasileiou-28
CSE5810
Environment controlEnvironment control BCIs used by disabled to improve quality of lifeBCIs used by disabled to improve quality of life Operation of devices likeOperation of devices like
Lights TV Stereo sets Motorized beds Doors Etc
Typically use of P300, SMR and EMG related BCIsTypically use of P300, SMR and EMG related BCIs
Papavasileiou-29
CSE5810
EMG-based human-robot interface exampleEMG-based human-robot interface example Motion prediction based on hand positionMotion prediction based on hand position EMG pattern classification as control commandEMG pattern classification as control command Combination of both yields motion command to Combination of both yields motion command to
prosthetic handprosthetic hand
Papavasileiou-30
CSE5810
Emotions detectionEmotions detection Use of facial expressions to imply user emotionsUse of facial expressions to imply user emotions ERD/ERS based BCIsERD/ERS based BCIs
Emotional state can change the asymmetry of the frontal alpha
P300 - SSVEPP300 - SSVEP Emotional state can change the amplitude of the
signal from 200ms after stimulus presentation
Papavasileiou-31
CSE5810
BCIs for recreationBCIs for recreation GamesGames
EPOC headset Mindset
Virtual realityVirtual reality Outputs of a BCI areShown virtual environment
Creative ExpressionCreative Expression Music
Generated form EEG signals Visual art
Painting for artists who are locked in as a result of ALS – amyotrophic lateral sclerosis
Papavasileiou-32
CSE5810
Security and EEGSecurity and EEG EEG has been used in user authenticationEEG has been used in user authentication Every brain is differentEvery brain is different Different characteristics of EEG waves are used in Different characteristics of EEG waves are used in
user authenticationuser authentication ProsPros
User has nothing to remember Harmless Automatically applied
ConsCons User has to wear an EEG headset Accuracy is still not 100% Still not used in practice
Papavasileiou-33
CSE5810
Thank you!Thank you!