Physiological sensors and EEG A short introduction to (neuro-)physiological measurements.

of 35 /35
Physiological sensors and EEG A short introduction to (neuro-)physiological measurements

Embed Size (px)

Transcript of Physiological sensors and EEG A short introduction to (neuro-)physiological measurements.

  • Physiological sensors and EEGA short introduction to (neuro-)physiological measurements

  • Outline

    1. Physiological sensors and EEG 2. Classification of affective states 2.1 Background & experiment design 2.2 Features, Classification and Resumee

  • The sensorsElectroencephalogram (EEG)Electroocculogram (EOG)

    Electromyogram (EMG)Electrocardiogram (ECG)Galvanic skin response (GSR)Blood volume pulse (BVP)RespirationTemperaturecentral nervoussystemperipheral nervoussystem

  • Electro-----gram..occulo....myo....cardio..

  • Blood volume pulseinfrared photoelectric sensor detects changes in tissue blood volume 60 80 pulses a minute> heart rate and inter-beat-interval can be derivedfaster when exercised or aroused

  • Galvanic skin responseimpedance of the skin is measuredeccrine glands at hands and food solesincreases linear with arousal

  • Respirationcircumference of chest is measured via strain gaugenormal breath rate: 12 - 16 breath per minute (~500ml)faster and more shallow with higher arousal

  • Electroencephalogramnormal signaltheta bandalpha bandbeta bandgamma band0 1000 Hz4 8 Hz13 30 Hz8 12 Hz30 100 Hz

  • The 10 20 system

  • The lobes of the brain

  • Time- vs frequency domainnormal signaltheta bandalpha bandbeta bandgamma band0 1000 Hz4 8 Hz13 30 Hz8 12 Hz30 100 Hz

  • Event-related potentials (ERPs)The raw signal with stimuli presentations

    The averaging of the raw signal epochsleading to the ERP

  • ERPs from affective pictures

  • Time-frequency analysisevoked frequency analysisinduced frequency analysisTallon-Baudry & Bertrand, 1999

  • RecapitulationEEG measures the electrical potentials from synchroneously active pyramidal cells in the cortexhigh temporal resolution vs low spatial resolutiononly weak signals from deeper structures & closed fields blindnessERP analysis in time domain shows mainly low frequency componentsanalysis in frequency domain (evoked vs induced) also pics up high frequency components

  • Classification of affective statesBackground & experiment design

  • MotivationAffective Computing aimes at the enrichment of human-computer interactionPhysiological and neurophysiological signals give access to (non-observable) affective and cognitive states

  • Emotion Models2 dimensional model of emotionvalencearousalposneglowhighboredomjoyfearrelaxationAn emotion, or an affective state, is a reaction to an internal (e.g. thought)or external event (e.g. visual or auditory stimuli), and a behavioural disposition. frustration

  • RegulationAffective StateAppraisalStimulus presentation+/-+/-Hippocampus (HC)dorsal Anterior Cingulate Gyrus (dACG)dorsal prefrontal cortex (PFC)Amygdala Insula ventral Anterior Cingulate Gyrus (vACG)ventral prefrontal cortex (PFC)InsideOutsideHCInsulaAmygdaladACGvACGPFCNeurobiology of Emotional PerceptionPhillips et al. 2003

  • Overview Issues aBCIElicitation of emotionsGround truth constructionSensor modalitiesModality fusionFeature selection and reductionClassification

  • Example Study

  • Elicitation of emotionsnot easy to establish a natural context of emotion elicitation:subject- vs event-elicited lab setting vs. real worldexpression vs feelingopen vs hidden recordingemotion vs other purpose (for subject)

  • Affective Stimulivalencearousalposneglowhighboredomjoyfearrelaxationfrustration

  • Ground truth construction normed stimulus sets (e.g. IAPS, IADS) versus self-assessment after each trial

  • Sensor modalitiesphysiological modalities:slow response to stimulus (seconds)long inter-trial intervals (seconds)few (but long presented) stimuli per subjectneurophysiological modality:fast response to stimulus (miliseconds)short inter-trial intervals (miliseconds)many (but short presented) stimuli per subject

  • Experiment: Design Resting Stimulus SAM rating3 4 s 6 s not limitedemotion induction with visual affective stimuliself-assessment of (induced) affective state after each stimulus with Self-Assessment Maneken (SAM)

  • Classification of affective statesFeatures, Classification and Resumee

  • Modality fusiondata-level fusionfeature-level fusiondecision-level fusion

  • Feature selection and reductionSelection:manual selection (e.g. literature research)

    automatical selection / wrapper methodReduction:mapping from high into low dimensional space (e.g. PCA,ICA) > filter method

  • Feature extractionEEG: 6 features (from 6 frequency bands over 6 brain regions)physiological Sensors: mean, variance, and(minimum & maximum)from GSR, blood pressure,heart rate, respiration, andtemperature

  • Classificationmany possible classification approaches: decision trees, linear discriminant analysis, support vector machine, neural networksoptimal method depends on structure of data!keep training trials appart from test data, to avoid the contamination of the classifier.. !

  • Results2 classes: high vs low arousal3 classes: high, medium and low arousal

  • Classificationphysiological studies:4 class: up to 95%8 class: 81% (!)generalization of classifiers over subjects and time

    neurophysiological studies:3 class: up to 67%2 class: up to 79% ? generalization of classifiers over subjects and time ?

  • Recapitulationnatural context of emotion elicitationcharacteristics of sensors are importantmany different approaches for data fusion, feature selection/reduction and classification no optimal method per sestill a long way toward an affect classification in an natural and multimodal real-world setting.. but we are getting there!

  • Thanks

    and neutrality is just assumed from normalized values..solution: taking neutral sounds from another sound database ..