1

2

Abstract

- Two experimental paradigms :

- EEG-based system that is able to detect high mental workload

in drivers operating under real traffic condition

① An auditory workload scheme

② A mental calculation task

- performance is strongly subject-dependent;

however, the results are good to excellent for the majority of subjects

3

Introduction

- Important issue in fields : pilots, flight controllers,

operators of industrial plants

- Be integrated with existing systems to maximize the performance

- Approach #1 : A closed-loop system

The system’s interaction with the operator is adjusted

According to the operator’s mental workload

- Approach #2 : The workload detector as an objective measure of mental workload

To develop improved modes of human-machine interaction

- Driving a car while performing additional tasks that model interaction

with the car’s systems To obtain a system that is able to measure and mitigate

mental workload (1) in real time and (2) in a real operational environment

4

The Experimental Setup

- A system that is able to measure and mitigate mental workload

in real time and in real operational environments.

⊙ the execution of secondary tasks not related to driving

⊙ Twelve male and five female subjects age 20 to 32 years old

⊙ Approximately 100 km/h on the highway in moderate traffic conditions

⊙ Not to speak during the experiment in order to avoid additional workload

- Perform three types of tasks : #1 Driving the vehicle

#2 An auditory reaction time task

#3 A High mental workload task

5

#3 - 2 An auditory reaction time task

#3 - 1 A mental calculation task

① German words links (left) and rechts (right) were randomly presented every 7.5 s

② Asked to count down in steps of twenty-seven

③ After two minutes, the subjects were asked for the final result

① A female news reader and a male voice reciting from a book

② The subject were instructed to follow the latter. ( To verify whether the subjects were engaged or not, they had to answer related

questions)

Subject had to their attention to one of two simultaneously presented voice recordings,

replicating a situation in which several vehicle occupants are talking at the same time.

#2 An auditory reaction time task

① Give three digit random number (between 800 and 999).

② Press corresponding buttons

6

A crucial purpose of the experiment is to investigate

whether the output of the workload detector can be used

to control the secondary task

7

Online Detection of Mental Workload

#1 The Workload detector

8

⊙ Artifact channel removal

Such broad-band differences are characteristic for muscle artifacts (> 20 Hz) or eye artifacts (< 6 Hz).

The channels that exhibit those differences are excluded.

⊙ Selection subset channelSubset is one of four candidate sets that potentially include frontal, occipital, and temporal scalp positions.

⊙ Spatial filteringEach of the selected channels is normalized by the common median reference signal

(the median of all channels is subtracted from each channel)

⊙ Bandpass filteringbandpass filter using one of the bands listed above

9

⊙ ClassificationLinear model whose parameters are computed by standard

linear discriminant analysis (LDA) of the feature vectors

The output is scalar value : Low workload (values below zero)

High workload (values above zero)

high and low workload, by means of a threshold scheme that employs

a hysteresis, which makes the classification substantially more robust

(ml < mh)

10

⊙ Parameter Calibration

To find suitable values for all the previously mentioned subject- and task-specific parameters,

they use the well known cross-validation Technique.

This procedure is performed for each possible combination of parameter candidates in the feature extraction part

(EEG channel subset, spatial filter, frequency band, window length)

11

Result

#1 Neurophysiological Interpretation

12

#2 Accuracy of the Workload Detector

Auditory workload : (95.6% correct) Mental calculation workload : (91.8% correct)

13

#3 Performance Improvement

14

15

AbstractMethodology for relating neural variability to response variability, describing studies for response accuracy and response latency during visual target detection.

Introduction

- Analysis of trial-averaged ERP in EEG has enabled one to assess the speed of visual recognition

and discrimination in terms of the timing of the underlying neural processes.

- More recent work has used single-trial analysis of EEG to characterize the neural activity directly

correlated with behavioral variability during tasks involving rapid visual discrimination.

- These results suggest that components extracted from the EEG can capture the neural correlates

of the visual recognition and decision making processing on a trial-by-trial basis.

‘ Cortically-coupled computer vision’

16

Linear methods for single-trial analysis

- The goal of a BCI system is to detect neuronal activity associated with cognitive events.

- Identify only one type of event (visual target recognition, differentiate this from other visual processing)

- 64channels , 1000Hz, time window is 0.5s (32000 samples, 32000 dimensional feature vector)

Sampling1000Hz (up to 500Hz) => L=50 time window, 20Hz (up to10Hz), 640 dimension

- The task is a binary classification

- The EEG activity is recorded as D x T values : D is the number of channels

T is the number of samples

- For reasonable classification :

(1) Reduce the trial-to-trial variability by filtering the signal to remove 60Hz interference and slow drifts.

(2) Reduce the dimentionality of the problem by stepping classification window every L-th sample.

L = 50, the signal of interest is at 10Hz while faster signal variation are considered noise.

1 , 2, 3. . . . . . . .500

1

1

.

.

1 . . . . 498,499,500

D

T

1 , 2, 3. . . . . . . . .10

1

1

.

.

1 . . . . . . . . . . 8,9,10

D

T

17

X is projected onto a single dimension y.

(X is a matrix of channels by samples, and

y is a row vector containing multiple samples).

In this equation the inner product computes

the average over trials and samples.

When the intensity y averaged within

the specified time window is used as classification

criteria we achieve on this data an Az-value of 0.84.

18

EEG correlates of perceptual decision making - Using single-trial linear discrimination analysis

To identify the cortical correlates of decision making during rapid discrimination of images

- A series of target (faces) and non-target (cars)

trials are presented in rapid succession

- Stimulus evidence is varied by manipulating

the phase coherence of the images

- Using a set of 12 face (Max Planck Institute face database)

and 12 car grayscale images

(image size 512 x 512 pixels, 8-bits/pixel).

19

Early component ( = 170 ms following stimulus)

Late component ( > 300 ms following stimulus)

20

Identifying cortical processes leading to response time variability

- Studying visual target detection using an RSVP paradigm

- During this task, participants are presented with a continuous sequence of natural scenes.

RSVP : rapid serial visual presentation

- 4 blocks of 50 sequences

- Each sequence has a 50% chance of containing one target image with one or more people

in a natural scene

- These target images can only appear within the middle 30 images of each 50 image sequence .

- The Each image was presented for 100 ms

21

- Gaussian profile :

- Linear regression coefficients :

latency of the component activity