USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY...

41
USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact [email protected] camillejeunet.wordpress.com February, 15 th – 2018 EPFL / Inria Workshop C. JEUNET in collaboration with A. Lécuyer, F. Argelaguet (Inria Rennes), J.d.R. Millan, R. Chavarriaga (EPFL)

Transcript of USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY...

Page 1: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND &

IMPROVE COGNITIVE PERFORMANCE.

[email protected]

February, 15th – 2018

EPFL / Inria Workshop

C. JEUNET

in collaboration with A. Lécuyer, F. Argelaguet (Inria Rennes), J.d.R. Millan, R. Chavarriaga (EPFL)

Page 2: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

Page 3: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Page 4: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique

Page 5: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique Controlled Environment

Page 6: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

/ EEGElectroEncephaloGraphy

/ VRVirtual Reality

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique Controlled Environment

Page 7: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Page 8: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

SENSE OF AGENCY

UnderstandingCognitive Performance

Using EEG & VR

Project #1

Page 9: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

SENSE OF AGENCY COVERT ATTENTION

UnderstandingCognitive Performance

Using EEG & VR

Project #1 Project #2

ImprovingCognitive Performance

Using EEG & VR

Page 10: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Project #1 – SENSE OF AGENCY

Page 11: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

SENSE OF EMBODIEMENT

Sense of Self-Location

Sense of Body-Ownership

Sense of Agency

Page 12: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

SENSE OF EMBODIEMENT

Sense of Self-Location

Sense of Body-Ownership

Sense of Agency

Page 13: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

Our approach: Rely on theories from psychology & philosophy

Sense of Agency

Page 14: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

Relies on

3 principlesPriority PrincipleExclusivity PrincipleConsistency Principle

Includes

2 componentsFeeling of AgencyJudgement of Agency

Our approach: Rely on theories from psychology & philosophy

Sense of Agency

Page 15: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Assess the relevance of our approach to manipulate and measure the Sense of Agency in Virtual Environments.

I - Objective

▼ Project #1 – Sense of Agency

Page 16: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

VIDEO

see HERE

Page 17: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Sense of Agency

III - Discussion

Our approach enabled us to manipulate and measure the SoA

Questions currently investigated: Can we measure variations of the SoA in real-time? Do all the principles impact equally the SoA?

Jeunet, C., Albert, L., Argelaguet, F., & Lécuyer, A. (2018). "Do you feel in control?": Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 1-10.

Page 18: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Project #2 - COVERT ATTENTION

Page 19: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

Page 20: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Biomechanics]

Page 21: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Biomechanics] [Physiology]

Page 22: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology]

Page 23: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

Page 24: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Page 25: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Using Neurofeedback

▼ Project #2 - Context

Page 26: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Modulation

Feedback

Learning to self-regulate their brainactivity in order to improve motor or

cognitive functions…

▼ Project #2 - Context

Neurofeedback

Central Florida biofeedback

Page 27: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Using Neurofeedback

▼ Project #2 - Context

Page 28: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

What ability should be trained?

▼ Project #2 - Context

Page 29: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

What ability should be trained?

Covert Visuo-Spatial Attention (CVSA)

▼ Project #2 - Context

Page 30: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

[Psychology]

▼ Project #2 - Approach

Page 31: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

Page 32: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

1. Does CVSA have specific neurophysiological correlates?

Page 33: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

1. Does CVSA have specific neurophysiological correlates?2. Are the characteristics of these correlates related to performance?

Page 34: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Investigating the neurophysiological correlates of CVSA &the relationship between these correlates and performance.

I - Objective

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 35: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

II - Protocol

▼ Project #2 - CVSA – Soccer Goalkeepers

VIDEO

Page 36: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

VIDEO

Page 37: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Lateralisation Index = Power_right – Power_left

III - Results

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 38: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Lateralisation Index = Power_right – Power_left

III - Results

2-way ANOVA for repeated measuresTime * Class interaction – p < 0.001

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 39: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

III - Results

Alpha (8-14Hz) power – task vs. rest

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 40: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

IV - Discussion

Replication of results from the literature [Schmidt et al., 2010; Tonin et al., 2013]

Questions currently investigated: Is this marker (LI) suitable for a neurofeedback training procedure?

Is it measurable on a single trial basis? Does it correlate with performance or expertise?

▼ Project #2 - CVSA – Soccer Goalkeepers

C. Jeunet, B. Bideau, F. Argelaguet, R. Chavarriaga, J. del R. Millán, A. Lécuyer & R. Kulpa, “Investigating neurophysiologicalcorrelates of covert attention in soccer goalkeepers”, World Conference on Science and Soccer (WCSS) 2017

Page 41: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND &

IMPROVE COGNITIVE PERFORMANCE.

[email protected]

February, 15th – 2018

EPFL / Inria Workshop

C. JEUNET

Thank You