Download - USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact [email protected]

Transcript
Page 1: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND &

IMPROVE COGNITIVE PERFORMANCE.

[email protected]

February, 15th – 2018

EPFL / Inria Workshop

C. JEUNET

in collaboration with A. Lécuyer, F. Argelaguet (Inria Rennes), J.d.R. Millan, R. Chavarriaga (EPFL)

Page 2: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

Page 3: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Page 4: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique

Page 5: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique Controlled Environment

Page 6: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

/ EEGElectroEncephaloGraphy

/ VRVirtual Reality

Understand & Improve

/ Cognitive Performance

NEED FOR

Brain Imaging Technique Controlled Environment

Page 7: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

Page 8: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

SENSE OF AGENCY

UnderstandingCognitive Performance

Using EEG & VR

Project #1

Page 9: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Rationale of the Approach

SENSE OF AGENCY COVERT ATTENTION

UnderstandingCognitive Performance

Using EEG & VR

Project #1 Project #2

ImprovingCognitive Performance

Using EEG & VR

Page 10: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Project #1 – SENSE OF AGENCY

Page 11: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

SENSE OF EMBODIEMENT

Sense of Self-Location

Sense of Body-Ownership

Sense of Agency

Page 12: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

SENSE OF EMBODIEMENT

Sense of Self-Location

Sense of Body-Ownership

Sense of Agency

Page 13: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

Our approach: Rely on theories from psychology & philosophy

Sense of Agency

Page 14: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Theoretical Background

Relies on

3 principlesPriority PrincipleExclusivity PrincipleConsistency Principle

Includes

2 componentsFeeling of AgencyJudgement of Agency

Our approach: Rely on theories from psychology & philosophy

Sense of Agency

Page 15: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Assess the relevance of our approach to manipulate and measure the Sense of Agency in Virtual Environments.

I - Objective

▼ Project #1 – Sense of Agency

Page 16: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

VIDEO

see HERE

Page 17: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #1 – Sense of Agency

III - Discussion

Our approach enabled us to manipulate and measure the SoA

Questions currently investigated: Can we measure variations of the SoA in real-time? Do all the principles impact equally the SoA?

Jeunet, C., Albert, L., Argelaguet, F., & Lécuyer, A. (2018). "Do you feel in control?": Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 1-10.

Page 18: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Project #2 - COVERT ATTENTION

Page 19: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

Page 20: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Biomechanics]

Page 21: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Biomechanics] [Physiology]

Page 22: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology]

Page 23: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

Page 24: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

▼ Project #2 - Context

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Page 25: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Using Neurofeedback

▼ Project #2 - Context

Page 26: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Modulation

Feedback

Learning to self-regulate their brainactivity in order to improve motor or

cognitive functions…

▼ Project #2 - Context

Neurofeedback

Central Florida biofeedback

Page 27: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

How to train athletes’ mental state?

Using Neurofeedback

▼ Project #2 - Context

Page 28: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

What ability should be trained?

▼ Project #2 - Context

Page 29: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

[Cognition]

[Neurophysiology]

[Psychology]

[Biomechanics] [Physiology] [Mental State]

What ability should be trained?

Covert Visuo-Spatial Attention (CVSA)

▼ Project #2 - Context

Page 30: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

[Psychology]

▼ Project #2 - Approach

Page 31: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

Page 32: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

1. Does CVSA have specific neurophysiological correlates?

Page 33: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

[Social]

Neurofeedback Training of CVSA

Using Virtual Reality

[Psychology]

▼ Project #2 - Approach

1. Does CVSA have specific neurophysiological correlates?2. Are the characteristics of these correlates related to performance?

Page 34: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Investigating the neurophysiological correlates of CVSA &the relationship between these correlates and performance.

I - Objective

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 35: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

II - Protocol

▼ Project #2 - CVSA – Soccer Goalkeepers

VIDEO

Page 36: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

VIDEO

Page 37: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Lateralisation Index = Power_right – Power_left

III - Results

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 38: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

Lateralisation Index = Power_right – Power_left

III - Results

2-way ANOVA for repeated measuresTime * Class interaction – p < 0.001

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 39: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

III - Results

Alpha (8-14Hz) power – task vs. rest

N = 12

▼ Project #2 - CVSA – Soccer Goalkeepers

Page 40: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

IV - Discussion

Replication of results from the literature [Schmidt et al., 2010; Tonin et al., 2013]

Questions currently investigated: Is this marker (LI) suitable for a neurofeedback training procedure?

Is it measurable on a single trial basis? Does it correlate with performance or expertise?

▼ Project #2 - CVSA – Soccer Goalkeepers

C. Jeunet, B. Bideau, F. Argelaguet, R. Chavarriaga, J. del R. Millán, A. Lécuyer & R. Kulpa, “Investigating neurophysiologicalcorrelates of covert attention in soccer goalkeepers”, World Conference on Science and Soccer (WCSS) 2017

Page 41: USING ELECTROENCEPHALOGRAPHY (EEG) AND ......USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND & IMPROVE COGNITIVE PERFORMANCE. Contact camille.jeunet@inria.fr

USING ELECTROENCEPHALOGRAPHY (EEG) AND VIRTUAL REALITY (VR) TO UNDERSTAND &

IMPROVE COGNITIVE PERFORMANCE.

[email protected]

February, 15th – 2018

EPFL / Inria Workshop

C. JEUNET

Thank You