eL&mL 2010

Inferencing emotions through the triangulation of pupil size data, facialheuristics and self-assessment techniques

Carles Fernàndez, Universitat Oberta de Catalunya (UOC)

AFFECTIVE TECHNOLOGY LAB

St. Maarten, Netherland Antilles

They design how you will feel

It is about EMOTIONS

Marketing, Cinema, Videogames…

IT Emotional designers - An existing reality

LeT’s GO into EDUCATION

In online learning… what we are offering is really engaging?

Users expectations…

Promises…

What do we really give them?

UOC – Antigua (2001-2009)

Emotions, the difference between expectations and reality

• Because we know that the emotional dimension is very rellevant for students’ decision making, joy and satisfaction

• Because through the evaluation of emotions we will be able to obtain a wider and global view of students’ experiences

• Because there is no meaningful learning without emotions (emotions are the fuel for learning). Cognitive and affective layers can not work individually without affecting each other.

“ If Education means ‘to change the world’ , let’s use emotions, because they are closer to the soul than thoughts and ideas”

Why do we need to take emotions into account?

• We do not have solid theoretical models to explain them

• Historically, emotions are considered of low importance

• Emotions or beahavior-related are not universal

• Difficulties to measure

“the student asked the zen master: what do I have to do when I have such a big wall in front of me?

And the master responded: ‘just a step ahead’”

Although our current limitations, we know that emotions are esential for success. As such, let’s use them consciously as educational designers

Emotions: why not? Educational positivism against focus on emotions

Concept of ‘affective computing’: “computing that relates to, arises from, or deliberately influences emotion“ (Picard,R.)

As such Affective Educational Technology, could be: “the line of knowledge that focuses in the design and assessment of virtual learning environments and educational tools in order to promote engaging learning experiences (I love learning at UOC) “

Affective computing: the science of emotions and computers

Virtual reality, virtual worlds

Robots with emotions (social robots)

Multicommunication modalities (speech, posture, face…)

Brain research and devices (Neuroscience)

New devices for human-computer interaction

Affective computing: the science of emotions and computers

What do we do with emotions and learning?

About UOC:

• Fully online university (Undergraduate, Graduate and PhD programs)

• Headquarters in Barcelona, Spain• Founded 15 years ago as an online university• 47.000 students, 500 f-t faculty, 2000 p-t faculty

About our students:

- between 24 and 50 years old

- often hold a previous degree

- have work and family

- stressed out

- have little leisure time

- they would like to feel as a part of

an educational and innovative

community

About OLT (Office of Learning Technologies)

OLT MISSION:• to design, develop and deliver a high quality virtual

environment• To test new tools with learning interest• User Centered Design (UCD) approach• Open source technologies• Incorporating the affective dimension

The design and assessment of learning tools and environment at UOC - overview

Since 1995, several teams within the staff of the university have made their contributions to the design of the virtual learning environment

7 years ago we adopted a more systematic approach through the use of User Centered Design methodologies (student at the center)

The environment fit their needs The environment was NOT motivating enough

Introduce elements of engagement in the design of learning environments and tools

we needed

The Affective Technology Lab: where we focus and where we do not

The focus of our analysis of affect are:Virtual learning environmentsLearning tools, spaces and services

Not contents Not pedagogical models

Our objective is:

To design learning environments and tools that promote appropriate affective states such as joy or motivation

To design learning environments and tools that avoid unappropriate affective states such as boring, anxious, anger…

FOUR main things we do with affect

To understand how emotions work

To promote an affective design of learning

environments and tools

To assess learning environments and tools

from the affective point of view

To experment, innovate and play with new

technologies applied to affective computing

Affective Technology Lab: some of our projects

Creating an affective musicplayer

Affective Technology Lab: some of our projects

Adapting an affective-inteligent tutoring system for students

Affective Technology Lab: some of our projects

Empowering the value of community through geolocalisation

Affective Technology Lab: some of our projects

Learning where students are: the facebook experience

Affective Technology Lab: some of our projects

Designing guidelines to support designers in emotional design

The Enjoy Guidelines“A checklist for all”

Based on a set of general principles that all our learning spaces should follow. Designers apply these principles to specific learning tools

Affective Technology Lab: some of our projects

Testing several tools, devices and techniques for affective assessment:

•EEG monitoring•Biosignals (pupil size)•Facial and postural expression•Self-assessment•…

ASSESSING AFFECTIVE LEARNING ENVIRONMENTS

AND TOOLS

Testing the possibilities of three techniques to assess emotions

Up to now...

• Universities mainly use questionnaires to assess tools and courses

• Generally these surveys do not gather affective data• They use typical overall items such as: ‘Do you think the

course is motivating enough? Has the course met your expectations?

Our main objective

To test a methodology that allows us to gather relevant affective information of the user-computer interaction

The test

• Our objective: to test the emotional user experience with the new ‘personal homepage of the student’ from the affective point of view

• 7 students interacting for the first time with the new homepage of UOC

• Students have to carry out 5 tasks within the new mainpage (read an email, add a new module, change the position of another module, access a forum an the virtual library).

• These tasks are prone to provoke several affective reactions in the students

A methodology based in the triangulation of three techniques

• Pupil size analysis (quantitative)

• Interpretation of facial and body expressions (quant. / qual.)

• Self-assessment (qualitative)

About the techniques(I) Pupil size

State of the art: Correlation between arousal and pupil size

Is pupil size reactive to particular emotions?

Pupil size reactive to other variables

About the techniques (II)

The 10 Emotion Heuristics“Beyond what they tell us”

Association of facial and body expressions to particular emotions

About the techniques (III)Self-assessment

To gather affective data from the main source: learners

To confirm the data gathered with pupil size and ten heuristics

Test performance: some examples

The results

• We inferred a total number of emotions for each student (TIE). TIE=Emotions Inferred through Pupil size+Ten Heuristics+Self-assessment

• Both pupil size and ten heuristics are able to measure emotional reactions in the test (most of the affective reactions were accompanied of increased pupil size or specific expressions)

• Pupil size does not seem to be a good predictor if we use it alone to infer affective states

• Heuristics were more accurate to validate emotional reactions. Only 40% of observable problematic interactions showed increased pupil size. Ten heuristics may be useful in usability tests to infer emotions (integrated in user-centred desing methodologies).

• Students self-assessment confirmed more than 90% of the inferred affective states with the other techniques

General conclusions – Design&assessment

• Both methodologies for design and assessment are useful, but we need to update them with new advancements in newer technologies

• The affective methodologies should be integrated in the validation and design of prototypes through user-center-design methods

• These methodologies need to be affordable. Guidelines for design of affective environments, for assessment of facial expressions and eyetrackers are some examples

Thank you for your attention

Carles Fernàndez, cfernandezba@uoc.eduOffice of Learning TechnologiesUniversitat Oberta de Catalunya (UOC)