Human-Computer Interaction: Adaptation, Distribution, Evaluation

94

Transcript of Human-Computer Interaction: Adaptation, Distribution, Evaluation

Page 1: Human-Computer Interaction: Adaptation, Distribution, Evaluation
Page 2: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Human-Computer Interaction:Adaptation, Distribution, Evaluation

Jean Vanderdonckt

Louvain School of Management (LSM)

Université catholique de Louvain (UCL)Place des Doyens, 1 – 1348 Louvain-la-Neuve (Belgium)

[email protected]

Page 3: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Who is talking?

Jean VANDERDONCKT

• Researcher in Human-Computer Interaction (HCI) since 1988

• Full professor at UCL, Past President of LSM Research Institute

• Co-editor-in-Chief of Springer HCI Series: www.springer.com/series/6033

• Co-editor-in-Chief of SpringerBriefs in HCI: www.springer.com/series/15580

• Associate Editor of ACM TiiS: http://tiis.acm.org/editorial.cfm

• Associate Editor of KAIS: http://www.springer.com/computer/information+systems+and+applications/journal/10115

• Adv. Board of Interacting with Computers: http://iwc.oxfordjournals.org/

• Coordinator of the UsiXML Consortium: www.usixml.org

3Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 4: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Who is talking?

• Google Scholar: http://scholar.google.com/citations?user=U-FgGrkAAAAJ

• LinkedIn Profile: http://www.linkedin.com/in/jeanvdd

• DBLP: http://www.informatik.uni-trier.de/~ley/pers/hd/v/Vanderdonckt:Jean

• Microsoft Academic: https://academic.microsoft.com/#/detail/2284081945

• Academia Profile: http://uclouvain.academia.edu/JeanVanderdonckt

• ResearchGate: https://www.researchgate.net/profile/Jean_Vanderdonckt2

• YouTube: http://www.youtube.com/results?search_query=usixml

• Slides: http://www.slideshare.net/jeanvdd

• Books: https://www.amazon.com/Jean-Vanderdonckt/e/B01640UKYK

4Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 5: Human-Computer Interaction: Adaptation, Distribution, Evaluation

What is adaptation?

• “The idea of an adaptive interface is straightforward. Simply, it means that the interface should adapt to the user; rather than the user adapting to the system”

• Adaptation consists in transforming different aspects of the systems in different levels, according to the context of use– For example: concerning the presentation, the contrast can

be adapted according to the preference of the user

• How to consider all the possible dimensions, levelsand contexts?

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

5

Norcio and Stanley, Adaptive Human-Computer Interfaces: A literature survey and perspectives, IEEE Transactions on Systems, Man and

Cybernetics, Vol 19, n 2, March/April, 1989

Page 6: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Why to adapt?

• ‘One size does not fit all’• Different contexts of use with different

– Users• Age, impairments, preferences, profiles…

– Platforms• Mobile phones, laptops, desktops, idTV…

– Environments• Levels of light, noise, temperature…

• A context of use: C=(U, P, E)

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

6

Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J.,

A Unifying Reference Framework for Multi-Target User Interfaces, Interacting with Computers, Vol. 15, No. 3, June 2003, pp. 289-308.

Page 7: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Interactive Software evolution: context of use =(U,P,E)

7

time

Platform

User

Environment

Language

What is the situation today?

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

Page 8: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Diversity of Users

• Able-bodied users

• People with disabilities

– Visual

– Motor

– Cognitive

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

8

Page 9: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Diversity of Users

• Virtual User Modeling and Simulation (VUMS)

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

9

http://www.veritas-project.eu/

Page 10: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Variety of Tasks

• Taxonomy of canonical task types

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

10

González-Calleros, J.M., Guerrero-García, J., Vanderdonckt, J., Muñoz-Arteaga, J., Towards Canonical Task Types for User Interface Design, Proc. of LA-Web/CLIHC'2009

Page 11: Human-Computer Interaction: Adaptation, Distribution, Evaluation

[Forrester Research,2003]

Multiple Interaction Preferences

• Users want to determine their own path on each platform

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

11

Page 12: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Heterogeneous Platforms

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

12

http://wurfl.sourceforge.net/

Page 13: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

13

Heterogeneous Platforms

Page 14: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Many Contexts of Use

Location Role Device Experience

Sporting Music selection, clips

Working Travel booking site

Powerful interface for complex operations

Driving Trafic, weather notification

Everywhere connectivity for simple data exchange

Playing Games

TV is multi-media family device #1

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

14

Page 15: Human-Computer Interaction: Adaptation, Distribution, Evaluation

15

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

1U, S or a T maintains

up-to-date a series of

goals to ensure user

interface adaptation

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

López-Jaquero, V., Vanderdonckt, J., Montero, F., González, P., Towards an Extended Model of User Interface Adaptation: the ISATINE framework, Proc. of EIS’2007

Page 16: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

Adaptation request

Detection of

adaptation need

Notification for

adaptation request

2

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

16

Page 17: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

3

Specification by

demonstration

Specification by

demonstration

Specification by

definition

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

17

Page 18: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

This stage specifies

which entity will apply the

adaptation specified in

the previous stage

4

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

18

Page 19: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

This stage specifies which

entity will ensure a smooth

transition between the UI

before and after adaptation

5

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

19

Page 20: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:

This stage specifies which

entity will produce meaningful

information in order to facilitate

the understanding of the

adaptation to the other entities

6

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

20

Page 21: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

Goals for user

Interface adaptation

Meaning of an

input adaptation

Meaning of an

input adaptation

Meaning of an

output adaptation

Specification

of adaptation

Shape of an

input adaptation

Shape of an

output adaptation

Application of

adaptation

INterpretation

of evaluation

Initiative for

adaptation

Evaluation

of adaptation

Interactive

System

Interactive

System

Transition with

adaptation

Adaptation

semantic

distance

in input

Adaptation

articulatory

distance

in input

Adaptation

semantic

distance

in output

Adaptation

articulatory

distance

in output

Gulf o

f adapta

tion e

xecution

Gulf o

f adapta

tion e

valu

atio

n

• The specialization of Norman’s model for adaptation

results into the ISATINE framework:This stage specifies the

entity responsible for

evaluating the quality of

the adaptation performed7

THE ISATINE FRAMEWORK

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

21

Page 22: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Definitions

• Multi-target user interface

– Support different contexts of use

• Plastic

• Adaptive

• Adaptable

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

22

Page 23: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Plastic user interface

• Example: the Virtual keyboard

Donatien Grolaux, Peter Van Roy, Jean Vanderdonckt:

QTk - A Mixed Declarative/Procedural Approach for Designing Executable User Interfaces, Proc. of EHCI’2001

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

23

Page 24: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Plastic User interface

• Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

24

Page 25: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Plasticity

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

25

• NOKIA plastic interface for remote control

Heikki Keränen & Johan Plomp,Adaptive Runtime Layout of Hierarchical UI Components, Proc. of NordiCHI’2002

Page 26: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Adaptivity x Adaptability

• Adaptivity

– Automatic

– Adaptive systems

• Adaptability

– Manually

– Adaptable systems

• Complementary approach

– Mixed-initiative

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

26

Page 27: Human-Computer Interaction: Adaptation, Distribution, Evaluation

PersonalizationX Customization

• Personalization

– Occurs when the system is adapted regarding user characteristics

• Customization

– Occurs when the users are able to modify the application according to their needs

• Both concepts can be considered as a subset of adaptation

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

27

Page 29: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Transformation rules

Add all >>

Add >

<< Remove all

< Remove

>>

>

<<

<

>

<

Group box

Item 1

Item 2

Item 3Item 4

Item 5

Item 6

Item 7Item 8

Item 1Item 2

Item 3

Item 4Item 5

Item 6

Item 7

Item 1

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

Page 30: Human-Computer Interaction: Adaptation, Distribution, Evaluation

• Alternate widgets for the same selection

Ja

nu

ary

Feb

rua

ryM

arc

hA

pril

Ma

yJu

ne

Ju

lyA

ug

ust

Se

pte

mb

er

Octo

be

rN

ove

mb

er

De

cem

ber

J F M A M J J A S O N D

30Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 31: Human-Computer Interaction: Adaptation, Distribution, Evaluation

• Alternate widgets for the same selection

31

January February March

Jan Feb Mar Apr Ma

Mar

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

Page 32: Human-Computer Interaction: Adaptation, Distribution, Evaluation

The Painter’s palette

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

32

+ =

Page 33: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Digital Painter’s palette

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

33

Page 34: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Useful area is waisted!

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

34

Page 35: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Migration• Problem: how to design a UI that adapts to

multiple displays?

Pencil

Palette

Painting

Painting

tool

[Grolaux & Vanderdonckt,2005]

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

35

Page 36: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Migration

36Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 37: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Migration

37Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 38: Human-Computer Interaction: Adaptation, Distribution, Evaluation

UI Migration: DEMIPLAT

• Detach

38Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 39: Human-Computer Interaction: Adaptation, Distribution, Evaluation

UI Migration: DEMIPLAT

• Detach - Migrate

39Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 40: Human-Computer Interaction: Adaptation, Distribution, Evaluation

UI Migration: DEMIPLAT

• Detach - Migrate - Plastify

40Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 41: Human-Computer Interaction: Adaptation, Distribution, Evaluation

UI Migration: DEMIPLAT

• Detach - Migrate - Plastify - Attach

41Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 42: Human-Computer Interaction: Adaptation, Distribution, Evaluation

This is not a floating toolbar

Process

42Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 43: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Computer B

Process

This is migration !

Process

Computer A

43Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 44: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Adaptation: Resources

• Many different resources can be subject to adaptation– Content

• Images

• Audio

• Video

• Text

• Widgets

– Presentation

– Navigation

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

44

Page 45: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Taxonomy – User Context

User

Personal Information

Age Gender Nationality LanguageInstruction

Level

Primary School

Secondary School

High School

Graduation

Post-Graduation

Biometric Information

Weight

Height

Fingerprint

Voice Sample

Iris

Head Diameter

Impairments

Sight

Myopia

Color

Blindness

Blind

Auditory Motor Cognitive

Technological Background

Novice user

Intermediary

Advanced

Preferences

Language

Color

Interaction

Mean

Performance

Modality

Media Type

Emotions

Anxious

Relaxed

Happy

Sad

Status

Online

Offline

Invisible

Stationary

Moving

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

45

Page 46: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Adapting the Presentation

AttachAugmentCollapse to ZoomDetachDistributionFish eyeFull-screenMigrationMirroringNo-scriptOptimize

Overview+DetailPrintRe-distributionRe-moldingRe-sizeReplaceSingle columnTextBox Expansion

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

46

Page 47: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Adapting the Navigation

Accessible NavigationAnnotation of LinksAlphabetical OrganizationAlternative NavigationChronological OrganizationCategorizationDirect GuidanceGeneration of LinksHiding of LinksHierarchical Organization (within a page)

application)Linear StructureMap AdaptationMesh StructurePopularity-based OrganizationSearchSimplify Input ControlsSorting of LinksSuggestion of Links

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

47

Page 48: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Conditional TextDirectionFrame BasedFilteringPersonalizeStretchSuggestion (Recommendation)Re-size

Adapting the Contents

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

48

Type

Taille

Couleur

Format

Film Image texte

24 bit 8 bit 256 C Echelle de gris Noir et blanc

BMP JPEG GIF (256c)

description

de l’image

9x6 pxls

Résolution

7x5 pxls 6x 3,5 pxls 3x 2,1 pxls

200x200 pxls 100x100 pxls 20x20 pxls

Compression

...Toncatureciblée sur le visage

Speed

Quality + -

+-

Page 49: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Adaptation Rules

First order: direct commands

Second order: govern first-order adaptation rules

Third order: adaptation strategies to promote or demote sets of second-order rules

if (platform = ‘mobile’)

then replace a radio box

by a drop-down list

prefer R1, than R2 if (user = ‘expert’) then

reverse the order of

“prefer R1, than R2”

if (platform = ‘mobile’)

and (entries are limited)

then

replace a radio box with

an edit field with codes

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

49

Page 50: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Task

Domain

User

Interaction object

Device

Computing platform

Physical environment

Organization

Application

Presentation

Dialog

Help

Guidance

Tutorial

Design

With respect to what?

What?

Action (Before)

Reaction (After)

Who?User Mixed System

When?Withwhat?

At design timeAt run time

At both design and

run time

How many?

OneSome

Many

For what?

Initiativ

e

Pro

po

sal

Decisio

n

Ex

ecutio

n

Source: Vanderdonckt, J., Grolaux, D., Van Roy, P., Limbourg, Q., Macq, B., Michel, B., A Design Space for Context-Sensitive User Interfaces, Proc. of ISCA 14th Int. Conf. on Intelligent and

Adaptive Systems and Software Engineering IASSE’2005 (Toronto, 20-22 July 2005), International Society for Computers and their Applications, Toronto, 2005, pp. 207-214. (FP6 Similar NoE)

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

50

Page 51: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Adaptation

means

Re-molding

Redistribution

• UI re-molding denotes any UI reconfiguration that is perceivable to the user and that results from the application of transformations on the UI (e.g., insertion, deletion, modification)

• UI redistribution denotes the re-allocation of the UI components to different interaction resources(e.g., for instance in the Painter's palette, the toolbars may move to a PocketPC until the end of the session)

Source: Grolaux, D., Vanderdonckt, J., Van Roy, P., Attach me, Detach me, Assemble me like You Work, Proc. of INTERACT’2005 (Rome, 12-16 September 2005), M.-F. Costabile, F. Paternò (eds.),

Lecture Notes in Computer Science, Vol. 3585, Springer-Verlag, Berlin, 2005, pp. 198-212.

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

51

Page 52: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design SpaceAdaptation

means

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

UI components granularity denotes the smallest software UI unit that can be affected by re-molding and/or re-distributionFor instance, in Qtk, any individual widget is subject to redistributionIn [Czerwinsky, 2000], any window is subject to redistribution

Source: Melchior, J., Grolaux, D., Vanderdonckt, J., Van Roy, P., A Toolkit for Peer-to-Peer Distributed User Interfaces: Concepts, Implementation, and Applications, Proc. of 1st ACM SIGCHI

Symposium on Engineering Interactive Computing Systems EICS’2009 (Pittsburgh, July 15-17, 2009), ACM Press, New York, 2009, pp. 69-78.

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

52

Page 53: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Source: Melchior, J., Grolaux, D., Vanderdonckt, J., Van Roy, P., A Toolkit for Peer-to-Peer Distributed User Interfaces: Concepts, Implementation, and Applications, Proc. of 1st ACM SIGCHI

Symposium on Engineering Interactive Computing Systems EICS’2009 (Pittsburgh, July 15-17, 2009), ACM Press, New York, 2009, pp. 69-78.

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

53

Page 54: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design SpaceAdaptation

means

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

State recovery

granularity

Session levelTask level

Action level

State recovery granularity characterizes the effort users must apply to carry on their activity after adaptation has occurred: at the session level, at the task level or at the action level.For instance, in Rich Internet Applications

Source: Paternò F. Santoro C. Scorcia A. "Preserving rich user interface state in Web applications across various platforms". Engineering Interactive Systems. Second Conference on Human-

Centered Software Engineering, HCSE 2008 TAMODIA 2008 (Pisa, Italy, 25-26 September 2008). Proceedings, pp. 255 - 262. (LNCS, vol. 5247). Springer.

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

54

Page 55: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Adaptation

means

UI

deployment

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

State recovery

granularity

Session levelTask level

Action level

Static

Dynamic

UI deployment denotes the installation of the UI components in its environment: static or dynamic

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

55

Page 56: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Adaptation

means

Context

of useUI

deployment

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

State recovery

granularity

Session levelTask level

Action level

Static

Dynamic

User

Computing platform

Environment

Coverage of context of use denotes the capability of the UI components to be context-aware: user, platform or environment

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

56

Page 57: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design SpaceAdaptation

means

Technological

space coverage

Context

of useUI

deployment

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

State recovery

granularity

Session levelTask level

Action level

Static

Dynamic

User

Computing platform

Environment

Multi-TSInter-TS Intra-TS

Technological space coverage denotes the capability of the UI components to support different working contexts with a set of associated concepts, body of knowledge, tools, required skills, and possibilities.

Source: Kurtev, I., Bézivin, J., and Aksit, M. (2002) Technological Spaces: an Initial Appraisal. CoopIS, DOA'2002 Federated Conferences, Industrial track, Irvine, 2002,.Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

57

Page 58: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Context-Aware Design Space

Adaptation

means

Technological

space coverage

Context

of useUI

deployment

Re-molding

Redistribution

Total level

Dialog space level

Interactor level

UI component

granularity

State recovery

granularity

Session levelTask level

Action level

Static

Dynamic

User

Computing platform

Environment

Multi-TSInter-TS Intra-TS

No Meta-UI

Meta-UI without negotiation

Meta-UI with negotiation

Plastic Meta-UI

Existence of

a meta-UI

Existence of a meta-UI denotes the capability of a UI to provide a UI for controlling its adaptation.For instance, the Collapse-to-zoom meta-UI [Baudish et al., 2004]

Source: Coutaz, J. (2006) Meta-User Interfaces for Ambient Spaces. In: Proc. of 4th Int. Workshop on Task Models and Diagrams for User Interface Design Tamodia’2006 (Hasselt, October 23-24,

2006). Lecture Notes in Computer Science, Vol. 4385, Springer, Heidelberg, pp. 1–15.

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

58

Page 59: Human-Computer Interaction: Adaptation, Distribution, Evaluation

1. Diffie

DiffIE

59

Always on

In-situ

New to you

Non-intrusive

Changes to page since your last visit

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

Page 60: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Large Screens

Adaptive Layout Template for Effective Web Content Presentation in Large-Screen ContextsMichael Nebeling, Fabrice Matulic, Lucas Streit and Moira C. NorrieProc. 11th ACM Symposium on Document Engineering (DocEng 2011), Mountain View, CA, USA, September 2011

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

60

XDStudio

ProgressiveEnhancementCar rental example

Page 61: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Full example of context-aware adaptation

• Walkaware = route planning

• Weatheraware = wheather forecasts

• WalkWeather = both combined

• Adaptation to

– User profile

– Platform configuration

– Location and environmental conditions

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

61

https://play.google.com/store/apps/details?id=be.defimedia.android.walkawaremobile&hl=frhttp://windowsphoneapks.com/APK_WalkAware_Windows-Phone.htmlhttp://www.appdropp.com/ios/walkaware/414443781

Page 62: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Decision Table

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

62

Page 63: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Decision Table

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

63

Page 64: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Architectural Approaches

• GUI generation

• Web services

• Client-server

• Model-view-control

• When?

– At the client (arrival)

– At the proxy (on the way)

– At the server (before it is sent)

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

64

Page 65: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Architectural ApproachUsing Web services

65Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 66: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Weatheraware

• Adaptation: Mobile, Tablet and Desktop PC

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

66

Page 67: Human-Computer Interaction: Adaptation, Distribution, Evaluation

For Desktop

67Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 68: Human-Computer Interaction: Adaptation, Distribution, Evaluation

68Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 69: Human-Computer Interaction: Adaptation, Distribution, Evaluation

69Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 70: Human-Computer Interaction: Adaptation, Distribution, Evaluation

70Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 71: Human-Computer Interaction: Adaptation, Distribution, Evaluation

71Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 72: Human-Computer Interaction: Adaptation, Distribution, Evaluation

72Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 73: Human-Computer Interaction: Adaptation, Distribution, Evaluation

73Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 74: Human-Computer Interaction: Adaptation, Distribution, Evaluation

For Tablet PC and iPads

74Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 75: Human-Computer Interaction: Adaptation, Distribution, Evaluation

75Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 76: Human-Computer Interaction: Adaptation, Distribution, Evaluation

76Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 77: Human-Computer Interaction: Adaptation, Distribution, Evaluation

77Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 78: Human-Computer Interaction: Adaptation, Distribution, Evaluation

78Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 79: Human-Computer Interaction: Adaptation, Distribution, Evaluation

79Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 80: Human-Computer Interaction: Adaptation, Distribution, Evaluation

80Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 81: Human-Computer Interaction: Adaptation, Distribution, Evaluation

81Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 82: Human-Computer Interaction: Adaptation, Distribution, Evaluation

82Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 83: Human-Computer Interaction: Adaptation, Distribution, Evaluation

83Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 84: Human-Computer Interaction: Adaptation, Distribution, Evaluation

84Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 85: Human-Computer Interaction: Adaptation, Distribution, Evaluation

85Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 86: Human-Computer Interaction: Adaptation, Distribution, Evaluation

86Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 87: Human-Computer Interaction: Adaptation, Distribution, Evaluation

87Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 88: Human-Computer Interaction: Adaptation, Distribution, Evaluation

88Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 89: Human-Computer Interaction: Adaptation, Distribution, Evaluation

89Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 90: Human-Computer Interaction: Adaptation, Distribution, Evaluation

90Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 91: Human-Computer Interaction: Adaptation, Distribution, Evaluation

91Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017

(Alger, 23-25 April 2017)

Page 92: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Conclusion of Adaptation:for what?

• Advantages– Improve task efficiency and effectiveness

• Optimizes task completion time and rate

• Reduces work load

– Positively impacts• User’s subjective satisfaction

• Accuracy, human performance

• Predictability, situation awareness

• Shortcomings– Induces user disruption and generates variability

– Loss of control

– Development cost and complexityPremière Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-

25 April 2017)92

Page 93: Human-Computer Interaction: Adaptation, Distribution, Evaluation

Thank you very much for your attention

Questions?

Première Ecole de Printemps Interaction Homme-Machine Réalité Virtuelle & Augmentée IVAR'2017 (Alger, 23-25 April 2017)

93

Page 94: Human-Computer Interaction: Adaptation, Distribution, Evaluation