Towards a toolkit forDistributed User Interfaces (DUIs):

Think everything distributed

Jean VanderdoncktLouvain Interaction Lab

Université catholique de LouvainJean.Vanderdonckt@uclouvain.be

Invited conference, University of Elche (Elche, 23 November 2010)

What is the situation today?One user performing one task on one domain in one context of use• The predominant paradigm: single…

• Monitor, device• Computing platform• User• Environment• Context of use• Domain and task

2

one many

Invited conference, University of Elche (Elche, 23 November 2010)

Let us distribute Uis, butone dimension at a time

3

• The predominant paradigm: single…

• Monitor, device• Computing platform• User• Environment• Context of use• Domain and task

one many

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Why take care of multiple monitors?– Device side

• Monitor surface isincreasing, while priceis decreasing

– User side• Users prefer more display surface• User’s productivity increases from 10% to 30%

(although some usability problems are raised)

[Harris,2002]

Projected LCD Pricing 2002-2005

$437$378 $327 $283

$699$597

$510$436

$1,089

$905

$752$625

$0

$200

$400

$600

$800

$1,000

$1,200

2002 2003 2004 2005

$U

S

15" -13.5%

17" -14.6%

18" -16.9%

Invited conference, University of Elche (Elche, 23 November 2010)

Mutiple PCs and Displays

0%

10%

20%

30%

40%

50%

60%

70%

80%

None Multiple monitorsattached to

multiplecomputers.

Laptop anddesktop monitor

connectedtogether.

Dualmon or higher

Config

Pe

rce

nt

Re

sp

on

da

nts

All

Multi-monitor/device

• Why take care of multiple monitors?– User side

[Harris,2002]

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Why take care of multiple monitors?Effects of Display Size on Task Times

0

20

40

60

80

100

120

140

160

DISPLAY

Avera

ge T

ask

Tim

e (

Seco

nds)

Small

Large

[Czerwinsky,2005]

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Why take care of multiple monitors?

[Czerwinsky,2005]

The tasks were easy to perform

0

1

2

3

4

5

Small LargeDisplay SizeA

vera

ge R

ati

ng (

1=

Dis

agre

e,

5=

Agre

e)

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Why take care of multiple monitors?

[Czerwinsky,2005]

I was satisfied with the ease of windows layout

012345

Display Size

Ave

rage

Rat

ing

(1=D

isag

ree,

5=Agr

ee)

Small

Large

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Distribution of UIs across monitors (hardware)– Coupling different monitors to the same

workstation

Hard wired, manual solution

Opportunistic

Lack of reusability

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Distribution of UIs across monitors (hardware)– Coupling different monitors: control rooms

© 2010 www.symbio.pro

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Distribution of UIs across monitors (software)– Partitioning of a window across several screens to

obtain a DUI

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-monitor/device

• Extending a Desktop with another Monitor

Ref_1 SCREEN_1 SCREEN_2

1 2

4 5

3

6

8 9

1 2

4 5

3

6

8 9 7

2

4

8

5

5

5 6

Without anti-aliasing. No pixel information is lost. But proportions are. Rendering quality is quite bad.

1 1-2 3 2-3 With anti-aliasing. Pixel information are lost (e.g. pixel 2). Proportions are quite ok. Rendering quality is good

1-4 3-6

4-7 6-9

7 9 7-8 8-9

7

[Demeure et al.,2004]

Invited conference, University of Elche (Elche, 23 November 2010)

One dimension at a time

• The predominant paradigm: single…– Monitor, device– Computing platform– User– Environment– Context of use– Domain and task

13

one many

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

1970 1980 1990 2000 2010 2020

10

100

1000

10000

(Amount of end users)(Amount of interactive systems)

50

500

5000

50000

1 system for 100 users

1 system for 1 user

2 or 3 systemsfor 1 user

10 to 50 systemsfor 1 user

Source: Forrester Research, 2002

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

• Problem: how to design a UI that takes care of multiple computing platforms?

• Two examples:– The Painter's palette [Grolaux et al., 2005]

– Migration [Molina et al., 2006]

Invited conference, University of Elche (Elche, 23 November 2010)

The Painter’s Palette

+ =

Invited conference, University of Elche (Elche, 23 November 2010)

Digital Painter’s Palette

Invited conference, University of Elche (Elche, 23 November 2010)

Useful area is wasted !

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

• Problem: how to design a UI that takes care of multiple computing platforms?

Pencil

Palette

Painting

Paintingtool

[Grolaux & Vanderdonckt,2005]

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

[Grolaux & Vanderdonckt,2005]

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

Invited conference, University of Elche (Elche, 23 November 2010)

Example using a Pocket PC

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform• DEMIPLAT Principle for migration: Detach

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform• DEMIPLAT Principle for migration: Detach - Migrate

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform• DEMIPLAT Principle for migration: Detach - Migrate - Plastify

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform• DEMIPLAT Principle for migration:

Detach - Migrate - Plastify - Attach

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-platform• This is not a floating bar

Process

Invited conference, University of Elche (Elche, 23 November 2010)

Computer B

Process

Multi-platform

Process

Computer A

Invited conference, University of Elche (Elche, 23 November 2010)

MigriXML Case study

• The user’s environment is a small office, with five different platforms

Invited conference, University of Elche (Elche, 23 November 2010)

One dimension at a time

• The predominant paradigm: single…– Monitor, device– Computing platform– User– Environment– Context of use– Domain and task

31

one many

Invited conference, University of Elche (Elche, 23 November 2010)

Multiple users

• Design guideline #1: provide multiple ways to carry out a task by different users– Interactive systems should better support multiple users

in multi-tasking because group definition changes

Extended from [Mandviwalla & Olfman, 2000]

Invited conference, University of Elche (Elche, 23 November 2010)

Multiple users

• Design guideline #2: support multiple work methods– A same task could involve different sub-tasks depending

on the context

• Design guideline #3:support group evolution– GR1 et GR2 changent en fonction du temps

[Mandviwalla & Olfman, 2000]

Invited conference, University of Elche (Elche, 23 November 2010)

One dimension at a time

• The predominant paradigm: single…– Monitor, device– Computing platform– User– Environment– Context of use– Domain and task

34

one many

Invited conference, University of Elche (Elche, 23 November 2010)

Multi-context

• The Sedan-Bouillon Tourist Application with Plasticity

• The Slides Manager

Title

Content

N A V I G A T I

[Demeure et al.,2004]

Invited conference, University of Elche (Elche, 23 November 2010)

One dimension at a time

• The predominant paradigm: single…– Monitor, device– Computing platform– User– Environment– Context of use– Domain and task

36

one many

Invited conference, University of Elche (Elche, 23 November 2010)

What we have for multi-tasking• Ext/Tk toolkit

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Current shortcomings– #1. No genuine distribution

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Current shortcomings– #2. Distribution is predefined, opportunistic or

optimized for some task

Desktop ComputerTablet PC

Designer's Outpost

[Klemmer et al., 2002]

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Current shortcomings– #3.

Distribution is at the container level (high granularity)

[Czerwinsky et al., 2000]

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Current shortcomings– #4. Distribution is not replicable

[Vandervelpen et al., 2000]

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Towards a Toolkit of distribution primitives (more…)– DISPLAY– COPY– MOVE– REPLACE– MERGE– SWITCH– DISTRIBUTE– …

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Toolkit of distribution primitives: examples– COPY button_1 TO shared_display

Meta-UI [Coutaz 2006] Source UI Shared_display

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Toolkit of distribution primitives: examples– REMOVE outputText FROM "window_comp_0"

Source UI Target UI

Invited conference, University of Elche (Elche, 23 November 2010)

DUI development

• Toolkit of distribution primitives: examples– MERGE "window_comp_0", "window_comp_1"

TO "window_comp_2"

Source UI Target UI

Invited conference, University of Elche (Elche, 23 November 2010)

Conclusion

• Toolkit for DUI distribution primitives– Pros

• Distribution is regulated by primitives in a scenario• Distribution is model-based, at any level of

granularity, is replicable, etc.

– Cons• Still need to develop a model-based approach• What is the best distribution graph?

– Distribution scene– Concrete User Interface Model

• More distribution algorithms are required (optim.)

Invited conference, University of Elche (Elche, 23 November 2010)

For more information and downloading,http://www.isys.ucl.ac.be/bchi

User Interface eXtensible Markup Languagehttp://www.usixml.org

FP7 Selfman projecthttp://www.ist-selfman.org/

Thank you very much!

FP7 Serenoa projecthttp://www.serenoa-fp7.eu