25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson...

25 juin 2010

Interactive Genetic Algorithmsfor Creative Enhancement in UI design

Dimitri MassonAlexandre DemeureGaelle Calvary

1

2

Problem statement

• User Interface (UI) design is a creative process

• General question: How to support designers’ exploration of the design space?

Instantmessenger

…

…Design spaceof possible UIs

My research proposal

• Presentation of generated design solutions for the application under study

• Motivation– Compliance with designers’ practice: samples browsing– Sources of inspiration

3

Outline

• State of the art

• Approach

• Prototype

• Early evaluation

• Conclusion and future work

4

State of the art

• Creativity– Creative person

– Creative product

– Creative process

• Creative evolutionary systems– Interactive Genetic Algorithms (IGA)

– IGA & HCI

• UI design process– Design process

– Tools

– Model Based UI

5

State of the art





– IGA & HCI


– Tools

– Model Based UI

6

State of the art





– IGA & HCI


– Tools

– Model Based UI

7

State of the art: Creative process

• A Darwinian process

8

Variation & recombination

Selective retention



9


Selective retention



10


Selective retention

State of the art





– IGA & HCI

• UI design process and places of creativity– Design process

– Tools

– Model Based UI

11

State of the art: IGA

Interactive Genetic Algorithms: Suitable for large problem space for which

– no analytical method can be applied– but candidate solutions can be evaluated

12


Selective retention

Performed by the system Performed by the designer

13

State of the art: IGA

Init a population of candidate solutions

Evaluate candidate solutions

Apply mutation and crossbreedings

Select promising candidate solutions

Design space

0.1

0.2

0.2

0.22

State of the art





– IGA & HCI


– Tools

– Model Based UI

14

• Three contributions: [Monmarché 1999 & Oliver 2002] [Quiroz 2007] [Plessi 2008]– Input: one UI sample– Dimensions of the design space: fonts, colors, layout– Output: definitive UIs for the end-user not for the

designer

15

State of the art: IGA & HCI


The exploration of the design space is limited

• Evolution is limited to predefined elements

• The UI remains basically the same

Absence of high level description of the UI.

16



As a result:


• The UI remains basically the same – No widget substitution

17

dd/mm/yy



As a result:


• The UI remains basically the same – No widget substitution– No UI enrichment

18

dd/mm/yy

dd/mm/yySelect a date:

State of the art: coverage





– IGA & HCI


– Tools

– Model Based UI

19

State of the art: Model based UI

• High level description of the UI: concept & task model

20

Transformation 1

Transformation 2

Transformation N

• Transformations

…

Outline


• Approach

• Prototype


• Conclusion and future works

21

My approach

• Three key points:#1 Combination of IGA and Model Based UI at C&T.

#2 Evolution of transformations from C&T UI

#3 Support from a knowledge database

22

Concept and task (C&T) Transformations(C&T UI)

IGA

Evolve Transformations(C&T UI)

uses

23

My approach: #2 transformation

• Transformation = one tree– Edges: Select C&T elements– Nodes: Translate C&T elements into UI elements

Root of transformation

Select choices

Translate into radiobuttons

Select abstract tasks

Translate into windows

Select descendants abstract tasks

Translate into frameswith a red background

Root of the task

24

My approach: #2 Genetic operations

• Mutation– Edge: changing selector– Node: Changing translation– Add/remove nodes

Select choices

Translate into radiobuttons

Select abstract tasks

Translate into windows

Select descendants abstract tasks

Translate into framesTranslate into comboboxes

Select children abstract tasks

24

25

My approach: #2 Genetic operations

• Crossbreeding

My approach: #3 Knowledge database

• Widget representations for each task

• Attributes for each widget and manipulation function

26

Choice of date

Entry textFormatBackground colorForeground color

CalendarWidthHeight

23/11/08

Outline


• Approach

• Prototype



27

• VIDEO

28

Prototype: Implementation

• TCL

• COMET: a widget toolkit [Demeure 2008]– Task level interactors supporting multiple presentations– Database storing design knowledge

29

Outline


• Approach

• Prototype



30

Early evaluation

• The purpose is to– assess the effectiveness of the approach by collecting as

many ideas and practices as possible– identify new areas for improvement

• The purpose is not to– assess the creativity improvement– evaluate the usability of the prototype

31

Early evaluation: Protocol

• Protocol defined with N. Mandran, sociologist

• Face-to-face semi-structured interviews– Questionnaire about design activity

• Personal activity, Design practices, tools, etc.

– Sketch design of an Instant Messenger• e.g.: Sketches done by E1, Appendix C

– Manipulation of Magellan• Thinking aloud technique

• 11 participants: Designers, HCI experts, Students

• 2h/interview + 4h/post-analysis (> 2 weeks)

32

Early evaluation: Results

I conducted an experiment to test 3 hypothesis

1. Designers find benefits from the IGA approach.– “Not perfect but interesting”.

– The process enable to reject bad UIs.

2. Designers find the evolution of widgets valuable.– Presentation of elements in the actual UI.

3. Designers find the UI enrichment valuable. – Few even notice the enrichment process.

As well as eliciting unforeseen needs about such approach.– Need for layout support.

– Need for User Interaction.

33

Outline


• Approach

• Prototype


• Conclusion and future work

34

Conclusion: My contributions

• Increment on previous approaches: IGA & MBUI– Transformation as individual representation– Widget substitution– UI enrichment

• Early inquiry of designers’ needs – Previous work focus more on the IGA than on the HCI

• Magellan

• [Masson 2010]: ACM EICS, Berlin, June 20-23

35

Conclusion: Difficulties

• Large scope of the state of the art

• Automatic generation of UI is complex

• Conducting experiment is complex– Very long to set up– Require experience

• IGA requires a good modeling of individuals and fine tuning of the process

36

Conclusion: Future work

• Direct edition of the samples presented – Integration of user sketches

• Support several level of fidelity

• Partial UI evaluation and evolution

• New evaluation

37

Questions

38

Research direction

• Presentation of generated design options.

39

• A faire nb de ligne de code.

40

Early evaluation: findings

• Purpose 1: Assess the effectiveness of the approach by collecting as many ideas and practices as possible

• H1: Designers find benefits from the IGA approach

– “Not perfect but interesting”.

– The process enable to reject bad UIs• H2

• H3

– identify new areas for improvement

41

25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson...

Documents

Transcript of 25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson...