Download - Designing Graphical User Interfaces Integrating Gestures

Transcript
Page 1: Designing Graphical User Interfaces Integrating Gestures

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012) 1

Designing Graphical User InterfacesIntegrating Gestures

François Beuvens and Jean Vanderdonckt [email protected]

Researcher at LiLab, http://www.lilab.beUniversité catholique de Louvain (Belgium)

http://www.slideshare.net/jeanvdd/designing-graphical-user-interfaces-integrating-gestures

Page 2: Designing Graphical User Interfaces Integrating Gestures

2

Roadmap- Context- A method for designing graphical user interfaces

integrating gestures- Underlying model- Supporting tool- Ongoing and future works

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 3: Designing Graphical User Interfaces Integrating Gestures

3

Roadmap- Context- A method for designing graphical user interfaces

integrating gestures- Underlying model- Supporting tool- Ongoing and future works

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 4: Designing Graphical User Interfaces Integrating Gestures

4

Motivations for pen-based gestures- Gestures are quick to learn and easy to reproduce- Stylus amplifies the hand movement and is

comfortable to use- Offers high precision- Gives a direct output and visual feedback- Allows recording meta-data

- Pressure- Inclination- Orientation- ...

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 5: Designing Graphical User Interfaces Integrating Gestures

5

Motivations for gesture recognition integration in UI development

- Many existing datasets and algorithms- Developers have little or no time to investigate all these

different algorithms, datasets and variations- Some platforms for gesture recognition are

helpful, yet hard for algorithm selection- Lack of availability- Lack of extensibility and flexibility

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 6: Designing Graphical User Interfaces Integrating Gestures

6

Roadmap- Context- A method for designing graphical user

interfaces integrating gestures- Underlying model- Supporting tool- Ongoing and future works

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 7: Designing Graphical User Interfaces Integrating Gestures

7

UI creation process

- 4 categories of contributors- Engineer/architect: requirements analyser- Designer: in charge of aesthetics- Gesture specialist: devoted to recognition mechanisms- Programmer: coder of the UI

- Work together around a tool/system

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 8: Designing Graphical User Interfaces Integrating Gestures

8

Method: definition of 3 roles

- Interface Users (IU)- End users of the interface

- System Users (SU)- Use the system to produce the user interface

- System Feeders (SF)- Feed the system with knowledge

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 9: Designing Graphical User Interfaces Integrating Gestures

9

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Interface Users define user interface requirements.

Page 10: Designing Graphical User Interfaces Integrating Gestures

10

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Based on the UI requirements, System Users define system requirements.

Page 11: Designing Graphical User Interfaces Integrating Gestures

11

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

If system requirements are met, System Users use the system to produce the user interface.

Page 12: Designing Graphical User Interfaces Integrating Gestures

12

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

If system requirements are not met, based on the system requirements, System Feeders feed the system.

Page 13: Designing Graphical User Interfaces Integrating Gestures

13

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

If UI requirements can be met, System Users can produce the interface so that Interface Users can use it.

Page 14: Designing Graphical User Interfaces Integrating Gestures

14

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

If UI requirements cannot be met, System Users refine system requirements, then go back to system feeding.

Page 15: Designing Graphical User Interfaces Integrating Gestures

15

Method

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

If Interface Users are not satisfied, Interface Users refine their requirements, then go back to system use.

Otherwise, the process ends up

Page 16: Designing Graphical User Interfaces Integrating Gestures

16

Roadmap- Context- A method for designing graphical user interfaces

integrating gestures- Underlying model- Supporting tool- Ongoing and future works

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 17: Designing Graphical User Interfaces Integrating Gestures

17

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 18: Designing Graphical User Interfaces Integrating Gestures

18

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 19: Designing Graphical User Interfaces Integrating Gestures

19

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 20: Designing Graphical User Interfaces Integrating Gestures

20

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 21: Designing Graphical User Interfaces Integrating Gestures

21

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 22: Designing Graphical User Interfaces Integrating Gestures

22

Model

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 23: Designing Graphical User Interfaces Integrating Gestures

23

Roadmap- Context- A method for designing graphical user interfaces

integrating gestures- Underlying model- Supporting tool- Ongoing and future works

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 24: Designing Graphical User Interfaces Integrating Gestures

24

UsiGesture demo:https://vimeo.com/49746767 or http://youtu.be/CZGPbUA8rgs

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 25: Designing Graphical User Interfaces Integrating Gestures

25

Contributions- Theoretical

- Conceptual modeling of 2D pen-based gestures for GUIs- Methodological

- Integration into UsiXML, a XML User Interface Description Language (www.usixml.org)

- Define the role of the System Users, etc.- Empirical

- User study on user satisfaction with gesture integration - Development benefits estimation

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 26: Designing Graphical User Interfaces Integrating Gestures

26

Future work- Empirical: with respect to

- System Feeders: to what extent is it expensive to incorporate new knowledge in UsiGesture

- A new algorithm (recognizer)- A calibration of an algorithm- A new variant of an existing algorithm

- System Users: to what extent is it straightforward to design a GUI based on UsiGesture

- Interface users: do they perceive a better recognition when the right algorithm is used

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)

Page 27: Designing Graphical User Interfaces Integrating Gestures

27

Thank you very much for your attention!

Any question?

See more at:

www.usixml.org, www.usixml.eu

www.lilab.eu, www.lilab.be

ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)