Designing Graphical User Interfaces Integrating Gestures
27
Designing Graphical User Interfaces Integrating Gestures ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012) 1 François Beuvens and Jean Vanderdonckt [email protected] Researcher at LiLab, http://www.lilab.be Université catholique de Louvain (Belgium) http://www.slideshare.net/jeanvdd/designing-graphical-user-inter faces-integrating-gestures
-
Upload
jean-vanderdonckt -
Category
Documents
-
view
1.425 -
download
2
description
The world of today and its new technologies like smartphones, tablets, or any flat interaction surface has increasing the need for graphical user interfaces integrating gestural interaction in which 2D pen-based gestures are properly used. Integrating this interaction modality in streamlined software development represents a significant challenge for designers or developers: it requires important knowledge in gestures management, in deciding which gesture recognition algorithm should be used or refined for which types of gestures, or which usability knowledge should be used for supporting the development. These skills usually belong to experts for gesture interaction and not actors usually involved in user interface design process. In this paper, we present a structured method for facilitating the integration of gestures in graphical user interfaces by describing the roles of the gesture specialist and other stakeholders involved in the development life cycle, and the process of cooperation leading to the creation of a gesture-based user interface. The method consists of three pillars: a conceptual model for describing gestures on top of graphical user interfaces and its associated language, a step-wise approach for defining gestures depending on the end user’s task, and a software that supports this approach. This method is exemplified with a running example in the area of document navigation.
Transcript of Designing Graphical User Interfaces Integrating Gestures
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012) 1
Designing Graphical User InterfacesIntegrating Gestures
François Beuvens and Jean Vanderdonckt [email protected]
Researcher at LiLab, http://www.lilab.beUniversité catholique de Louvain (Belgium)
http://www.slideshare.net/jeanvdd/designing-graphical-user-interfaces-integrating-gestures
2
Roadmap- Context- A method for designing graphical user interfaces
integrating gestures- Underlying model- Supporting tool- Ongoing and future works
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
3
Roadmap- Context- A method for designing graphical user interfaces
integrating gestures- Underlying model- Supporting tool- Ongoing and future works
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
4
Motivations for pen-based gestures- Gestures are quick to learn and easy to reproduce- Stylus amplifies the hand movement and is
comfortable to use- Offers high precision- Gives a direct output and visual feedback- Allows recording meta-data
- Pressure- Inclination- Orientation- ...
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
5
Motivations for gesture recognition integration in UI development
- Many existing datasets and algorithms- Developers have little or no time to investigate all these
different algorithms, datasets and variations- Some platforms for gesture recognition are
helpful, yet hard for algorithm selection- Lack of availability- Lack of extensibility and flexibility
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
6
Roadmap- Context- A method for designing graphical user
interfaces integrating gestures- Underlying model- Supporting tool- Ongoing and future works
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
7
UI creation process
- 4 categories of contributors- Engineer/architect: requirements analyser- Designer: in charge of aesthetics- Gesture specialist: devoted to recognition mechanisms- Programmer: coder of the UI
- Work together around a tool/system
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
8
Method: definition of 3 roles
- Interface Users (IU)- End users of the interface
- System Users (SU)- Use the system to produce the user interface
- System Feeders (SF)- Feed the system with knowledge
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
9
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
Interface Users define user interface requirements.
10
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
Based on the UI requirements, System Users define system requirements.
11
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
If system requirements are met, System Users use the system to produce the user interface.
12
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
If system requirements are not met, based on the system requirements, System Feeders feed the system.
13
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
If UI requirements can be met, System Users can produce the interface so that Interface Users can use it.
14
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
If UI requirements cannot be met, System Users refine system requirements, then go back to system feeding.
15
Method
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
If Interface Users are not satisfied, Interface Users refine their requirements, then go back to system use.
Otherwise, the process ends up
16
Roadmap- Context- A method for designing graphical user interfaces
integrating gestures- Underlying model- Supporting tool- Ongoing and future works
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
17
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
18
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
19
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
20
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
21
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
22
Model
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
23
Roadmap- Context- A method for designing graphical user interfaces
integrating gestures- Underlying model- Supporting tool- Ongoing and future works
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
24
UsiGesture demo:https://vimeo.com/49746767 or http://youtu.be/CZGPbUA8rgs
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
25
Contributions- Theoretical
- Conceptual modeling of 2D pen-based gestures for GUIs- Methodological
- Integration into UsiXML, a XML User Interface Description Language (www.usixml.org)
- Define the role of the System Users, etc.- Empirical
- User study on user satisfaction with gesture integration - Development benefits estimation
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
26
Future work- Empirical: with respect to
- System Feeders: to what extent is it expensive to incorporate new knowledge in UsiGesture
- A new algorithm (recognizer)- A calibration of an algorithm- A new variant of an existing algorithm
- System Users: to what extent is it straightforward to design a GUI based on UsiGesture
- Interface users: do they perceive a better recognition when the right algorithm is used
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
27
Thank you very much for your attention!
Any question?
See more at:
www.usixml.org, www.usixml.eu
www.lilab.eu, www.lilab.be
ACM Int. Conf. on Design of Communication SIGDOC'2012 (Seattle, October 3-5, 2012)
