Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

77
UIQ | Touch | 2008-10-22

description

http://www.symbian-freak.com/news/008/10/presentation_uiq_about_touch_based_user_interfaces.htm Here is the presentation that was held by UIQ Technology at the Symbian Smartphone Show 2008 in London. It is very detailed and very informative overview of the touchscreen technology as well as different styles of the touch based user interfaces and if you would like to learn more about the technology behind touch user interfaces, limitations, opportunities and the future of touch user interface you should get it. I really enjoyed in all 77 slides and I would highly recommend it to anyone who is interested in touch technology, developers as well as the end users. Download it and enjoy in UIQ’s “tour through the mobile evolution” and don't forget that best parts of this technology will be ported to Symbian Foundation platform! Presentation was the part of the 'Designing Touch-based User Interfaces for Mobile Applications' workshop. It was presented by Matthias Reik, who was one of the key architects of the UIQ platform and Fredrik Sjölin, Senior Interaction Architect of the UIQ platform. Take part of Matthias Reik’s and Fredrik Sjölin’s skills and experience and enjoy their “tour of the mobile evolution” http://www.symbian-freak.com/news/008/10/presentation_uiq_about_touch_based_user_interfaces.htm

Transcript of Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

Page 1: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

UIQ | Touch| 2008-10-22

Page 2: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.2

Welcome!

• General– Mobile phones– Questions

• About UIQ:– Company background– Touch background

• About us:– Fredrik Sjölin, Senior Interaction Architect– Matthias Reik, Chief System Architect

Page 3: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

Introduction

Touch affordance

Invite the user to touch the screen

Benefits with touch

Direct manipulation

Screen technology

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 3

Page 4: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.4

The Fundamentals for User Satisfaction hasn't Changed

Easy to learn & remember how to use

Right set of functions that fulfill your needs

Fun & attractive

Page 5: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.5

Designing for Interaction on the Move

Offer direct access which can increase effectiveness and efficiency Offer direct access which can increase effectiveness and efficiency

Design for quick and short interaction Design for quick and short interaction

Fit everything onto a small screenFit everything onto a small screen Touch demands large hit areasTouch demands large hit areas

TouchTouch

TouchTouch

Easy to learn and useEasy to learn and use Intuitive interaction where it’s obvious how to interact with objects on screen

Intuitive interaction where it’s obvious how to interact with objects on screen

TouchTouch

Page 6: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.6

Touch Ergonomics

Page 7: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.7

Portrait- One-handed Finger Touch

Low precision for finger touch

Some regions of the screen are harder to use

Page 8: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Portrait- Two-handed Use

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.8

Improved precision for finger touch

Not always practicable on the move

Page 9: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Landscape- two-handed use

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.9

Page 10: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Landscape- two-handed use

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.10

Page 11: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.11

Conclusion

One-handed interaction means thumb usage and low finger precision

Decide which use cases that are most important to perform on the move i.e. practical for one handed use

Page 12: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

http://media.bestofmicro.com/4/Z/156995/original/How%20projected%20capacitiv e%20touch%20technology%20works%20-%20Elo%20TouchSystems%20-

%20Tyco%20Electronics-1.jpg

Touch Screens Technologies

• Resistive• Capacitive

– PCT (projected)– RRFC (reversing ramp field)

| © UIQ Technology AB 2008. All rights reserved.

http://media.bestofmicro.com/5/5/157001/origi nal/How%20AccuTouch%20five-

wire%20resistive%20touch%20technology%2 0works%20-%20Elo%20TouchSystems%20-

%20Tyco%20Electronics.pnghttp://www.wacom-

components.com/english/technology/index.html

29 October 2008 12

Page 13: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Resistive vs. Capacitive

Resistive

Pros:• Low Price (~$1) due to competition • High Resolution• Small-target activation• Not affected by dirt, dust, water or light

Cons:• Image clarity • Durability (PET plastic front)• Not good for finger touch

Capacitive

Pros:• Very high durability (depending on the

cover material)• No optical loss (longer battery life, higher

contrast)• Multi-touch (depending on the controller)• Good for finger touch

Cons:• Higher costs, but now rapidly dropping from

$6-10 to $2-3• No stylus (not good for handwriting

recognition).

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 13

Page 14: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Touch Screens Technologies

• Optical– Infrared– Optical Imaging

• Magnetic– EMR (electro-magnetic resonance)

• Acoustic:– SAW (surface acoustic wave)– APR (acoustic wave recognition)– DST (dispersive signal technology)

• Others

| © UIQ Technology AB 2008. All rights reserved.

http://www.japancorp.net/Article.Asp?Art_ID=15247

29 October 2008 14

Page 15: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Touch Screens Technologies Future

• Combination– Will compensate for individual problems

• Integration– Tighter integration with screen– Touch logic integrated

• Gestures handled already by hardware

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 15

Page 16: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Screen technology

Dot pitchdecrease

Resolutionincrease

| © UIQ Technology AB 2008. All rights reserved.

Year: 2000dot pitch: 0.22-0.26 mmscreen size: 5.5 cm x 7.5 cm

20060.14-0.165 mm3.5 x 5.0 cm2

Today0.08 mm3.5 cm x 5.0 cm

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 16

Page 17: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Size of Hit Area Evaluation

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.17

• Prototype• Random number combination• Measured time to enter a sequence of

numbers• User opinion ranked on scale 0-10

Page 18: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.18

Average Time

0

0,5

1

1,5

2

2,5

5*5 hit,5*5 Button 7*7 hit,7*7 button 9*9 hit, 9*9 button 7*7 hit, 5*5 button 9*9 hit, 7*7 button

1

TimeUse no less than 7x7 mm target size

Page 19: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

User points

4

4,5

5

5,5

6

6,5

7

7,5

8

5*5 hit,5*5 Button 7*7 hit,7*7 button 9*9 hit, 9*9 button 7*7 hit, 5*5 button 9*9 hit, 7*7 button

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.19

Use white space between objectsLike

Page 20: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.20

*Target Size Study for One-Handed Thumb Use on Small Touchscreen Devices. http://www.mediateam.oulu.fi/publications/pdf/1076.pdf

Page 21: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.21

Conclusion

Minimum 7.7 x 7.7 mm hit area

Whitespace between objects

The graphic can preferably be smaller than the hit area

1 = most comfortable region

8 4 5

6 1 2

9 3 7

Page 22: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.22

Resolution Versus Physical size

WVGA 480x864

3.2”(16:9)

QVGA 240x320

2.6”(4:3)

QVGA 240x320

2.4”(4:3)

HVGA 320x480

3.0”(3:2)

50x50 pixels50x50 pixels

7,7 mm 8,25 mm 6,6 mm 4,1 mm

Page 23: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.23

Resolution Versus Physical size

WVGA 480x864

3.2”(16:9)

QVGA 240x320

2.6”(4:3)

QVGA 240x320

2.4”(4:3)

HVGA 320x480

3.0”(3:2)

7,7 mm7,7 mm

50x50 pixels 46x46 pixels 58x58 pixels 93x93 pixels

Page 24: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Design by Pixels (static)• Allows different Industrial Designs

(with different screen sizes)• Easy for programmers

(natural unit)• Easy for UI & Graphic designers

(allows pixel perfect results)

• Controls can become unusable• Fonts can become unreadable• Wasted screen estate

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 24

Page 25: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Design by Size (dynamic)

• Easy for Usability Engineers

• The screen is build up of pixels (Half pixel lines won’t look good)

• Unpredictable outcome• Complete re-validation of the software

stack (extremely expensive)

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 25

Page 26: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 26

Combination is the solution

• Use static elements but combine dynamically• Logical dimensions

• Icon sizes• Font sizes• Line heights• Margins

• Relate things to logical sizes• Allow for product specific optimizations

– UI configurations

| © UIQ Technology AB 2008. All rights reserved.

Page 27: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.27

Touch Affordance

Page 28: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Use space between objects

Use space between objects

29 October 2008 | © UIQ Technology AB 2008. All rights reserved.28

Invite to Touch

Objects look touchable

Objects look touchable

Reduce number of objects

Reduce number of objects

Obvious size & shape

Obvious size & shape

Page 29: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.29

Invite to Touch

Use space between objects

Use space between objects

Objects look touchable

Objects look touchable

Reduce number of objects

Reduce number of objects

Obvious size & shape

Obvious size & shape

Page 30: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.30

Invite to Touch

Use space between objects

Use space between objects

Objects look touchable

Objects look touchable

Reduce number of objects

Reduce number of objects

Obvious size & shape

Obvious size & shape

Page 31: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.31

Feedback

Page 32: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.32

Feedback

Page 33: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.33

Performance examples

From when a pen/finger touches the display until feedback is provided

< 50 ms

Open an applications from main menu. New view is completely updated

< 500 ms

Page 34: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Key Event

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 34

Page 35: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Pointer Event

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 35

Page 36: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

Con

trol S

tack

Event Path Key Event

12

3

4

PointerEvent

12

3

| © UIQ Technology AB 2008. All rights reserved.29 October 2008 36

Page 37: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

PointerDown Handling

| © UIQ Technology AB 2008. All rights reserved.

Child Control

Control 1(initially non-focused)

PointerDown

ControlObserver

SetFocus(ETrue)

RequestFocus

Control 2(initially focused)

SetFocus(EFalse)

HandleE

vent SetFocus

PrepFocusTransition

PointerDown

HandleP

ointerEvent

HitTest

...

SetFocus

ProcessP

ointerEvent() Rect

Contains

PointerGrab(ETrue)

29 October 2008 37

Page 38: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

PointerUp Handling

| © UIQ Technology AB 2008. All rights reserved.

Child Control

Control 1(focused)

PointerUp

ControlObserver

PointerUp

HandleP

ointerEvent

HitTest

...

ProcessP

ointerEvent() Rect

Contains

PointerGrab(EFalse)

29 October 2008 38

Page 39: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|

What should you do?

• Touch down– Take Active Focus

• Touch release– Perform Function

• Touch down and cancel– Dragging away cancels the function

• Touch down and drag– Selecting multiple objects in lists– Highlighting text

• Touch down and hold:– Hidden functionality

(simulating right mouse)

| © UIQ Technology AB 2008. All rights reserved.

Performance Tips

• Goal:– Fast feedback

• Possible solutions:– Remove unnecessary tasks– Move expensive tasks– Optimize– Fake– Less functionality

29 October 2008 39

Page 40: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.40

Touch Interaction Styles

Page 41: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.41

Touch Screen Products

No navigation hardware keysFocus or non focus driven UI

Navigation hardware keysFocus driven UI

Page 42: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.42

Touch Interaction Style

“Double tap” “Drill down”

Focus driven UI

“Drill down”

Non focus driven UI

Page 43: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.43

• Visible focus

Touch Interaction Style “Double tap”

Page 44: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.44

• Tap an object sets the focus

Page 45: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.45

• Object specific commands are then accessed from a toolbar

Page 46: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.46

• Or from the Options menu

Page 47: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.47

• Or from the Options menu

Page 48: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.48

• A second tap on the focused object launch the details

Page 49: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.49

• Visible focus

Page 50: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.50

• Tap an object sets the focus

Page 51: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.51

• Tap an object sets the focus

Page 52: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.52

• A second tap on the focused object launch a pop-out menu where object commands are available

Page 53: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.53

• A second tap on the focused object launch a pop-out menu where object commands are available

Page 54: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.54

• Object specific commands are also available from the Options menu

Page 55: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.55

• Object specific commands are also available from the Options menu

Page 56: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.56

Touch Interaction Style

“Double tap” “Drill down”

Focus driven UI

“Drill down”

Non focus driven UI

Page 57: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.57

• Immediate access to the primary action on tap

Touch Interaction Style “Drill down”

Page 58: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.58

• Immediate access to the primary action on tap

Page 59: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.59

• Immediate access to the primary action on tap

Page 60: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.60

• Immediate access to the primary action on tap

Page 61: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.61

• Immediate access to the primary action on tap

Page 62: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.62

• Alternative actions available for each object

Page 63: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.63

• Alternative actions available for each object

Page 64: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.64

• Non focus UI Greatly decreased Options menu

Page 65: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.65

• Non focus UI Greatly decreased Options menu

Page 66: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.66

Benefits with Touch

Page 67: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.67

Place Functions on Screen, Not in a Drawer

• Touch gives direct access to primary functions

• A toolbar is an open drawer on screen

Page 68: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.68

Direct Touch Manipulation

• But the most effective way is direct and natural manipulation with the objects on screen

• Touch interaction involve your muscle memory together with your cognitive memory

Page 69: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.69

• Drag down

Page 70: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.70

• Set picture

Page 71: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.71

• Select

Page 72: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.72

• Drag & drop

Page 73: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.73

• Grab & drag

Page 74: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.74

• Pan

Page 75: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 75

Conclusion

• The future:– Mobiles with touch screen– New hardware– New interaction paradigms

• Stay close to the platform you are targeting– Caution with using the wrong paradigm– Caution with new innovative interaction models.

• Prepare your application/platform– Consider the following when you write your model

• (Multi-) Touch• Gestures• KeyEvents

– Thin UI layer

| © UIQ Technology AB 2008. All rights reserved.

Page 76: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.76

Designing for touch

• Reduce number of objects and functions on screen• Give access to the most used actions directly on screen• Represent them in a recognizable way• Design for direct manipulation with the objects on screen

Page 77: Technology behind touch user interfaces - Limitations and Opportunities by symbian-freak.com

|29 October 2008 | © UIQ Technology AB 2008. All rights reserved.77

Questions?

UIQ Technology ABSoftCenter VIIIS-37225 RonnebySweden

Phone: +46 457 46 47 00Email: [email protected]: www.uiq.com