EPFL, spring 2012 – week 8!mobile i/o

overview

➝  types of mobile device ➝  design challenges ➝  in: text entry ➝  in: overcoming finger occlusion ➝  in: movement ➝  out: overviews ➝  out: off-screen visualizations ➝  out: audio & haptic output

✱this lecture is based on Will Seager’s (UCL) lecture of mobile systems

handhelds: three broad categories

why touchscreens?

➝  larger screens for video, maps, websites, documents etc

➝  easier to point

finger vs stylus

advantages of finger… ➝  can’t lose stylus ➝  fast response to alerts e.g.

phone calls ➝  one-handed operation

disadvantages… ➝  low pointing accuracy ➝  finger occlusion ➝  dirty screen

design challenge: screen space

design challenge: context

in: text entry

➝  > 1 billion text messages sent per day ➝  most common type of mobile interaction ➝  companies are looking for improvements to mobile text

entry methods ➝  many methods currently exist

Text entry research timeline

Mackenzie 2008

three broad categories

Physical Virtual Keyboards

key-based finger-based stylus-based

physical vs virtual keyboards

➝  physical keyboards ➝  mobile phone keypad, mobile qwerty (e.g. Blackberry), 5 button

pager, 3-key date stamp, 1 key input etc

➝  virtual keyboards ➝  aka “soft keyboards” or “on-screen keyboards” ➝  similar to clicking buttons in a GUI ➝  used with a stylus or a finger (but also with other input mechanisms

e.g. eye tracking)

➝  design issues ➝  number of keys, key layout, key size, key shape, activation force,

feedback, disambiguation, language modelling, word prediction etc

number of keys & layout

➝  both physical & virtual keyboards vary in number of keys & layouts

➝  for mobile text input, 26 key qwerty & 9/12 key ABC are by far the most common

➝  other 26-key layout variations include Opti, Dvorak & Fitaly

➝  other 9/12 key variations include 9/12 key qwerty

QWERTY

MOBILE

number of keys & layout

➝  other layouts have been shown to lead to better performance BUT familiarity a crucial factor

opti outperforms qwerty (faster, fewer errors) after a few hours practice

QWERTY

OPTI

number of keys continuum

more less ambiguity continuum

ambiguity

➝  ambiguity occurs if there are fewer keys than symbols in the language => disambiguation is needed to select the intended word from the possibilities.

➝  disambiguation methods include multi-tap and T9

or, is it SUMMER, is it STONES ?

7 PQRS

8 TUV

6 MNO

6 MNO

3 DEF

7 PQRS

?

virtual keyboards

stylus methods ➝  tapping on virtual

keyboards ➝  handwriting recognition

finger methods ➝  tapping on virtual

keyboards

new method for stylus & finger

➝  sliding stylus/finger across the screen

SWYPE

virtual keyboards: feedback

➝  performance with virtual keyboards improves with vibro-tactile feedback

➝  visual and audio feedback may also be useful

➝  finger/thumb occludes ➝  lower precision when

pointing

offset cursor

“shift” target selection technique

➝  “shift” – a technique for enabling fine cursor pointing using fingers

“escape” target selection technique

a)  the user presses his/her thumb near the desired target b)  the gestures in the direction indicated by the target c)  the target is selected, despite several nearby distractors.

behind touch

pseudo transparency

tilting

(Rekimoto uist 96)

“chameleon”

(Fitzmaurice 1993)

peephole display

(Yee 2003)

camera phone based motion sensing

(Whang, Zhai & Canny 2006)

viewing large documents on small displays

overviews

(O’Hara et al 1999) (Woobrock et al 2002)

off-screen visualizations: edge radar

off-screen visualizations

(Baudisch & Rosenholtz 2003 Gustafson et al 2008)

audio & haptic output

➝  non speech audio output ➝  bleeps, earcons,

auditory icons

➝  haptics ➝  refers to interaction

via sense of touch and/or motor activity.

Example earcons from (Brewster et al 2008)

why use audio and/or haptic output?

➝  attention grabbing ➝  saves screen real

estate ➝  can provide

information without requiring visual attention

earpod

head-mounted displays

➝  user can look at environment & display at the same time

➝  potentially good for location based and augmented reality servies as potential for clear link between information & the environment

but… ➝  require separate input device

e.g. trackball or else speech only input

summary: some key points

➝  Key design challenges: small screens & context ➝  mobile text entry research ongoing, in particular for

finger-based input via touch screens ➝  movement as input ➝  importance of overviews when browsing documents on

small screens ➝  off-screen visualizations ➝  audio & haptic output is a way to reduce demand on

visual attention