Dialog Design - Gesture & Pen Interfaces, Mobile Devices
description
Transcript of Dialog Design - Gesture & Pen Interfaces, Mobile Devices
![Page 1: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/1.jpg)
IAT 334
Dialog Design - Gesture & Pen Interfaces, Mobile Devices
1
This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Jim Foley, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, Bruce Walker, and Melody Moore Jackson. Comments directed to [email protected] are encouraged. Permission is granted to use with acknowledgement for non-profit purposes. Last revision: October 2007.
![Page 2: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/2.jpg)
IAT 334
Dialog Styles
1. Command languages 2. WIMP - Window, Icon, Menu, Pointer3. Direct manipulation4. Speech/natural language5. Gesture & pen
2
![Page 3: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/3.jpg)
IAT 334
Agenda
PDA overviewPen input styles
3
![Page 4: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/4.jpg)
IAT 334
How to use a PDA
4
![Page 5: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/5.jpg)
IAT 334
Personal Digital Asst. (PDA)
5
Palm Treo
Apple Newton (1993)
Apple iPhone
Blackberry Curve
![Page 6: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/6.jpg)
IAT 334
PDAs
Now ubiquitousSmall displaysOften touch and pen interfacesSmall thumb-based keyboardsRecent Improvements
Wi-Fi, GPS, more memory, better CPU, better OS, BlueTooth
6
![Page 7: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/7.jpg)
IAT 334
Is it a PDA? Phone? GPS? Camera? Computer?
Line between devices is blurred today
Apple iPhone – phone, MP3 player, PDA, camera
Palm Treo 700w – phone, Windows computer, PDA, camera
Nokia N82 – Phone, GPS, 2 cameras, robot brain
7
![Page 8: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/8.jpg)
IAT 334 8
Cally/Callo
![Page 9: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/9.jpg)
IAT 334
No Shredder…
9
![Page 10: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/10.jpg)
IAT 334
Input Options Pen / Stylus is dominant form Main techniques
Free-form ink Soft keyboard Numeric keyboard => text Stroke recognition - strokes not in the shape of characters Hand printing / writing recognition
Sometimes have or can connect keyboard
10
![Page 11: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/11.jpg)
IAT 334
Free-form Ink
Ink is the data, take as is
Human is responsible forunderstanding andinterpretation
Like a sketch pad
11
![Page 12: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/12.jpg)
IAT 334
Soft Keyboards
Common on PDAs and mobile devices
12
![Page 13: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/13.jpg)
IAT 334
Soft Keyboard
Presents a small diagram of keyboard
You click on buttons/keys with pen or finger
QWERTY vs. alphabetical Tradeoffs? Alternatives?
13
Apple iPhone soft keyboard
![Page 14: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/14.jpg)
IAT 334
Numeric Keypad -T9
Tegic Communications developedYou press out letters of your word, it matches
the most likely word, then gives optional choices
Faster than multiple presses per keyUsed in mobile phoneswww.tegic.com/t9
14
![Page 15: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/15.jpg)
IAT 334
Stroke Recognition - GraffitiGraffiti - Unistroke alphabet on Palm PDA
What are yourexperienceswith Graffiti?
Graffiti demo:http://www.youtube.com/watch?v=R-hbjG2hzuk
15
![Page 16: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/16.jpg)
IAT 334
Stroke Recognition - Cirrin
Developed by Jen Mankoff (GT -> Berkeley CS Faculty)
Word-level unistroke techniqueUIST ‘98 paperUse stylus to go
from one letterto the next ->
Nokia N8 does similarwith QWERTY
layout 16
![Page 17: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/17.jpg)
IAT 334 17
QuikWriting
• Break the gesture into octant components:• Start pen in center,– drag in one of 8 directions– drag along edge– drag to center
http://mrl.nyu.edu/~perlin/demos/quikwriting2_1.html
![Page 18: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/18.jpg)
IAT 334
Hand Printing & Hand Writing Recognition
Recognizing letters and numbers and special symbols Lots of commercial systems English, kanji, etc. Not perfect, but people aren’t either!
People - 96% handprinted single characters Computer - >97% is really good
OCR (Optical Character Recognition)
18
![Page 19: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/19.jpg)
IAT 334
Recognition Issues Off-line vs. On-line
Off-line: After all writing is done, speed not an issue, only quality.Work with either a bit map or vector sequence
On-line: Must respond in real-time - but have richer set of features - acceleration, velocity, pressure
Use best-guess pattern matching, including digram, trigram probabilities and word lists to remove ambiguity 1 I l
19
![Page 20: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/20.jpg)
IAT 334
More Issues
Boxed vs. Free-Form inputSometimes encounter boxes on forms
Printed vs. CursiveCursive is much more difficult to impossible
Letters vs. WordsCursive is easier to do in words vs individual letters,
as words create more context
20
![Page 21: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/21.jpg)
IAT 334
Pen Gesture Commands
21
- Might mean delete
- Insert
- Paragraph
Define a series of (hopefully) simple drawing gesturesthat mean different commands in a system
![Page 22: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/22.jpg)
IAT 334
Pen Use Modes
Often, want a mix of free-form drawing and special commands
How does user switch modes?Mode icon on screenButton on penButton on device
22
![Page 23: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/23.jpg)
IAT 334
Error CorrectionHaving to correct errors can slow input
tremendously
StrategiesErase and try againWhen uncertain system shows list of best guesses...
23
![Page 24: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/24.jpg)
IAT 334
A Different Application
Signature verification
But not with a mouse :)
24
![Page 25: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/25.jpg)
IAT 334
Multi-touch interfaces
Apple iPhone
25
Capacitive touchscreen:http://www.apple.com/iphone/features/index.html#touch
Gestures:flick, tap, pinch, un-pinch
http://www.apple.com/iphone/features/index.html#map
![Page 26: Dialog Design - Gesture & Pen Interfaces, Mobile Devices](https://reader036.fdocuments.net/reader036/viewer/2022081513/5681682f550346895dddd46c/html5/thumbnails/26.jpg)
IAT 334 26
Pen Videos• Pick-and-Drop by Rekimoto
http://www.youtube.com/watch?v=rFw9aMubL-Y• I-Love-Sketch by Seok-Hyung Bae
http://www.youtube.com/watch?v=5Hd2clwURlQ• Jabberstamp by Hayes Raffle
http://www.youtube.com/watch?v=NIe-XDHcsOE• ShapeWriter iPhone App
http://www.youtube.com/watch?v=TOg91yfvZpo• Marginalia : The Hybrid Textbook
http://www.youtube.com/watch?v=zwYdmsdqqTk