Dialog design How do we communicate with computers?
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
1
Transcript of Dialog design How do we communicate with computers?
Dialog design
How do we communicate with computers?
Agenda
Exam review Project Part 3 Presentation: Moo Nam Dialogue styles
Command line WIMP Direct Manipulation Speech/Natural Language Pen/Mobile Device
Exam review
Top score: 97 Average: 79 Median: 86
Project Part 3
Due in 3 weeks Choose one design
Go more in depth on details Create testable prototype(s)
Demo for me Create evaluation plan
Discuss in class on 3/30
Dialog Design
How does a user interact with the interface?
Dialog Styles
1. Command languages 2. WIMP - Window, Icon, Menu,
Pointer 3. Direct manipulation
4. Speech/Natural language 5. Pen & PDA
General Issues in Choosing Dialogue Style Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
Command Languages
Earliest UI interaction paradigm Examples: MS-DOS shell, UNIX, Linux Little or nothing is visible so…
Work primarily by recall, not recognition Heavy memory load
Poor choice for novices but...
CL Advantages
Advantages for experts Speed, conciseness
• % ls (hard to beat)
Can express actions beyond a limited set• Flags, piping one command to another
Repetition, extensibility• Scripting, macros
Easier implementation, less overhead Power
• Abstraction, wild cards
CL Dangers
With added power, comes added responsibility and dangerUNIX
• % rm -r *• Deletes every file that you have, and you
can’t get them back
CL Design Goals
ConsistencyHave options and arguments
expressed the same way everywhere Good naming and abbreviations
UNIX fails here because commands were developed by lots of different people at different organizations
• No guidelines provided
General Issues - CL
Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
Direct Manipulation
Definition: 1) Continuous visibility of the objects
and actions of interest 2) Rapid, reversible, incremental
actions whose effect is immediately noticeable
3) Replacement of command language syntax by direct manipulation of object of interest (physical actions, buttons, etc.)
Shneiderman ‘82
DM Essence
Representation of reality that can be manipulated
The user is able to apply intellect directly to the task
The tool itself seems to disappear
Related to Directness Engagement
Direct Manipulation
See pgs 171-174 in DFAB Examples
WYSIWYG editors and word processors
VISICALC - 1st electronic spreadsheetCADDesktop metaphorVideo games
Example: Homefinder
DM Issues
Advantages Flexible, easily
reversible actions helps reduce anxiety in users
WYSIWYG provides context & instant visual feedback
Exploits human use of visual spatial cues
Limits types of errors that can be made
Disadvantages visual representation
not self-explanatory Mouse ops may be
slower than typing Not good at:
Repetition History keeping Certain tasks
(Change all italics to bold)
Abstract elements (variables)
General Issues - DM
Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
WIMP
Windows, Icons, Menus, PointersFocus: Menus, Buttons, Forms
Predominant interface paradigm now (with some direct manipulation added)
Advantages:?
Menus
Advantages: 1 keystroke or mouse operation vs. many No memorization of commands Limited input set
Disadvantages: Less direct user control - have to find correct
menu / menu item Not so readily extensible Slower than keyboarding for experienced
users, at least without accelerators
Menu Items
Organization strategiesCreate groups of logically similar
itemsCover all possibilitiesEnsure that items are non-overlappingKeep wording concise,
understandable
A Good Example
Logical grouping Visual separation of
groups Disabled items
“grayed out” Shortcuts shown … indicates leads to
dialogue
Presentation Sequence
Forms, dialogue boxes, menus Use natural if available
Time• e.g. Breakfast, Lunch, Dinner
Numeric ordering• e.g. Point sizes for font
Other possibilities: Alphabetical Group related items Frequently used first Most important first
Other WIMP issues
Windows managementHow to locate, move, findTransfer information between
IconsNeed graphic design attention
Toolbars
General Issues - WIMP
Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
Natural input
Universal design Take advantage of familiarity,
existing knowledge Alternative input & output Multi-modal interfaces Getting “off the desktop”
When to Use Speech
Hands busy Mobility required Eyes occupied Conditions preclude use of keyboard Visual impairment Physical limitation
Speech Input
Speaker recognitionTell which person it is (voice print)Monitoring, recording
Speech recognitionIdentify wordsIBM ViaVoice, Dragon Dictate, ...
Natural language understanding
Recognition Dimensions
Speaker dependent/independent Parametric patterns are sensitive to
speaker With training (dependent) can get better
Vocabulary Some have 50,000+ words
Isolated word vs. continuous speech Continuous: where words stop & begin Typically a pattern match, no context
usedDid youvs.Didja
Recognition Example:
My flight experience uh, I started off as a private pilot and then I spent 20 years flying as a navigator on C130s in the air force and I currently fly as a first officer on the SAAB 340 turboprop.
my flight experience a and I started off as a private pilot and I spent 20 years flying is a navigator on C-130s in the Air Force and I currently fly as a first officer on the sound 340 turboprop.
And another example
Here here's one of the
thoughts I'd throw in on
that. I'd agree with
Howard that the
combination of checklists
and flows is the way way
to manage things best.
hair and here's one of the
outside troodon and
an eye to agree with
power that the
domination checklists
and flows is the way
that managed things best
Errors
Systems make four types of errors:Substitution - one for anotherRejection - detected, but not
recognizedInsertion - addedDeletion - not detected
Problems with recovery
Natural Language Understanding
Putting meaning to the words
Input might be spoken or typed
Holy grail of Artificial Intelligence problems
Eliza - a “Faked” Understanding Person : Men are all alike.
ELIZA : In what way?Person : They're always bugging us about something or other.ELIZA : Can you think of a specific example?Person : Well, my boyfriend made me come here.ELIZA : Your boyfriend made you come here?Person : He says I'm depressed much of the time.ELIZA : I am sorry to hear that you are depressed.Person : It's true. I am unhappy.ELIZA : Do you think coming here will help you not to be unhappy?Person : I need some help, that much seems certain.ELIZA : What would it mean to you if you got some help?Person : Perhaps I could learn to get along with my mother.ELIZA : Tell me more about your family
http://www-ai.ijs.si/eliza/eliza.html Weizenbaum, J., "ELIZA -- A computer program for the study of
natural language communication between man and machine", Communications of the ACM 9(1):36-45, 1966
NL Factors/Terms
SyntacticGrammar or structure
ProsodicInflection, stress, pitch, timing
PragmaticSituated context of utterance, location,
time Semantic
Meaning of words
SR/NLU Issues
Advantages Easy to learn and
remember Powerful Fast, efficient (not
always) Little screen real estate
Disadvantages Assumes domain
knowledge Doesn’t work well enough
yet Requires confirmation And recognition will
always be error-prone Expensive to implement Unrealistic expectations
can generate mistrust
Speech Output
Tradeoffs in speed, naturalness and understandability
Male or female voice? Technical issues (freq. response of phone) User preference (depends on the application)
Rate of speech Technically up to 550 wpm! Depends on listener
Synthesized or Pre-recorded? Synthesized: Better coverage, flexibility Recorded: Better quality, acceptance
Speech Output
Synthesis Quality depends on software ($$) Influence of vocabulary and phrase choices http://www.research.att.com/projects/tts/demo.php
Recorded segments Store tones, then put them together The transitions are difficult (e.g., numbers)
Designing Speech Interaction Constrain vocabulary
Limit valid commandsStructure questions wisely (Yes/No)Manage the interactionExamples?
Slow speech rate, but concise phrases Design for failsafe error recovery Visual record of input/output Design for the user – Wizard of Oz
Speech Tools/Toolkits
Java Speech SDK FreeTTS 1.1.1
http://freetts.sourceforge.net/docs/index.php
IBM JavaBeans for speech Microsoft speech SDK (Visual Basic, etc.) OS capabilities (speech recognition and
synthesis built in to OS) (TextEdit) VoiceXML
General Issues – Speech/NL
Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
Non-speech audio
Good for indicating changes, since we ignore continuous sounds Traditionally used for warnings, alarms or
status information Provides secondary representation
Supports visual interface Provides information that helps reduce error
Tradeoff in using natural (real) sounds vs. synthesized noises.
Non-speech audio examples
Error ding Info beep Email arriving ding Recycle Battery critical Logoff LogonOthers?
Pen & Mobile dialog
Stylus or finger Tradeoffs of each?
Pen as a standard mouse (doubleclick?) Variety of platforms
Desktop touch screens or input pads (Wacom)
Tablet PCs Handheld and Mobile devices Electronic whiteboards
Platforms often involve variety of size and other constraints
Mobile devices
More common as more platforms available PDA Cell phone Ultra mobile tablets
Smaller display (160x160), (320x240) Few buttons, different interactions
Free-form ink Soft keyboard Numeric keyboard => text Stroke recognition Hand printing / writing recognition
http://www.oqo.com/
http://www.blackberry.com/
http://www.intel.com/design/mobile/platform/umpc.htm
Ultra-Mobile PC (Samsung)Palm Z22 handheldhttp://www.palm.com
Soft Keyboards
Common on PDAs and mobile devicesTap on buttons on screen
Soft Keyboard
Presents a small diagram of keyboard You click on buttons/keys with pen QWERTY vs. alphabetical
Tradeoffs?Alternatives?
Numeric Keypad -T9
Tegic Communications developed You press out letters of your word, it matches the
most likely word, then gives optional choices Faster than multiple presses per key Used in mobile phones http://www.t9.com/
Cirrin - Stroke Recognition
Developed by Jen Mankoff (GT -> Berkeley CS Faculty -> CMU CS Faculty)
Word-level unistroke technique UIST ‘98 paper Use stylus to go
from one letterto the next ->
Quikwriting - Stroke Recogntion
Developed by Ken Perlin
Quikwriting Example
p l
http://mrl.nyu.edu/~perlin/demos/Quikwrite2_0.html
e
Said to be as fast as graffiti, but have to learn more
Hand Printing / Writing Recognition Recognizing letters and numbers and
special symbols Lots of systems (commercial too) English, kanji, etc. Not perfect, but people aren’t either!
People - 96% handprinted single characters Computer - >97% is really good
OCR (Optical Character Recognition)
Recognition Issues
Boxed vs. Free-Form input Sometimes encounter boxes on forms
Printed vs. Cursive Cursive is much more difficult
Letters vs. Words Cursive is easier to do in words vs individual
letters, as words create more context Usually requires existence of a dictionary
Real-time vs. off-line
Special Alphabets
Graffiti - Unistroke alphabet on Palm PDAWhat are your
experienceswith Graffiti?
Other alphabets or purposesGestures for commands
Pen Gesture Commands
-Might mean delete
-Insert
-Paragraph
Define a series of (hopefully) simple drawing gesturesthat mean different commands in a system
Pen Use Modes
Often, want a mix of free-form drawing and special commands
How does user switch modes?Mode icon on screenButton on penButton on device
Error Correction
Having to correct errors can slow input tremendously
StrategiesErase and try again (repetition)When uncertain, system shows list of
best guesses (n-best list)Others??
Free-form Ink
Ink is the data, take as is
Human is responsible forunderstanding andinterpretation
Often time-stamped Applications
Signature verification Notetaking Electronic whiteboards Sketching
Electronic whiteboards
Smartboard and Mimio Can integrate with projection Large surface to interact with
Issues?
http://www.smarttech.com/http://www.mimio.com/
Real paper
Anoto digital paper and pen technology (http://www.anoto.com/)
Issues?
http://www.logitech.com/
Logitech io Digital Writing System
General Issues – Pen input
Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required
Other interesting interactions
Gesture inputSpecialized hardware, or tracking
3D interactionStereoscopic displaysVirtual reality
• Immersive displays such as glasses, caves
Augmented realityHead trackers and vision based
tracking
What’s coming up
Upcoming related topicsMultimodal UIs: Ted3D user interfaces: AmyConversational agents: Evan
Next week: evaluation