Command and Natural Languages Interaction Devices (Shneiderman and Plaisant Chs. 7 and 8) from 8.
-
Upload
clare-butler -
Category
Documents
-
view
226 -
download
0
Transcript of Command and Natural Languages Interaction Devices (Shneiderman and Plaisant Chs. 7 and 8) from 8.
Command and Natural LanguagesInteraction Devices
(Shneiderman and Plaisant Chs. 7 and 8)
from http://wps.aw.com/aw_shneider_dtui_7, 8
Overview
• “Languages” for interaction - styles– Notion of language at core of cs, here, one means of interaction
• Some things about command languages– In practice blurs distinction between interaction and programming– Pretty old school, but part of complete picture – “quick view” mode
• Interaction devices– Not too glamorous maybe, …, but where hci happens
• Pointing devices– Taxonomy and tasks– Motor control of pointing device
• A component of Model Human Processor• Fitts’s law, power law of practice
• Speech interfaces
Recall, Interaction Frameworks
• Interaction:– Communication between
user and system
• Why have a framework?– Allows “precision” in
accounting for differences• E.g., gulfs of execution and
evaluation
– Presents global view• All elements receive
attention
Task
“wor
k on
task
”“com
mands”
System
User
“gives”“per
form
s”
“feedback”
Physical System
Goals
Gulf of Execution
Gulf of Evaluation
And … Interaction Styles
• Interaction: – dialogue between computer
and user
• Interaction styles– Menus– Form-fills (and spreadsheets)– Dialog boxes– Question/answer and query– “WIMP”– Command line interface– Natural language
Task
“wor
k on
task
”“com
mands”
System
User
“gives”“per
form
s”
“feedback”
S
O
U
I
system
output
user
input
And … Command Line Interfaces
• Way of expressing instructions to computer directly– Using function keys, single characters, short
abbreviations, whole words, or a combination– Suitable for repetitive tasks– Better for expert users than novices– Offers direct access to system functionality– Command names/abbreviations should be
meaningful
• Typical example: the Unix system (shell)
• Recall, user intentions translated into actions at interface
translated into alterations of system state
reflected in the output display interpreted by user
• Are gulfs likely to be large or small?
S
O
U
I
system
output
user
input
Physical System
Goals
Gulf of Execution
Gulf of Evaluation
And … Natural Language
• Certainly, familiar to user
• Speech recognition or typed natural language
• Problems (next slide):– Vague, ambiguous, hard to do well– … and all the “deep” problems of true natural language understanding
• E.g., world knowledge, user’s knowledge and intention– … and probably don’t want to have a “conversation” with most (of today’s) computer
system• “just do it”
• Solutions– Try to understand a subset – Pick out key words
Command and Natural Languages
• Command languages are both the best and worst interaction style…– How worst?– How best?
• Natural language is easy for people, but harder than you might think for computers …
– “Promise of the future …”
Task
“wor
k on
task
”“com
mands”
System
User
“gives”“per
form
s”
“feedback”
About Languages
• In computer science, formal languages great success story– In some ways “defines the discipline”– There are descriptive languages, symbol systems, programming languages, …
• According to Shneiderman, basic goals of (any) language design– Precision – Compactness – Ease in writing and reading – Speed in learning – Simplicity to reduce errors – Ease of retention over time
• According to Shneiderman, higher-level goals of language design– Close correspondence between reality and the notation – Convenience in carrying out manipulations relevant to user's tasks – Compatibility with existing notations – Flexibility to accommodate novice and expert users – Expressiveness to encourage creativity – Visual appeal
Functionality to Support User’s TasksGuidelines for Language Design – in fact more general
• Excessive functionality not only does not help, it in fact hinders
• Language designers should:– Determine functionality of the system by studying users' task domain
– Create a list of task actions and objects
– Abstract this list into a set of interface actions and objects
– Represent low-level interface syntax
– Create a table of user communities and tasks, with expected use frequency
– Determine hierarchy of importance of user communities (i.e. prime users)
– Evaluate destructive actions (e.g. deleting objects) to ensure reversibility
– Identify error conditions and prepare error messages
– Allow shortcuts for expert users, such as macros and customizing system parameters
Command-Organization Strategies
• A unifying interface concept or “metaphor” (or “view”) aids • Learning • Problem solving • Retention
• Designers often err by choosing a metaphor closer to machine domain than to the user's task domain
• Simple command set:
• Each command is chosen to carry out a single task• Number of commands match the number of tasks
• For small number of tasks, this can produce a system easy to learn and use• E.g. vi and emacs editors
Command-Organization StrategiesCommand plus arguments/options
• Shneiderman:
• Follow each command by one or more arguments that indicate objects to be manipulated
– COPY FILEA, FILEB – DELETE FILEA – PRINT FILEA, FILEB, FILEC
• Keyword labels for arguments are helpful for some users– COPY FROM = FILEA TO = FILEB.
• Commands may also have options to indicate special cases– E.g., to produce 3 copies of FILEA on the printer in the headquarters building
• PRINT/3,HQ FILEA • PRINT (3, HQ) FILEA • PRINT FILEA -3, HQ
• Error rates and need for extensive training increase with number of options
Benefits of Structure
• Learning, problem solving, and memory facilitated by meaningful structure
• Beneficial for everything – task concepts – computer concepts – syntactic details of command languages
• E.g., consistency of argument order in command language:
Inconsistent order of arguments Consistent order of argumentsSEARCH file no, message id SEARCH message id, file noTRIM message id, segment size TRIM message id, segment sizeREPLACE message id, code no REPLACE message id, code noINVERT group size, message id INVERT message id, group size
Hierarchical Command Structure
• When possible hierarchical organization of commands powerful
• Full set of commands is organized as tree structure
– Each action • can be applied to
– Each object • for
– Each destination
• So,– 5x3x4 = 60 tasks with – 5 command names and – 1 type of rule of formation
Action Object Destination
CREATE File File
DISPLAY Process Local printer
REMOVE Directory Screen
COPY Remote printer
MOVE
Symbols versus Keywords
• Historically interesting (maybe) study• Command structure affects performance
Symbol Editor Keyword Editor
FIND:/TOOTH/;-1 BACKWARD TO "TOOTH"
LIST;10 LIST 10 LINES
RS:/KO/,/OK/;* CHANGE ALL "KO" TO "OK"
Naming and Abbreviations
• Create consistency or obvious strategy for construction of command abbreviations
• Specificity Versus Generality
Infrequent, discriminating words insert delete
Frequent, discriminating words add remove
Infrequent, nondiscriminating words amble perceive
Frequent, nondiscriminating words walk view
General words (frequent, nondiscriminating) alter correct
Nondiscriminating nonwords (nonsense) GAC MIK
Discriminating nonwords (icons) abc-adbc abc-ab
Abbreviation StrategiesGuidelines
• Forming commands• Consider copy, print, delete
1. Simple truncation:
The first, second, third, etc. letters of each command.
2. Vowel drop with simple truncation:
Eliminate vowels and use some of what remains.
3. First and last letter:
Since the first and last letters are highly visible, use them.
4. First letter of each word in a phrase:
Use with a hierarchical design plan.
5. Standard abbreviations from other contexts:
Use familiar abbreviations.
6. Phonics: Focus attention on the sound.
Command-language GuidelinesShneiderman
Natural Language in ComputingCommon Uses
• Natural-language interaction
• Natural-language queries and question answering
• Text-database searching
• Natural-language text generation
• Adventure games and instructional systems
• But, … a difficult problem …
Interaction Devices
Interaction Devices
• It was pretty rugged when …
– There were switches and cards for input• Then there was the keyboard!
– … and there still is
– Then there were menus and icons• And a pointer to select
– … and there still is
• Much of what follows may not seem “cutting edge”, but it is about the 90+% of time not spent on the cutting edge
• And incremental gains on that scale are important
Keyboard Layouts - QWERTY
• 1870 Christopher Latham Sholes
• Good mechanical design and a clever placement of letters that slowed down users enough that key jamming was infrequent !
• Put frequently used letter pairs far apart, thereby increasing finger travel distances
Keyboard Layouts - Dvorak
• 1920, Dvorak
• Reduces finger travel distances by at least one order of magnitude
• Acceptance has been slow despite dedicated efforts of devotees
• Takes about 1 week of regular typing to make the switch, but few users have been unwilling to invest effort
QWERTY and Dvorak Keyboards
• .
Keyboard Layouts - ABCD
• 26 letters of the alphabet laid out in alphabetical order
• Easier for nontypists to locate keys
FYI – Keyboards - Keys
• Tactile and audible feedback important
• Certain keys should be larger (e.g. ENTER, SHIFT, CTRL)
• Some keys require state indicator, such as lowered position or light indicator (e.g. CAPS LOCK)
• Key labels should be large, meaningful, permanent
• Some "home" keys may have additional features, such as deeper cavity or small raised dot, to help user locate their fingers properly (no standard for this)
• Details– 1/2 inch square keys – 1/4 inch spacing between keys – Slight concave surface – Matte finish to reduce glare finger slippage – 40- to 125-gram force to activate – 3 to 5 millimeters displacement
FYI – Keyboards - Function Keys
• What do you think? … pros and cons?
FYI – Keyboards - Function Keys
• Can reduce number of keystrokes and errors – Typically simply labeled F1, F2, etc, though some may also have meaningful
labels, such as CUT, COPY, etc. • Lights next to keys used to indicate availability of the function, or on/off
status – Alternative is to use closer keys (e.g. ALT or CTRL) and one letter to indicate
special function
• Users must either remember each key's function, identify them from the screen's display, or use a template over the keys in order to identify them properly
– Meaning of each key can change with each application – Placement on keyboard can affect efficient use
• Frequent movement between keyboard home position and mouse or function keys can be disruptive to use
Small Devices – Small Keyboards
• RIM Blackberry –reduced-size keyboard• Nokia device - French AZERTY keyboard • Larger keyboard uses longer dimension of device and can be slid back into device
Keyboards - Bizarre
• Dasher predicts probable characters and words as users make their selections in a continuous two-dimensional stream of choices
Other Text Entry Methods
• Another method is to handwrite on a touch sensitive surface, typically with a stylus using Graffiti® on the Palm devices
Pointing DevicesShneiderman
• Taxonomy of devices – Shneiderman
Interaction Tasks for Pointing Device
• 1. Select: user chooses from a set of items
– E.g., traditional menu selection, identification of a file in a directory, or marking of a part in an automobile design
• 2. Position: user chooses a point in a one-, two-, three-, or higher-dimensional space
– E.g., create a drawing, to place a new window, or to drag a block of text in a figure
• 3. Orient: user chooses a direction in a two-, three-, or higher-dimensional space
– E.g., direction may simply rotate a symbol on screen, indicate direction of motion, or control operation of a robot arm
• 4. Path: user rapidly performs a series of position and orient operations.
– E.g., may be realized as a curving line in a drawing program, the instructions for a cloth cutting machine, or the route on a map
• 5. Quantify: user specifies a numeric value
– E.g., usually a one-dimensional selection of integer or real values to set parameters, such as the page number in a document, the velocity of a ship, or the amplitude of a sound
• 6. Text: user enters, moves, and edits text in a two-dimensional space
– Pointing device indicates the location of an insertion, deletion, or change– E.g., more elaborate tasks, such as centering; margin setting; font sizes; highlighting,
boldface or underscore; and page layout
Direct-control Pointing Devices
• Old School
• Lightpen– Point to a spot on a screen and to
perform a select, position, or other task
– Allows direct control by pointing to a spot on the display
– Button for user to press when cursor is resting on desired spot on the screen
– Disadvantages: • Users' hands obscured part of screen• Users had to remove their hands from
the keyboard• Users had to pick up lightpen
Direct-control Pointing Devices
• Touchscreen
– Allows direct control touches on screen• Drag cursor with finger movement to position
– Early designs criticized for causing fatigue, hand-obscuring-the-screen, hand-off-keyboard, imprecise pointing, and smudging of display
– Lift-off strategy enables users to point at single pixel
– Can produce varied displays to suit task
– Can be fabricated integrally with display surfaces
Direct-control Pointing Devices
• Multi-touch– Pervasive
• Software layer, e.g., iPhone below
Direct-control Pointing Devices
• Multi-touch walls
Direct-control Pointing Devices
• Multi-touch walls– Promise of the future?
Indirect Pointing Devices• Mouse
– Hand rests in a comfortable position, buttons on the mouse are easily pressed, even long motions can be rapid, positioning can be precise
• Trackball– Usually implemented as a rotating ball 1 to 6
inches in diameter that moves a cursor
• Joystick– Useful for tracking purposes
• Graphics tablet– Touch-sensitive surface separate from screen
• Touchpad– Built-in near keyboard, offers the convenience
of touchscreen while keeping user's hand off display surface
And …
• Enhances rapid positioning and selection
Motor Processing
• Motor processor can operate in two ways
• Open-loop control– Motor processor runs a program by itself – no feedback about correctness
– Maximum rate, cycle time is Tmotor ~ 70 ms
• Closed-loop control– Muscle movements (or their effect on the world) are perceived by cognitive
system and compared with desired result
– cycle time is Tprocess + Tcognitive + Tmotor ~ 240 ms
• Experiment: Scribble without looking and trying to stay in lines
Fitts’s Law
• Fitts’s Law– Fundamental law of human sensory-motor system
• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-391.
– E.g., for direct (reach) and mouse use
– Demo: http://www.tele-actor.net/fitts/index.html - good
– Demo: http://www.millisecond.com/products/demos/fitts/fitts.web - runs here
Fitts’s Law - demo
• Fitts’s Law– Fundamental law of human
sensory-motor system• Fitts, P. M. (1954). The information
capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-391.
– E.g., for direct (reach) and mouse use
– Demo: http://www.tele-actor.net/fitts/index.html
Fitts’s Law - demo
• Fitts’s Law– Fundamental law of human
sensory-motor system• Fitts, P. M. (1954). The information
capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381-391.
– E.g., for direct (reach) and mouse use
– Demo: http://www.tele-actor.net/fitts/index.html
Fitts’s Law
• Fitts’s Law– Fundamental law of human sensory-motor system– E.g., for direct (reach) and mouse use– The time to acquire a target is a function of distance to and width (size) of target
• Time T to move your hand to a target of size S at distance D away:– T = RT + MT
= a + b*log2(1 + D/S)
– Depends only on index of difficulty log(2D/S)• E.g., if D is large and S small, above large – difficult• Conversely if D is small and S large, above small – less diff.
Explananation of Fitts’s Law
• Moving hand to a target is closed-loop control
• Each cycle covers remaining distance D, with error εD
Implications of Fitts’s Law
• Buttons, etc. should be reasonable size; – hard to click small targets.
• Edges and corners of the computer display are easy to reach– Mac single menubar better than multiple Windows menubars– Also, pointer is "caught" at the edges
• Popup menus can usually be opened faster than pull-down menus– User avoids movement
• Pie menu items are typically selected faster than linear menu items– Small distance from the center of the menu – Wedge-shaped target areas are large
Power Law of Practice
• Time to do a task decreases with practice
– Involves all of perceptual-cognitive-motor system
• Time Tn to do a task the nth time:– Decaying exponential rate
– Tn = T1n α
– α is typically 0.2-0.6
• Example:– Novices get rapidly better at task with
practice, but performance “levels off”– Though still increasing
End?
Novel DevicesShneiderman miscellania
1. Foot controls2. Eye-tracking3. Multiple-degrees-of-freedom devices4. DataGlove5. Haptic feedback6. Bimanual input7. Ubiquitous computing and tangible user interfaces8. Handheld devices9. Smart pens10. Table top touch screens11. Game controllers
Novel devices (cont.)
8-50
Speech and Auditory Interfaces
• “Speech recognition still does not match the fantasy of science fiction”
– Turns out to be a hard problem ...– Demands of user's working memory – Background noise problematic – Variations in user speech performance impacts effectiveness – Bost useful in specific applications, such as to benefit handicapped users
Speech and auditory interfaces (cont.)
Discrete Word Recognition
• Discrete word recognition
– recognize individual words spoken by a specific person; can work with 90- to 98-percent reliability for 20 to 200 word vocabularies
– Speaker-dependent training, in which the user repeats the full vocabulary once or twice
– Speaker-independent systems are beginning to be reliable enough for certain commercial applications
– been successful in enabling bedridden, paralyzed, or otherwise disabled people
– also useful in applications with at least one of the following conditions: • speaker's hands are occupied • mobility is required • speaker's eyes are occupied • harsh or cramped conditions preclude use of keyboard
– voice-controlled editor versus keyboard editor • lower task-completion rate • lower error rate
– use can disrupt problem solving
Speech and auditory interfaces (cont.)
• Continuous-speech recognition – Not generally available:
• difficulty in recognizing boundaries between spoken words • normal speech patterns blur boundaries • many potentially useful applications if perfected
• Speech store and forward – Voice mail users can
• receive messages • replay messages • reply to caller • forward messages to other users, delete messages • archive messages
• Systems are low cost and reliable.
Speech and auditory interfaces (cont.)
• Voice information systems– Stored speech commonly used to provide information about tourist sites,
government services, after-hours messages for organizations– Low cost– Voice prompts– Deep and complex menus frustrating– Slow pace of voice output, ephemeral nature of speech, scanning and
searching problems– Voice mail– Handheld voice recorders– Audio books– Instructional systems
Speech and auditory interfaces (cont.)
• Speech generation – Michaelis and Wiggins (1982) suggest that speech generation is "frequently
preferable" under these circumstances:
• The message is simple. • The message is short. • The message will not be referred to later. • The message deals with events in time. • The message requires an immediate response. • The visual channels of communication are overloaded. • The environment is too brightly lit, too poorly lit, subject to severe
vibration, or otherwise unsuitable for transmission of visual information. • The user must be free to move around. • The user is subjected to high G forces or anoxia
Speech and auditory interfaces (cont.)
• Audio tones, audiolization, and music
– Sound feedback can be important: • to confirm actions • offer warning • for visually-impaired users • music used to provide mood context, e.g. in games • can provide unique opportunities for user, e.g. with simulating various
musical instruments
Displays – Small and Large
• Display is primary source of feedback to user from computer
• Important features:
• Physical dimensions (usually the diagonal dimension and depth)• Resolution (the number of pixels available)• Number of available colors, color correctness• Luminance, contrast, and glare• Power consumption• Refresh rates (sufficient to allow animation and video)• Cost• Reliability
Displays – Small and Large (cont.)
Usage characteristics distinguish displays:• Portability• Privacy• Saliency• Ubiquity• Simultaneity
8-59
Display Technologies• Monochrome displays
– are adequate, and are attractive because of their lower cost
• RGB shadow-mask displays – small dots of red, green, and blue phosphors packed closely
• Raster-scan cathode-ray tube (CRT) – electron beam sweeping out lines of dots to form letters – refresh rates 30 to 70 per second
• Liquid-crystal displays (LCDs) – voltage changes influence the polarization of tiny capsules of liquid crystals – flicker-free – size of the capsules limits the resolution
• Plasma panel – rows of horizontal wires are slightly separated from vertical wires by small
glass-enclosed capsules of neon-based gases
Display Technologies (cont.)
• Light-emitting diodes (LEDs) – certain diodes emit light when a voltage is applied – arrays of these small diodes can be assembled to display characters
• Electronic ink– Paper like resolution– Tiny capsules with negatively and positively charged particles
• Braille displays– Pins provide output for the blind
Displays – Large and Small (cont.)
• Large displays– Informational wall displays– Interactive wall displays– Multiple desktop displays
Mobile device displays
Animation, image, and video
• Accelerated graphics hardware• More information shared and downloaded on the web• Scanning of images and OCR• Digital video• CD-ROMs and DVDs • Compression and decompression through MPEG• Computer-based video conferencing
End
• Materials from:– Shneiderman publisher site:
• http://wps.aw.com/aw_shneider_dtui_4 &5/
– MIT OpenCourseware, Robert Miller’s User Interface Design and Implementation• http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-831Fall-2004/
CourseHome/index.htm