Speech and Music Retrieval LBSC 796/INFM 718R Session 12, November 18, 2007 Douglas W. Oard.
Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.
-
Upload
tyler-fisher -
Category
Documents
-
view
215 -
download
0
Transcript of Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.
![Page 1: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/1.jpg)
Interaction Design
Session 12
LBSC 790 / INFM 718B
Building the Human-Computer Interface
![Page 2: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/2.jpg)
Agenda
• Questions
• Interaction design
• Some examples
• Project presentations
![Page 3: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/3.jpg)
Interaction Design
• Play to the strengths of machine and human
• Place the locus of control with the user
• Make it easy to do the right thing
• Support multiple interaction styles
![Page 4: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/4.jpg)
Strengths
• Machine– Speed– Storage– Repeatability
• Human– Initiative– Flexibility– Recognition
![Page 5: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/5.jpg)
Putting the User in Control
• Familiar metaphor
• Visible objects– Non-modal design
• Predictable behavior
• Feedback on progress
• Explicit user models– Basic and expert modes
• Optional “wizards”
![Page 6: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/6.jpg)
Making it Easy
• Visible context
• Understandable icons and messages– Tool tips and drill-down
• Atomic actions
• Obvious results
• Previews and reversability
![Page 7: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/7.jpg)
Multiple Interaction Styles
• Point-and-click
• Keyboard shortcuts
• Command line
• Spoken dialog
![Page 8: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/8.jpg)
Interactive Voice Response Systems
• Operate without graphical interfaces– Hands-free operation (e.g., driving)– Telephone access
• Built on three technologies– Speech recognition (input)– Text-to-speech (output)– Dialog management (control)
• Example: TellMe (1-800-555-TELL)
![Page 9: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/9.jpg)
Speech Recognition
• Isolated words recognition is easy– Specialized dictation and telephone applications
• Continuous speech is slow and error prone– Hands-free tasks, dictation, speech retrieval
• Performs best when trained for one speaker• Limited vocabulary and language coverage• Does not work well in noisy environments
![Page 10: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/10.jpg)
Speech Recognition
PhonemeDetection
WordConstruction
WordSelection
PhonemeString
PhonemeLattice
WordString
Pronunciationdictionary
Word n-gramlanguage
model
One-best phoneme transcription
N-best phoneme sequences
One-bestword transcript
![Page 11: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/11.jpg)
Speech Recognition Lattice
![Page 12: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/12.jpg)
Dialogue Management
• Turn-taking– User initiative– System initiative (allows smaller vocabulary)– Mixed initiative (e.g., barge in)
• Interaction style– Direct answers
• Achieving conversational goals
![Page 13: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/13.jpg)
System Initiative
• Finite state control automates scripts– Restaurant, airline reservation, …
• A “state” encodes everything you know– What prompt to offer– What to do for each possible answer
• Loops allow for compact representations
![Page 14: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/14.jpg)
Finite State Control Example
Where areyou departing
from?
Where doyou want
to go?
What daydo you want
to travel?
VerificationGoodbye
Wrong
Confirmed
BaltimoreNationalDulles
San FranciscoOaklandSan JoseAnywhere
else
Day when thereare flights
Not a day
Sorry
AnotherdayAnywhere
else
![Page 15: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/15.jpg)
Cooperative Responses
• I want to fly to Tysons Corner on Friday– Completion
• All of the flights are sold out
– Correction• There is no airport in Tyson’s Corner
– Suggestion• Dulles is the closest airport
– Conditional answer• The only flight is on Tuesday
– Summary answer• I have flights on US carriers or KLM
![Page 16: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/16.jpg)
The CSLU Toolkit
• IVRS development environment
• Graphical finite state dialog editor
• Text-to-speech, plus an animated face
• Isolated-word speech recognition
• Available at http://cslu.cse.ogi.edu/toolkit/
![Page 17: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/17.jpg)
Supporting Information Access
SourceSelection
Search
Query
Selection
Ranked List
Examination
Recording
Delivery
Recording
QueryFormulation
Search System
Query Reformulation and
Relevance Feedback
SourceReselection
![Page 18: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/18.jpg)
![Page 19: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/19.jpg)
![Page 20: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/20.jpg)
NPR Online
![Page 21: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/21.jpg)
![Page 22: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/22.jpg)
SpeechBot
![Page 23: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/23.jpg)
![Page 24: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/24.jpg)
Thesaurus-Based Search
![Page 25: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/25.jpg)
![Page 26: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/26.jpg)
Design Critique
• Select any 3 GUI’s you know and can use here– e.g., Windows XP, Google, USMAI catalog
• Work in in groups of 3 to critique each– Using IBM design guidelines
• http://www-3.ibm.com/ibm/easy/eou_ext.nsf/publish/6
– What are the 3 best features of each?– What are the 3 principal weaknesses of each?
![Page 27: Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface.](https://reader036.fdocuments.net/reader036/viewer/2022062801/56649e7f5503460f94b83a7d/html5/thumbnails/27.jpg)
An Example
• http://www.philipglass.com/