Lecture #3.3 USABILITY TESTS *39TUR Summer 2011/2012.
-
Upload
willis-turner -
Category
Documents
-
view
219 -
download
0
Transcript of Lecture #3.3 USABILITY TESTS *39TUR Summer 2011/2012.
Lecture #3.3USABILITY TESTS*39TUR Summer 2011/2012
(2)
Who is the “customer” of the usability tests?
Who asks you to do the usability test?– Designers– Software engineers– Management– Government agencies
(3)
How well do they specify what to do?
Thorough assignment– “Here’s a detailed overview of what situations to test. We want to
test such and such users, in such and such conditions.”
Partial assignment– Specific user group: “We would like to know how well this works
with {elderly | children | violists }.”
… Vague assignment
– “Find out whether it works.”– In such case it is the responsibility of the tester to refine the
assignment.
Terminologická poznámka
“Tester”– Rozpor ve významu– Ten, s kým se testuje (dobryweb.cz, h1.cz)– Ten, kdo organizuje a provádí test (správná)
Lepší termíny– “Participant”– “Účastník testu”
METHODS OF USABILITY TESTING
(6)
Usability Testing
Usability testing– Main goal: Looking for design problems
Acceptance testing– Refers to user satisfaction. A subset of the usability
testing.• “Will users like it?”
– Make sure the system meets expected criteria from the user side
• “80% customers need 1—3 minutes to withdraw amount of $50.”
(7)
Methods
Non-user based (without users)– Expert review– Heuristic evaluations– Cognitive walkthrough
User-based (with users)– User surveys– Ethnographic observations– Usability engineering
(8)
Methods with Users
Observing users in their own environment– Their office, etc.– Natural and therefore realistic conditions
• Environment plays very important role (ringing phones, traffic on the streets, …)
• The user is accustomed to these conditions
– Difficult to organize, time consuming– Results can not be generalized
(9)
Methods with Users
Controlled experiments– Controlled (laboratory) environment– Controlled conditions
• Limited influence of external stimuli
• No unwanted distractions by other people
• No unwanted distractions by software running on background
– Repeatable• The same procedures will give the same results
– Non-realistic conditions• problem with ecological validity
(10)
User Surveys
Questionnaires with questions oriented towards expected usability problems
Quantitative– Get precise measurement of one particular aspect
Qualitative– Open-ended questions
– Check how much you agree with the following statements:– “It is easy to enter the telephone number”– [strongly agree] [agree] [neither] [disagree] [strongly diagree] – “It is easy to setup the owner’s address”– [strongly agree] [agree] [neither] [disagree] [strongly diagree]– “The menu item ‘Games’ is easy to find”– [strongly agree] [agree] [neither] [disagree] [strongly diagree]
(11)
User Surveys – Quantitative
Check how much you agree with the following statements:“It is easy to enter the telephone number”
[strongly agree] [agree] [neither] [disagree] [strongly diagree] “It is easy to setup the owner’s address”
[strongly agree] [agree] [neither] [disagree] [strongly diagree]“The menu item ‘Games’ is easy to find”
[strongly agree] [agree] [neither] [disagree] [strongly diagree]
Your age: ………
Your income: ………+ Easy to process+/– Limited amount of possible answers
(12)
User Surveys – Qualitative
What are the biggest problems you find whileusing the telephone XX-1234?
………………………………………………………………………………………………………………
What functions do you use most frequently?………………………………………………………………………………………………………………
– More difficult to process+/– Intended to map topics that were not previously studied
(13)
User Surveys
Pros– Inexpensive– Can be done remotely
• “While waiting for the video to load, why don’t you fill out our little questionnaire?”
– Can provide data on the current trends Cons
– Relies only on the data provided by the users– Recruiting the right people is a problem
• Self-selection bias• Add some screening questions• Provide incentives
(14)
Ethnographic Observation
Observing users in their environment, without any intervention– Can be rural India, can be office of the US government.– E.g.: “Find out how cell phones are used in rural areas
of India.”– E.g.: “Find out how / whether the elderly people spend
their day. Could they use instant messaging of some sort?” (ICQ, Skype…)
The observer stays with the users and watches them how they use (struggle using) some system.
(15)
Ethnographic Observation
Pros:– Based on the reality
Cons:– Needs a thorough preparation– Not an interactive method– Possible ethical problems– Expensive
(16)
Usability Engineering
Nielsen (1983) Assessing the usability of a design
Observing the users using the system in a simulated environment– Aspects of the real world are simulated– Users work on predefined, but realistic tasks– Users are observed (measured) while carrying out the
tasks– Identification of the problematic parts
(17)
Usability Engineering
Are the results of such a test comparable to the results obtained from the “real-world observation”?– Selected users are being tested– The tasks are defined by the experimenters– Users are not in their natural “habitat”– Different social settings
Solutions– Use the real users– Make the environment as real as possible– Consult the tasks with the users
(18)
Usability Engineering
Make the tasks realistic Make the tasks ecologically valid
LAYOUT OF A USABILITY LAB
(20)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
(21)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
(22)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
(23)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
6-126-12
(24)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
TasksTask 1: xx xxx xx x xxx xx xxx.
Tasks
Task 1:In few minutes explore the new IDE environment. Please think aloud during the exploration process.
Task 2:Pleas create and run simple mobile application "Hello World", which will display "Hello World!" message on the display of the cell phone emulator.
Task 3:Your colleague finished the application GoodBye. Your complete this application wit
(25)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
TasksTask 1: xx xxx xx x xxx xx xxx.
Tasks
Task 1:In few minutes explore the new IDE environment. Pleas think aloud during the exploration process.
Task 2:Please create and run simple mobile application "Hello World", which will display "Hello World!" message on the display of the cell phone emulator.
Task 3:Your colleague finished the application GoodBye. Your complete this application wit
(26)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
I am trying to create new
project
TasksTask 1: xx xxx xx x xxx xx xxx.
Lo
gT
ask 1: xx xxx xx x xxx xx xxx.
Test Session Log
Task 1:12:01 start12:07 end
Task 2:12:09 start12:10 creates new project
(27)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
I am trying to create new
project
TasksTask 1: xx xxx xx x xxx xx xxx.
Záp
isÚ
loh
a 1: xx xxx xx x xxx xx xxx.
I can't find appropriate menu item
Test Session Log
Task 1:12:01 start12:07 end
Task 2:12:09 start12:10 creates new project
12:12 can not find appropriate menu item
(28)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
I am trying to create new
project
TasksTask 1: xx xxx xx x xxx xx xxx.
Záp
isÚ
loh
a 1: xx xxx xx x xxx xx xxx.
I expected item "New Project" in menu
"File".
I can't find appropriate menu item
Test Session Log
Task 1:12:01 start12:07 end
Task 2:12:09 start12:10 creates new project12:12 can not find appropriatemenu item
12:13 expects "New Project" in menu "File"
(29)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
TasksTask 1: xx xxx xx x xxx xx xxx.
Záp
isÚ
loh
a 1: xx xxx xx x xxx xx xxx.
Test Session Log
Task 1:12:01 start12:07 end
Task 2:12:09 start12:10 creates new project12:12 can not find appropriatemenu item12:13 expects "New Project" inmenu "File"12:27 end
Task 3:12:31 start
(30)
Usability Lab Usability Lab – – usability testingusability testingObserver roomObserver room
(UI designers(UI designers, , programmersprogrammers, , test organizers)test organizers)Participants roomParticipants room(test participant(test participant, , moderator)moderator)
Záp
isÚ
loh
a 1: xx xxx xx x xxx xx xxx.
Rep
ort
1. Go
al xx xxx xx x xxx xx xxx.
Final report
1. Test goal2. Summary3. Results
3.1 Issue 1Problem descriptionSolution proposal
3.2 Issue 24. Conclusion
(31)
We do need a lab!
Everything is at hand– The software that is tested, the tools for recording– The experts are present– The stakeholders can witness the process
A chance to observe and record non-obvious findings– Survey:
• User fills out: “I can use the function X to do Y.”
– Usability engineering:• User says: “I can use this function X to do Y.”• Experimenter: “Show me.”• User: “Sure, I just …” (struggles) “uh … actually, I can not!”