User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly...
-
Upload
derick-snow -
Category
Documents
-
view
217 -
download
4
Transcript of User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly...
User-Centered Design and Development
Instructor: Franz J. KurfessComputer Science Dept.Cal Poly San Luis Obispo
FJK 2005
Copyright Notice
• These slides are a revised version of the originals provided with the book “Interaction Design” by Jennifer Preece, Yvonne Rogers, and Helen Sharp, Wiley, 2002.
• I added some material, made some minor modifications, and created a custom show to select a subset.
– Slides added or modified by me are marked with my initials (FJK), unless I forgot it …
FJK 2005
484-W09 Quarter
• The slides I use in class are in the Custom Show “484-W09”. It is a subset of the whole collection in this file.
• Week 8 contains slides from Chapter 14 of the textbook.
FJK 2005
Chapter Overview
• user testing• DECIDE framework for user testing• experiments
– variables and conditions– data collection and analysis
• predictive models– GOMs– keystroke level model
FJK 2005
FJK 2005
Motivation
• in addition to asking users about their experiences or tasks, it may be necessary to conduct tests where they perform tasks
• ideally, these tests should be done with the final product– in practice, this is often not feasible– user testing can also be done on prototypes to find
out how well the general approach works
• user modeling tries to predict user performance for tasks performed on a system
FJK 2005
Objectives
• become familiar with the main principles and constraints of user testing
be aware of the differences between user testing, usability testing and research experiments
be able to design simple experiments become familiar with the GOMS user model, the
keystroke level model, and Fitts’ law determine when these techniques can be applied know how to do a keystroke level analysis
The aims Describe how to do user testing. Discuss the differences between user testing, usability
testing and research experiments. Discuss the role of user testing in usability testing. Discuss how to design simple experiments. Describe GOMS, the keystroke level model, Fitts’ law
and discuss when these techniques are useful. Describe how to do a keystroke level analysis.
Activity: Portal and Blackboard in K-12 Education• the SLO County Office of Education is using a
version of the portal and Blackboard• identify an strategy to perform user testing in
their context– goals and questions– paradigm and techniques– practical aspects:
• typical tasks, typical users• test conditions, running tests
– ethical issues– evaluation, analysis, and presentation of data
FJK 2005
Activity: Tablets in Grade School
• development of an educational Graphical Information System (GIS) – team of CSC students advised by Michael Haungs,
Rollin Strohman– use of tablet PCs as client platforms
• they collaborate with a grade school teacher who wants to use their system in a lesson on California missions
• sketch a usability evaluation strategy for this context
FJK 2006
Activity Guidelines
• goals and questions• paradigm and techniques• practical aspects:
– typical tasks, typical users– test conditions, running tests
• ethical issues• evaluation, analysis, and presentation of data
FJK 2006
Activity cont: Constraints
• student age• no tablet PC experience• main objective for the teacher is student
learning about California missions• limited time• unfamiliar environment for students
– outside of their classroom– foreign people present– use of new computer equipment
FJK 2006
Experiments, User Testing and Usability Testing
• experiments – test hypotheses to discover new knowledge – investigate the relationship between two or more things
• variables
• user testing– applied experimentation– developers check that the system being developed is
usable by the intended user population for their tasks
• usability testing – uses a combination of techniques
• user testing • user satisfaction questionnaires
User Testing vs. ResearchUser testing
• Aim: improve products• Few participants• Results inform design• Not perfectly replicable• Controlled conditions• Procedure planned• Results reported to
developers
Research experiments
• Aim: discover knowledge• Many participants• Results validated
statistically • Replicable• Strongly controlled
conditions• Experimental design• Scientific paper reports
results to community
User Testing
• goals and questions focus on user performance– how well do users perform tasks with the product
• comparison of products or prototypes common• major part of usability testing• focus is on
– time to complete task – number and type of errors
• informed by video and interaction logging• user satisfaction questionnaires provide feedback
– experiences of users with the product– data about users’ opinions
FJK 2005
User Testing with DECIDE
• Determine the goals and Explore the questions
• Choose the paradigm and techniques• Identify practical issues• Deal with ethical issues• Evaluate, analyze, and present data
FJK 2005
Goals and Questions
• generic goal is usually related to usability issues– interaction– navigation
• specific goals are influenced by tasks, system, user aspects– how well do certain users perform a particular task
• usability hypothesis– it is sometimes helpful to formulate a hypothesis to
be confirmed or rejected
Paradigms and Techniques
• the main paradigm is usability testing– confusion between “user testing” and “usability
testing” is common
• recording user data– video– interaction logging
• questionnaires– user impressions, satisfaction
• interviews– more open format for user impressions
FJK 2005
Task Design
• task selection– tasks that are critical for the user’s overall job– tasks that illustrate the use of the system– frequently performed tasks– “completion” tasks are often used
• performance measurement– ideally quantifiable
• see “Data Collection” slide
– realistic• typical for users in their regular environment
FJK 2005
User Selection
• “typical” users– what are typical users?– how can typical users be found?– typical users may not be the best choice
• volunteers as test participants– self-selection may lead to bias
• users comfortable with computers• confident about their performance
• assigned participants– may be reluctant to collaborate– may intentionally perform poorly
FJK 2005
Testing Conditions
• usability lab – or other controlled space
• major emphasis on– selecting representative users– developing representative tasks
• 5-10 users typically selected• tasks usually last no more than 30 minutes• test conditions should be the same for every participant• informed consent form
– overview of the task– expectations – explains ethical issues
Data Collection
• type of data (Wilson & Wixon, ‘97)– time to complete a task– time to complete a task after a specified time away
from the product– number and type of errors per task– number of errors per unit of time– number of navigations to online help or manuals– number of users making a particular error– number of users completing task successfully
Running the Tests
• preparation– reserve room, invite participants, check licencing
issues, install systems, …
• schedule– times and activities
• script– activities of the testers– activities to be performed by the participants
• equipment setup• pilot run• aftermath
FJK 2005
Ethical Issues
• informed consent form• setup of facilities and experiments
– observation facilities• one-way mirrors• closed-circuit cameras
– recording facilities• audio, video• interaction logging
• participant discomfort– participants can interrupt or abort the experiment at
any time– contingency plans for dealing with such problems
FJK 2005
Evaluation, Analysis, and Presentation of Data
• high-level evaluation– identification of significant events or trends
• statistical analysis– maximum, minimum, average, mean, standard deviation– restricted to quantifiable aspects
• in-depth evaluation– careful observation and analysis of specific aspects
• unbiased, objective presentation– avoid secondguessing what stakeholders might want to
hear– don’t suppress unpleasant outcomes
FJK 2005
Usability Engineering Orientation
• current level of performance • minimum acceptable level of performance• target level of performance
How many participants is enough for user testing?
• largely a practical issue• depends on:
- schedule for testing- availability of participants- cost of running tests
• typically 5-10 participants • some experts argue that testing should
continue until no new insights are gained
Experiments
• predict the relationship between two or more variables
• independent variables – manipulated by the experimenter– typical experimental designs have one or two
independent variables
• dependent variable – depends on the independent variable– beyond the control of the experimenter
Experimental Designs
• different participants – a single group of participants is allocated randomly
to the experimental conditions
• same participants– all participants appear in both conditions
• matched participants– participants are matched in pairs
• e.g., based on expertise, gender • match criteria should be relevant for the task
Advantages & Disadvantages
Design Advantages Disadvantages
Different No order effects Many subjects & individual differences a problem
Same Few individuals, no individual differences
Counter-balancing needed because of ordering effects
Matched Same as different participants but individual differences reduced
Cannot be sure of perfect matching on all differences
Predictive Models
• evaluating products or designs without directly involving users
• psychological models of users are used to test designs
• less expensive than user testing• usefulness limited to systems with predictable
tasks – telephone answering systems, mobile phones, etc.
• based on expert behavior– assumes experts or experienced users
GOMS• Goals
– the state the user wants to achieve • e.g., find a website
• Operators– the cognitive processes and physical actions performed to
attain those goals• e.g., decide which search engine to use
• Methods– how to accomplish the goals
• e.g., drag mouse over field, type in keywords, press the go button
• Selection rules– determine which method to select
developed by Stu Card, Tom Moran, and Alan Newell (Card et al., 1983)
Keystroke Level Model
• quantitative refinement of the GOMS model• allows predictions to be made about how long
it takes an expert user to perform a task– identifies basic actions involved– time is measured for each action– overall time is computed
• sum of individual actions in simple cases
• applied by measuring user interaction activities– keystrokes, mouse movements– mental preparation, hand re-positioning
Response times for keystroke level operators
Operator Description Time (sec)K Pressing a single key or button
Average skilled typist (55 wpm)Average non-skilled typist (40 wpm)Pressing shift or control keyTypist unfamiliar with the keyboard
0.220.280.081.20
P
P1
Pointing with a mouse or other device on adisplay to select an object.This value is derived from Fitts’ Law which isdiscussed below.Clicking the mouse or similar device
0.40
0.20H Bring ‘home’ hands on the keyboard or other
device0.40
M Mentally prepare/respond 1.35R(t) The response time is counted only if it causes
the user to wait.t
Fitts’ Law
• predicts the time to point at an object using a device – function of the distance from the target object and the
object’s size– the further away and the smaller the object, the longer the
time to locate it and point• time to locate an object is important for some devices
and activities– handheld devices like mobile phones– computer games– navigation in multi-screen Web pages
• may indicate the use of alternatives– default actions, keyboard shortcuts
formulated by Paul Fitts (Fitts, 1954)
Key Points
• user testing is a central part of usability testing • testing is done in controlled conditions• user testing is an adapted form of experimentation• experiments aim to test hypotheses by
manipulating certain variables while keeping others constant
• the experimenter controls the independent variable(s) but not the dependent variable(s)
• three types of experimental design– different-participants, same-participants, matched
participants
A project for you …
Use the web and other resources to research Jakob Nielsen’s claims about the efficacy of heuristic evaluation and Bill Bailey’s complaints that heuristic evaluation often identifies problems that are not serious and may not even be problems at all.
Pretend that you are either Nielsen or Bailey and write a brief statement arguing your position.
Provide practical evidence & evidence from the literature to support your position.
A project for you …
Search the web for examples of the three different types of experimental designs.
Write a brief critique of each.
A Project for you …Fitts’ Law
Visit Tog’s website and do Tog’s quiz, designed to give you fitts!
http://www.asktog.com/columns/022DesignedToGiveFitts.html