1 ISE 412 - 2 User-centered design Theme - “know thy user,” “honor thy user” Involves four...
-
Upload
francine-poole -
Category
Documents
-
view
219 -
download
1
Transcript of 1 ISE 412 - 2 User-centered design Theme - “know thy user,” “honor thy user” Involves four...
1ISE 412 - 2
User-centered design
Theme - “know thy user,” “honor thy user” Involves four major components:
Early focus on user and task Empirical measurement Iterative design Participatory design
2ISE 412 - 2
Norman’s Theory of Action
The World
Goals
Intention to act
Sequence of actions
Execution of actions
Interpreting the perceptions
Perceiving the state of the world
Evaluation of interpretations
3ISE 412 - 2
“Gulfs” of Evaluation and Execution
The World
Goals
Complexity, inappropriate model
Complexity, unknown alternatives
Complexity, poor design
Inappropriate feedback, mental models
Invisible states, time lags
Inappropriate mental models
Gulf of Execution Gulf of Evaluation
4ISE 412 - 2
Design to “bridge the gulfs”
The World
Goals
Evaluate the need/ability to affect the state
Understand what should be done
Perform the desired actions
Interpret data to understand what's happening
Get information about what's happening
Understand what the system state is/should be
How easy is it to ...? How easy is it to ...?
5ISE 412 - 2
The key
Understanding human behavior, cognitive processes, capabilities, and limitations
Understanding what people are doing, why, where, and under what conditions To explore further …
Norman, D.A. (2005) Human-Centered Design Considered Harmful, http://www.jnd.org/dn.mss/ human-centered_desig.html
6ISE 412 - 2
The Information Processing Model
from: Wickens, Lee, Liu, & Becker (2004) An Introduction to Human Factors Engineering (2nd ed.). New Jersey: Pearson Prentice Hall. pg.122
7ISE 412 - 2
HUMAN FACTORS RESEARCH What can we measure?
Attitude and motivation Preferences Knowledge Skills and abilities (physical and mental) Aptitude and potential to perform/succeed Emotion (happy, sad, mad, excited, tense) Physiological states Behavior Performance
8ISE 412 - 2
RESEARCH METHODS IN HUMAN FACTORS ENGINEERING
Basic vs Applied Definitions:
Basic Applied
9ISE 412 - 2
Dimensions of Research Methodologies:
Tradeoff: Experimental Control vs Real-World Relevance
Degree of Control: over conditions of observation over "treatment" (independent) variables
Degree to which behavioral (dependent) variables are representative of some larger population
Will the results "scale up" to complex real-life situations?
10ISE 412 - 2
Research Methodology “In a Nutshell”
Define problem and hypotheses What questions do you want to answer? What cause/effect relationships do you expect to see?
Specify the experimental plan Independent variables (Treatment):
Dependent variables (Effect):
Experimental design:
Specifics:
11ISE 412 - 2
Research Methodology “In a Nutshell” (cont.) Conduct the study
follow the plan you defined use a “pilot study” if you are unsure of your plan
Analyze the data Quantitative/objective data
Subjective data
Protocol analyses
12ISE 412 - 2
Research Methodology “In a Nutshell” (cont.) Draw conclusions
Were the hypotheses supported by the results of the experiment?
Statistically significant? (Usually use p = 0.05 or 0.01, but these are not carved in stone!)
Recall Type I and Type II errors … If not significant, are there interesting trends in the data? Interaction effects? (e.g., mouse vs keyboard confounded
with type of input required when filling out online forms) Can you say why?
Will effect the degree to which you can generalize your results.
Can you explain interaction effects, trends that are not statistically significant?
13ISE 412 - 2
An overview of experimental types:Laboratory Experiment Example: Carter's 1979 study of the effects of various display characteristics
on search time using college students in a dark quiet room.
Experimental Control Relevance
High
Low
14ISE 412 - 2
An overview of experimental types:Field Experiment / Quasi-Experiment Examples: (1) Comparing new ATC interface with existing using real air traffic
controllers in real-life settings (field experiment). (2)Comparing "New Math" to conventional instruction by assigning programs to different classrooms (quasi-experiment).
Experimental Control Relevance
Moderate
15ISE 412 - 2
An overview of experimental types:
Simulation Study Examples: Flight simulators, power plant control simulators.
Experimental Control Relevance
High
Low
16ISE 412 - 2
An overview of experimental types:Field Study (Case study) Examples: (1) flight crew task analyses conducted in-flight to
determine task allocation, workload, etc. prior to specification and design of automated systems; (2) investigators observation and modeling of satellite ground operations
Investigator observes and records real-life activities without any experimental manipulations. Typically, such a study seeks to articulate how and why some
behavioral phenomenon occurs.
Generally, should be the first step in investigating human interaction with complex systems.
17ISE 412 - 2
Descriptive Methods
Survey/Questionnaire Used to gauge attitudes, preferences, etc., often on a
numerical scale (e.g., 1=strongly disagree, ..., 5=strongly agree). Amenable to statistical analyses.
Problems: how to avoid biased questions
Respondents may not be totally truthful
18ISE 412 - 2
Descriptive Methods (cont.)
Interview Also used to gauge attitudes, preferences, etc.
Can delve into the reasons behind those attitudes.
Can be used with a questionnaire to clarify or expand on responses.
Problems: Same as with survey/questionnaire.
Tricky to analyze.
19ISE 412 - 2
Descriptive Methods
Concurrent Verbal Protocols While the subject performs a task (in a laboratory, field, or
simulation setting), he/she verbalizes- thoughts, reasons for actions, etc. This is often very illuminating in terms of identifying subject strategies, difficulties, etc. But difficult to analyze and may affect task performance itself.
Retrospective Verbal Protocols After the subject has performed a task (in a laboratory, field, or
simulation setting), he/she verbalizes thoughts, reasons for actions, etc. This may also be quite interesting, but again is difficult to analyze. Also this relies on the subject's memory, which is prone to error.
Also: critical incident technique, accident analysis.
20ISE 412 - 2
General Remarks: Before you do any experiment, you should have one or
more hypotheses in mind that you want to test or explore.
If possible, use multiple methods (e.g., field study and interviews and questionnaires). If you obtain converging evidence from a number of separate
sources, you can be more confident that the hypothesis is correct.
Consider reliability and validity issues: Reliability:
Construct Validity:
21ISE 412 - 2
General Remarks (cont.):
Reliability and validity issues (concl.): Internal Validity (for causal or explanatory studies):
External Validity:
Ethical Issues: Ethical treatment of human subjects
protection from mental and physical harm privacy informed consent
Code of Engineering Ethics
22ISE 412 - 2
Contextual Inquiry (Field Study +) A fact-based approach to
understanding the reality of users’ goals, processes, and tasks.
Puts systems designers directly in the “world” of the user to see, hear, feel, taste, smell, and better understand: the goals of the user the activities and specific behaviors
users engage in to achieve those goals the places in which they occur the tools, supplies, etc. that are used the artifacts in the environment and
their meanings etc.
Provides rich, context-specific data from which to develop a shared understanding of users that will guide design
23ISE 412 - 2
C.I. vs Field StudyField Study Contextual Inquiry
Typical Focus
(Traditionally) operators, their activities, tools, information requirements, decision making, work environment, etc., with emphasis on specific tasks or goals.
Interactions of people, environments, tools, and artifacts in specific workplaces or doing specific types of work, with emphasis on communication & cultural aspects.
Typical Approach
Researcher is an observer who tries not to interfere with task performance. Follow-up interviews are used to gather more info.
Researcher is a participant and the operator is considered a partner in the process. Questions may be asked during observations, with follow-up interviews as well.
Typical Product
Task analyses, physical models, information flow models.
Flow models, cultural models, physical models, sequence models, and artifact models.
24ISE 412 - 2
The C. I. Process(Source: Beyer, H. & Holtzblatt, K. (1998) Contextual Design: Defining Customer-Centered
Systems. San Diego: Academic Press)
Design the research Recruit participants Collect data
interview (video taping optional) introduction initial interview contextual interview wrap-up
research notes artifacts “homework”, prework, etc. if applicable
Analyze video, notes, visuals, artifacts, etc. Synthesize results
25ISE 412 - 2
Designing Contextual Studies: General Approach What will you be
investigating? specific activities, problems,
etc. environments target population
Script what questions will you ask in
the interview(s)? are there specific tasks or
problems you want to be sure to observe?
what follow-up do you anticipate?
Equipment & supplies notepad and pencil/pen are
minimum requirement, but usually sufficient
video or audio tape optional pros and cons …
Prework / Homework ‘depth’ questions you’d like
participants to answer before or after the interview
not always necessary, but can give broader perspective of the workplace, goals, attitudes, etc.
NOTE: This applies to all field studies, not just CI.
26ISE 412 - 2
To discuss … What are you investigating? (e.g., what types of
market/customers?, doing what?, etc.) What kind of questions? Do you want to “shadow” one or more individuals or
observe a random sample? if individuals, do you want them to do any kind of “pre-work”?
Are you going to try to video or audiotape? (If so, you’ll need permission!)
REMEMBER! … This is your baseline research, so the broader and more
inclusive you can be at this point, the better you will be able to narrow the scope of your project later.
27ISE 412 - 2
Recruiting for a Study Number
depends on the process you’re studying, the target population, etc. typically 10-20, but can be less for smaller, more focused design
problems recruit at the high end of your target sample size to cover ‘drop-
outs’
Diverse, representative sample, considering (as appropriate for the process you are supporting): age range male/female mix type of work, workplace, etc. Ethnic heritage
Once removed from interviewer (friend of a friend) or more familiarity breeds too much shared understanding
28ISE 412 - 2
Interview Techniques: Getting Started (5-10 min.)
DO:Come to the interview prepared:
video equipment ready to gonotes, pen, other materials accessibleright hand free to shake hands at the
door!Set the participant at ease:
introduce yourselves and remind them why you’re there
use equipment setup time to “break the ice”
Select initial interview sitein proximity to “work location” or
starting point, if possibleroom for participant and interviewer to
sit comfortablyask participant where they are most
comfortableMaintain a polite, professional demeanor“set the tone” for the next 2 hours
DON’T: Show up at the door with video
“rolling” Fall into the “buddy trap”
get distracted by side conversations
ask questions about personal matters not related to the study
tell “your” story instead of hearing his/hers
Presume or demand
29ISE 412 - 2
DON’T:Lose sight of the purpose and direction of the interviewby allowing broad motivational discussions to
veer into side issues“gently but firmly” guide the conversation
back on courseDelve into details of what and how
these should be addressed in the contextual interview
Ask leading questions“Suggest an answer” or finish the sentence for them
Suggest judgement of the goals, motivations, activities, etc. (either of the participant or anyone else.)
Interview Techniques: The Initial Interview (15 min.)
DO:Think “broad part of the funnel”
start with broad understanding of goals, motives, priorities, etc.
get “big picture” understanding of their viewpoint
ask for opinions about “tools” and equipment typically used
Listen for and note things you will want to observe and ask about during the contextual interview
Ask open-ended questionsFollow up on interesting points, but maintain pace and direction of the interview
At the end of this phase, move the interview to the “workspace”
30ISE 412 - 2
DON’T:Interrupt the flow unnecessarily.
Ask yourself, “Can this question wait until she/he’s finished with this?”
On the other hand, some questions HAVE to be asked in context - it’s a judgement call!
Ask leading questions.“Is the reason you’re doing that because
___?”“Many people ___. Do you feel the same
way?”“When you say ‘sometimes’, you mean about
once a month, right?”
Interview Techniques: Contextual Interview (60-90 min.)
DO:Watch and listen more than talk and
ask.Provide occasional feedback (“okay,” “I
see”, etc.) to let the participant know you’re listening
Use questions sparingly to gain clarification, understand “why” something is done the way it is, or to guide the participant’s comments
Questions, when warranted, should be open-ended and clear
“I noticed you ___ - why is/was that?”“ What are your thoughts (or feelings)
about that?”“You said you sometimes ____ - when
would you do that and why?”“What do you mean by ‘___’?” “Say
some more about that.”“Tell me what that is for.” “Tell me what
you’re doing now.” (sparingly)
31ISE 412 - 2
Interview Techniques: Contextual Interview (cont.)
DON’T: Finish the sentence or suggest an
answer It is not necessary to fill all silences
Imply judgement in questions or responses “You don’t do THAT, do you?” “Aren’t you afraid of ___?” “Don’t you hate it when ___?”
DO: Take notes, make sketches, write down
questions you want to delve into in detail at the end note what is happening, your
interpretation, your immediate thoughts, etc.
ALWAYS keep your attention on the participant and her/his activities
Pay close attention to key artifacts in the environment: videotape or photograph individual and
groups of items if there are “disposable” items, ask if you
can take them with you
32ISE 412 - 2
Interview Techniques: The Wrap-Up (15 min.)
DO: Ask any remaining questions you
have about the goals, activities, methods, tools, etc.
Summarize key points you heard during the interview, giving the participant a chance to respond, elaborate, correct your interpretation, etc.
Thank the participant for taking the time and allowing you to learn about his/her activities.
Be sure to take all notes, supplies, equipment, etc., when you leave.
DON’T: Overstay your welcome.
Leave any trash behind when you leave.
33ISE 412 - 2
Your turn …
Write one or two questions you will want to ask during your interview.
Test your questions with me as your “participant.” Does the participant understand the question? Does the answer help you to better understand the
situation you are investigating? What follow-up or depth questions are suggested by
the participant’s answer? Revise your questions accordingly. Use this method to help you refine your
interview script.