UT User Experience Requirements Definition...

1

Copyright 2010 John Morkes

User Experience Requirements Definition

John Morkes, Ph.D., [email protected]

UT Introduction to Usability Sept. 29, 2010

2

About Expero, Inc.

•  User experience consulting for complex applications and websites –  definition, design, usability, content –  located in Austin –  clients worldwide

•  www.experoinc.com

2

3

Overview

•  Exercises, examples and discussion •  Requirements and user experience •  Some techniques for defining requirements

–  personas – mental modeling –  interviews –  surveys –  observation

•  Final Q&A

4

Requirements and User Experience

3

5

What You Need to Succeed

A useful, usable and appealing UI: •  Visual Design/Look and Feel •  Content and Terminology •  Layout and Detailed Interactions •  Information Architecture and Navigation •  Functionality/Usefulness •  User Audience, Needs and Goals

6

Types of Requirements

•  Business goals •  e.g., increase user adoption, reduce support costs, etc.

•  Functional requirements •  from competitors, subject matter experts, users

•  Technical requirements •  from development team

4

7

Types of Requirements (cont.)

•  User experience metrics, standards and guidelines •  e.g., accessibility, efficiency, etc.

•  Design/workflow requirements •  from subject matter experts, users

•  End-user requirements •  personas, use cases, mental models, etc.

•  from prospective users, not just current users

8

What to Ask Users About

•  How they think/work/perform their tasks •  Possible ways to reach business goals

–  increase adoption, etc.

•  Usefulness/value of features and functions –  validate the requirements!

•  Usability issues –  known or suspected –  UI prototypes, final versions

•  User satisfaction •  Messaging and brand perception

5

9

What Users Say about a Bad Experience

•  “Unpleasant” •  “Didn’t work the way I expected” •  “Confusing and frustrating” •  “Took too much time” •  “There were errors” •  “Don’t want to use it again”

10

Bad User Experience Can Lead to:

•  Lower user adoption rates •  Inefficiency •  High support costs •  Accessibility/compliance problems •  Higher error rates •  High user dissatisfaction •  Increased application maintenance burden

(especially, Development & Documentation) •  Negative perception of your organization •  etc.

6

11

What Causes a Bad User Experience?

Wrong requirements. Often, building something without focusing on the users’ “mental models” and what they expect.

“Mind-meld” with your users

12

Personas: Defining the Users

7

13

Persona Typical Elements:

Name: Alex Age: 43

- Personal Details - Income/Spending - Work/Job Details - Use Environment/Artifacts - Activities/Use Scenario - Knowledge/Skills/Abilities - Goals/Motivations/Concerns - Likes/Dislikes - Quotes - Market Size/Influence

Source: Adlin T., Pruitt J. (2004). Creating and Using Personas. NN/g User Experience 2004.

14

Enhancements for Expert/Enthusiast Personas:

Name: Alex Age: 43 - Personal Details - Income/Spending - Work/Job Details - Use Environment/Artifacts - Activities/Use Scenario

+ Number of Years in Domain/Job - (Expert indicator)

+ Primary “Goals” for Using Your Technology - (Usefulness) (Note: Different from “goal-directed” - e.g., Reduce reliance on spreadsheets)

+ Technology Trust Indicators (e.g., data freshness, details of calculation, rationale for recommendation, etc.)

+ Key Terms in Job/Glossary - (Aid product team in grasping domain)

+ Other Technology Used in Job & Time Spent in Other Technologies - (Where does your technology fit in bigger picture)

+ Linkage with Other Personas - (e.g., Approver User / Submitter User)

- Knowledge/Skills/Abilities - Goals/Motivations/Concerns - Likes/Dislikes - Quotes - Market Size/Influence

8

15

Additional Enhancements Based-on Adlin Ad-hoc Personas:


+ Attitudes Toward Our Product - (Example multiple choice)

____ It’s a dream. It saves me tons of time and has made our product sales more profitable

____ I liked the product once I learned the system.

____ I like the product for the most part, but it still falls short for me. I rely on other methods as well. What methods?

____ I like this product, but I’m concerned about job security once it’s deployed.

____ I don’t like this product, but have to use it anyway.


16

Mental Models Should be Captured with Personas



+ Mental Model(s) – (Activity-Based/Sky Scraper Method)

9

17

Personas #1 Problem for Personas: •  Many organizations have a “check mark”

mentality – created and forgotten

•  Personas are working documents

•  Need to keep personas fresh

Additionally: •  Often too demographic-focused

(e.g., borrowed from Marketing)

•  Neglect to capture important psychographic attributes (e.g., attitudes, feelings)

Name: Alex

Age: 43

- Personal Details

- Income/Spending

- Work/Job Details

- Use Environment/Artifacts

- Activities/Use Scenario

- Knowledge/Skills/Abilities

- Goals/Motivations/Concerns

- Likes/Dislikes

- Quotes

- Market Size/Influence

18

Mental Models

10

19

Mental Models and User Experience

User’s Mental Model: How the user thinks about a task or activity

Designer’s Mental Model: How the designer thinks about a task or activity

System Image: How the technology works

(Goal is to Align!) www.asktog.com

20

Benefits of Mental Models for User Experience

•  Main purpose is to understand an activity/problem, the context from the user’s perspective

•  Documenting mental models provides a concrete way of discussing internal thought process

•  We can share with users – is this how you think about it?

•  We can use mental models to start the design

•  End Game is for Designer and User to Align in the way they think about an activity/tasks

11

21

Example: Grocery Shopping

www.appliedhumanfactors.com

22

Sky Scraper Style: Grocery Shopping

Mental Spaces

Diagram Source: Expero Research 2008-2010. Diagram modeling style from Mental Models - Aligning Design Strategy with Human Behavior. Indi Young. Rosenfeld Media, 2008.

12

23

Benefits of Sky Scraper Method

•  Structured, simple and scalable system

•  Get started mental modeling with minimal time investment

•  Straightforward approach makes it easier to examine tasks, spot areas of interest, map patterns, etc.

•  Less time managing diagrams and more time for analysis and getting the big picture right

•  Easy to share with users – did we get it right?

Diagram modeling style from Mental Models - Aligning Design Strategy with Human Behavior. Indi Young. Rosenfeld Media, 2008.

24

Segmenting Grocery Shopping

Expero Research 2008-2010.

3 Personas

Shopper Cashier Store Manager

13 Use Cases Preparation Travel to Store At the Store Travel Home

Register Login Check Out Customers Scan Problems Shift Change

Floor Efficiency Talking with Customers Inventory Status Checks Safety Checks Schedule Employees

4 Mental Models 1 1 2

13

25

Interviews

26

•  Pair up with someone. •  Person A will interview Person B to find out the most

interesting thing about B. •  Then you will switch and Person B will interview

Person A with the same goal in mind. •  Each interview should be 3 minutes. •  Then we will discuss.

Exercise 1: Interview Your Classmate

14

27

Lombardi Blueprint: Business Process Modeling Software

“Blueprint is causing us to sell more deals than we have capacity to service. Expero’s work was a big part of that.” ~ David Marquard Lombardi Blueprint Product Manager

“Blueprint, an innovative, high-level modeling product, gives business analysts a user-friendly, low-cost, but powerful requirements-gathering tool for uncovering processes when interviewing process participants. The combination packs a powerful punch, giving Lombardi a powerhouse product in the BPM space.”

~ Forrester Research

28

Blueprint

15

29

Key User Research Findings

1 32

30

Best Times for an Interview?

16

31

Best Times for an Interview?

•  Definition –  to determine user requirements, preferences, features

•  ex.: during contextual inquiry, focus group •  Design

–  for feedback on UI prototypes •  ex.: during think-aloud usability study

•  Development –  for feedback on working prototypes

•  ex.: during think-aloud usability study •  Post-Launch

–  to assess user reaction, adoption •  ex.: during competitive usability test

32

Is an Interview Appropriate?

•  Depends on: –  what you want to know –  interviewer ability –  access to customers/prospects/users –  timing –  budget –  audience for the results

17

33

More Likely to Do Interviews...

•  If: –  you want data on attitudes, beliefs, opinions, preferences,

motivations (why and how) –  you care more about qualitative data –  you want to go deep –  you care less about understanding populations –  you care less about measuring actual behavior

•  could combine with observation –  there’s enough time –  there’s enough budget

34

•  Great way to start collecting data, prioritize •  Explains why •  Yields a lot of data about the interviewees •  Can be done in many ways

–  F2F, focus groups, phone

•  Easily combined with other techniques –  observation, discount usability test, survey

•  Usually inexpensive

Interview Pros

18

35

Interview Cons

•  Data hard to quantify •  Results might not generalize

–  small samples

•  Hard to draw conclusions about populations

36

The Interview Participants

•  Try for representative sample of defined profile(s) –  based on demographics and psychographics

• ex.: dating <1 year, 16-34, seeking advice, etc. –  randomly select

•  stratified sample: 60% F, 40% M –  but convenience sample likely –  clients, market research firms, ads, people you know,

trade shows, conferences, referrals

19

37

The Interview Participants (cont.)

•  Use a screening questionnaire –  ex.: selected based on specific job title, use of the Web an

average of once a day, makes purchasing decisions for their organization

20

40

•  Do research and prepare –  but also improvise

•  General specific •  Make questions clear and relevant •  Short session, questions, words •  Avoid double-barreled:

–  “Should the website include a search function and a sitemap?”

–  better: “How useful would a search function be?” and “How useful would a sitemap be?”

How to Do the Interview

21

41

•  Avoid negative items: –  “Should the following things not be included in the site?

…”

•  Don’t lead or bias: •  “Do you agree that the site should include a search

function?” •  “Do you like the way we designed this page?” • better: “What do you think of the design of this page?” • may be OK to lead for clarification (as follow-up)

•  Pretest your questions

How to Do the Interview (cont.)

24

47

•  Dress the part •  Minimize distractions

–  no meals –  close the door –  crowd control; no bosses

•  Take careful notes –  don’t let notes distract –  record if necessary


48

•  Don’t stifle the interviewee –  be a little friendly when meeting –  be polite –  start with some easy questions –  don’t make the person feel self-conscious –  pay attention; make eye contact –  don’t interrupt –  “yes, OK, I understand…” –  no answers are stupid


25

49

•  Wait for an answer –  uncomfortable silence will be filled –  investigative journalists are great at this

• U.S.: 60 Minutes, U.K.: Panorama

•  Repeat a question with different words •  Don’t offer your opinion or disagree •  Observe work; stay out of the way for the most part


50

•  Extract more data “after” the interview: –  “Those were my questions. Thank you for your time. Is

there anything else I should know?” –  end like Lt. Columbo

•  come back with “Just one more thing…”


26

51

Sample Interview Questions

•  Why did you choose AcmeBank.com? •  What had the most impact on your decision? •  What was good about the process of opening your

account? •  What was bad about the process of opening your

account? •  What are the best things about AcmeBank.com? •  What are the worst things about AcmeBank.com?

52

Analyzing the Data

•  Look for patterns and exceptions •  Summarize and prioritize •  Content analysis

–  quantify the qualitative • ex.: # people who said Search was top item

–  objective • different analysts reach same results

–  systematic •  categories of content are defined

27

53

Analyzing the Data (cont.)

•  Content analysis categories –  relevant – manageable: limit the number

54

Reporting the Results

•  Use whatever format works – may need to report quickly, informally, concisely

•  Executive summary, with data •  Report all relevant findings

–  positive and negative

•  Prioritize feature requests and issues –  based on user data

28

55

Reporting the Results (cont.)

56

Reporting the Results (cont.)

•  Include key user quotations –  “Those fees are outrageous. I’ll never pay that. Other

banks let me pay bills for free.”

•  Describe method, users –  include list of main questions asked

•  Make informed recommendations

29

57

Focus Groups

•  Be careful –  social dynamics at play –  cross-contamination and group-think –  don’t rely on solely –  try not to use for feedback on new UI designs

58

Focus Groups (cont.)

•  How to make it work –  limit to 12 participants –  use for generating ideas, lists

• ex.: first pass at prioritizing features –  avoid contamination:

• ask your questions •  collect written responses •  then discuss

–  serve caffeine, sugar, etc. • get them talking

–  rein in the dominant talkers

30

59

Mess Up Your Interview

•  Tell them what you want to hear •  Stop them from talking •  Draw too many conclusions from the data

60

•  Imagine you are creating a new website that will help people figure out whether they are in love. You are researching requirements and features that potential users would want and need.

•  How would you conduct the research? •  What questions would you ask, if any?

Exercise 2: amiinlove.com

31

61

Surveys

32

64

33

65

66

Best Times for a Survey?

34

67

Best Times for a Survey?

•  Definition –  to determine user requirements, preferences, features

•  ex.: Web survey, focus group •  Design

–  for feedback on UI prototypes •  ex.: during think-aloud usability study

•  Development –  for feedback on working prototypes

•  ex.: during think-aloud usability study •  Post-Launch

–  to assess user reaction, adoption •  ex.: Web survey about a site

68

Is a Survey Appropriate?

•  Depends on: –  what you want to know –  ability to deliver a survey –  access to customers/prospects/users –  timing –  budget –  audience for the results

35

69

More Likely to Do Surveys...

•  If: –  you want quantitative and/or qualitative data on attitudes,

beliefs, opinions, preferences, motivations, self-reported behavior

–  you care less about observing actual behavior •  could combine with a usability study

–  you want to reach many people •  to understand entire populations

–  there’s enough time –  there’s enough budget

70

•  Great way to collect quantitative data •  Great way to collect qualitative data •  Lots of data •  Flexibility

–  phone, Web, email, paper –  self-administered or 1:1 or 1:many

•  Easily combined with other techniques –  observation, focus group, usability test

Survey Pros

36

71

Survey Cons

•  Requires special skill –  careful sampling, data analysis for generalizable data

•  Can take time •  Not the best for data on usability

–  what people say vs. do –  combine with observation

•  Hard to show cause-effect

72

The Survey Participants

•  Randomly select participants from the populations you care about –  see Interview Participants/screener above –  surveying all = census

•  Non-random/convenience samples will skew results •  More respondents = more confidence in data

37

Sample Size Calculator

74

The Survey Participants (cont.)

•  Report user details •  Watch out for people who really like surveys •  Provide incentive

38

75

Survey Data Types

•  Quantitative –  ex.: “New users rated the site a 5.4 out of 10 on Ease of

Use.”

•  Qualitative –  help to interpret quantitative –  ex.: “I think one of the worst things about the site is that

it’s hard to use.” –  quantify it:

•  “70% of the users cited Ease of Use as one of the worst things about the site.”

76

Question Types

39

77

Question Types (cont.)

•  Closed-ended –  uniform responses easy to analyze –  focuses the participant on what you care about

•  that’s good and bad • may need distraction questions

–  questions and response items must be carefully worded • exhaustive categories • mutually exclusive

78

Question Types (cont.)

•  Open-ended – What is your favorite site? –  How much did you like or dislike the site you just used? –  How would you describe the design of the site? – What else, if anything, would you like to see in the site? – What advice, if any, would you give on … ? – Why?

40

79

Response Scales

•  Nominal scale –  set of categories (names) with no numerical relationship –  ex.: What is your favorite search engine? _ Google _ Yahoo (etc.)

•  Ordinal scale –  order or sequence matters –  ex.: What length do you prefer for Web pages? _ short _ medium _ long –  can assign numbers for coding/analysis

80

Response Scales (cont.)

•  Interval scale –  numeric, ordered, fixed interval, arbitrary start (not 0) –  ex.: temperature; satisfaction: 4.3 out of 5 –  range: 2-10 in our research –  Likert-type scale

• ex.: The site is hard to use. strongly disagree disagree agree strongly agree 1 2 3 4

41

81

Response Scales (cont.)

•  Interval scale (cont.) –  Semantic Differential scale

•  choose between opposite positions • make sure terms are opposites

–  no: confused…happy

• ex.: How do you feel right now? unhappy 1 2 3 4 5 happy

–  for all responses, consider DK, NA

82

Writing Survey Questions

•  Tips –  use familiar language; be clear –  don’t lead or bias –  include instructions and an example –  avoid double-barreled questions –  keep questions and sessions short

•  fatigue –  start with easy questions –  pretest questions

42

84

Writing Survey Questions (cont.)

•  More tips – more response options yield richer data

• ask “how much do you…” instead of “do you…” –  item order affects responses

• general then specific –  don’t encourage a response pattern

•  vary items a bit; use negative terms –  ask a question more than once

•  indices to reduce error –  use closed-ended and open-ended

43

85

Indices

•  Index is a composite measure –  ask several questions to measure a concept –  ex.: User Satisfaction = Ease of Use + Quality + Appeal –  reduces measurement error

•  Creating indices –  can be tricky

• use factor analysis, Cronbach’s alpha –  better to use established questions, indices

86

Sources of Scales and Surveys

•  Publications on surveys and scales –  Rea, L.M. and Parker, R.A. (2005). Designing and

Conducting Survey Research: A Comprehensive Guide. John Wiley & Sons.

–  Rubin, R.B. et al. (Eds.) (2004). Communication Research Measures: A Sourcebook. Lawrence Erlbaum.

–  Coleman, W.D. et al. (1985). Collecting detailed user evaluations of software interfaces. Proceedings of the Human Factors Society 29th Annual Meeting, 240-244.

•  System Usability Scale (SUS) – John Brooke –  www.usabilitynet.org/trump/documents/Suschapt.doc

44

87

88

Conducting the Survey

•  Tips from Interview section apply here •  Use one method for whole survey

–  all Web or all phone, etc. –  or check for differences before combining

•  Blind survey removes brand effect •  Don’t interpret questions or response sets

–  “whatever you think it means”

•  Avoid survey fatigue

45

89

Conducting the Survey (cont.)

•  Test the survey and refine •  When ready, administer to dozens, hundreds,

thousands… •  Face-to-face surveys

–  can go longer than other types –  higher response rates –  can ask follow-up items, get more data –  can observe –  take more time and money

90

•  Respondents tend to answer politely –  they don’t want to hurt your feelings

•  Distance yourself from your topic –  “Acme Bank has asked me to collect feedback on their

site…” –  “They are interested in your reactions, both positive and

negative.” –  “I did not design the site …”

Avoid Polite Survey Responses

46

91

•  Polite responses are automatic •  People respond politely to computer surveys, too!

–  Stanford study: When a computer asked about its own performance, users gave more positive (polite) responses than when a different computer asked the same questions.

–  Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television and new media like real people and places. Cambridge: Cambridge University Press.

Avoid Polite Responses (cont.)

92

47

93

48

95

•  Create distance between your survey and the site/application being evaluated –  if possible, use a paper survey or separate computers –  if a Web survey, distance through:

•  careful language • distinct survey design • no branding

Avoid Polite Responses (cont.)

96

Data Analysis and Statistics

•  Typical data analysis: – mean, std. deviation

49

97

Data Analysis and Statistics (cont.)

•  Typical data analysis: –  frequency distribution

http://www.odinjobs.com/Usability_Engineer_job_market_overview.html

98


•  Typical data analysis: –  cross tabulations

http://www.custominsight.com/articles/crosstab-sample.asp

50

99


•  Typical data analysis: –  open-ended comments

•  see Content Analysis section above

100

Sources on Basic Statistics

•  Ott, R.L. and Longnecker, M.T. (2000). An Introduction to Statistical Methods and Data Analysis. Duxbury Press.

•  Gonick, L., and Smith, W. (1993). The Cartoon Guide to Statistics. Collins. –  a great introduction

51

101

Reporting the Results

•  Typical report: –  Executive summary –  Background –  Methodology –  Results –  Discussion –  Recommendations –  Appendices: The Survey, Data, Analysis Details

102

•  Confuse them with bad questions or response sets • participants will just guess or skip over

•  Make it too long •  Draw too many conclusions from the data

Mess Up Your Survey

52

103

104

53

105

Observation

106

A Few Tips on Observation

•  What people say vs. do •  Combine with other techniques

–  interview, survey, etc.

•  Watch and listen carefully •  Stay out of the way •  Ask for artifacts •  Look at web analytics data

–  what’s being used, not used –  live A/B testing

54

107

Conclusion

108

55

109

Q&A

UT User Experience Requirements Definition...

Documents

Transcript of UT User Experience Requirements Definition...