Tales from the Lab

Tales from the Lab

Reports from the field --psychology experiments relevant

to Usability Professionals

Paul Sas March 16, 2004

March 16, 2004 Paul Sas 2

Unlike my CHI2003 TutorialUser bias & judgment:

The Subjective side of decisionmaking

Part I. Studies on judgment and decision making

90 minute review of findings on JDM

+ 4.5 Hours more, trying to squeeze in a semester of content


Goals of this talk

• NOT an Ed Tufte dot of infinite data density• NOT a demonstration of what happens with

Choice Overload

Rather:• Tales. Many psychological principles best

shown through stories• Suggestive applications for Designers and

User Testing Experts


Heuristics, defined

• Not an arcane term to usability experts

• Quick, ready rules of thumb

• In JDM, experiments focus on the biases that fall out from these rules of thumb in many well-defined cases


One demonstration of a Heuristic


E.g.: Representativeness Heuristic

Q: Which delivery room will have more days during which the number of boys delivered will outnumber the girls?

Hospital A -- 15 deliveries/day

Hospital B -- 45 deliveries/day

Or is it C: Just as likely to happen at A as at B


Law of small numbers (Tversky and Kahneman)

Representative Heuristic: If X resembles Y, people judge the likelihood as high that X will generate/cause Y.

• A delivery room is a delivery room (for more than 60% of the people polled)

• Undue confidence in early trends

• Unreasonably expect replication.

• Underestimate confidence interval width• Rarely attribute deviation to sampling


Relevance to Designers/Testers

• Remember that a user test is a very useful heuristic

• Emphasize to your audience that it necessarily has biases

• Don't bet your job on the patterns of preferences observed


Less is More (Hsee)

• Music Dictionary A– 10,000 entries– condition is like new

• Music dictionary B– 20,000 entries– cover is torn

• When evaluated separately: A $24 B $20• When evaluated jointly: A $19 B $27


Less is More, again

• China set A– 24 pieces of plateware– Full set for 8

• China set B– 32 pieces of plateware intact out of 40– Includes all 24 of A, plus 8 (with 8 more chipped)

• When evaluated separately: A $24 B $20• When evaluated jointly: A $19 B $27


Separate v Joint Evaluation (JE)

• SE: analogous to purchasing one item from an auction, w/o comparative shopping

• JE: Occurs when choosing from a range of items

• Ease of evaluability drives decision• Lay rationalism: a tendency to overweight

attributes that appear rationalistic, such quantity and economic value, and downplay attributes that appear subjective


Primed to feel or Primed to Calculate(Rottenstreich)

• Box of Madonna CDs– If evaluated from an affective/emotional valence,

then quantity is irrelevant– If viewed as tokens with exchange value, then

more equals more

• Primed to feel: No more price for box of 10 CDs than for 5 CDs

• Primed to calculate: Box of 10 CDs receives higher valuation than 5


What does this tell Designers?

• It is possible to move people toward an emotionally valent sphere where there is no longer cost-competition

• Steve Jobs at Apple is superb at transcending the commodity space of price


Anchoring (Ariely)

• People's valuation is incredibly sensitive to random sources of information

• Classic: Roulette wheel's impact on Africa• Auction to earn $ for stress:

Digits of SSN were starting point (2 or 3)– Low anchored SSN: earned $.08– High anchored SSN: earned $.59


Can Designers Use Anchoring?

• In a competitive market, it's not possible to whisper a price(Yet realtors get 6%, and waiters 15)

• Newly developed niches can be creatively partitioned.

• Shipping fees, handling fees, etc.


Paradoxes of Hedonomics

• Experienced vs. Remembered Utility– Certain dimensions are easy to evaluate:

• Most intense moment• Last moment

– Other dimensions are impossible to guess:• Average height on a roller coaster ride• Distance covered on a roller coaster


Peak and End Rule (Kahneman)

• Experienced vs. Remembered Utility– "Our mind does not make movies; it takes

snapshots"– Rather than guess the total amount of suffering,

people recall the worst instant, and the last instant.– If you increase the amount of suffering, but

arrange for the last minutes to be less intense, people report the longer period as less painful


Peak and End Rule -- Pictograph


Peak and End Rule for User Design

• Jared Spool's "Truth About Download Time"– Nielsen reports the most popular sites took an average of 8

seconds to download, whereas the pages of the less popular sites took an average of 19 seconds. "He therefore concludes that users will be annoyed … by pages that take any longer than about 10 seconds to load."

– no correlation between download times and perceived speeds reported by our users. Amazon.com, rated as one of the fastest sites by users, was really the slowest (average: 36 seconds).

– a strong correlation between perceived download time and whether users successfully completed their tasks on a site


Uncertainty Effects (Shafir)

• Buy the trip to Hawaii– If you pass, you'll take the trip to celebrate– If you fail, you'll go on the same trip to console

yourself

• Choose to buy 32%; Choose to not buy, 7%– Pay $5 fee to wait, 61% – Not knowing why I'm going feels irresponsible.


More choice causes less purchasing (Iyengar)

• Every other hour, a set of 24 jams/jellies to sample. On odd hours, only 6 jams available.

• Choice of sampling any of 24 jams: 3% redeemed coupon.

• Choice of sampling any of 6 jams: 30% redeemed coupon.

• Recdently validated in 401K selection behavior


Solutions to Choice Overload

• "Oracles" (a magus, not a 'wizard')

• Investors preferred the portfolio selected by a professional investment manager to the portfolio they selected themselves, when comparing the implied distribution of outcomes [Benartzi and Thaler (2001)]


Eliciting Goals that Matter

• Can't simply affirm past efficacy

• Distraction fails as well


Delmore Effect

• Related domain, outside the single-most important, will improve effectiveness


Creating Personas to Design For

• Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

A. Linda is a teacher in elementary school.

B. Linda works in a bookstore and takes Yoga classes.

C. Linda is active in the feminist movement.

D. Linda is a psychiatric social worker.

E. Linda is a member of the League of Women Voters.

F. Linda is a bank teller.

G. Linda is an insurance salesperson.

H. Linda is a bank teller and is active in the feminist movement.


Conjunction Errors

• Heuristics lead to violations– Linda = feminist bank teller

• 85% Respondents displayed the predicted order (H>F for Linda)

– Explanation of P(F and H)>P(F)• Adding features increases representativeness – but

cannot increase probability


Descriptive Choice: Prospect Theory


Sources for followup

• Less is More (Chris Hsee)http://gsbwww.uchicago.edu/fac/christopher.hsee/vita/

• Primed for Feeling (Yuval Rottenstreich)http://gsb.uchicago.edu/fac/yuval.rottenstreich/

• Anchoring (Dan Ariely)http://web.mit.edu/ariely/www/papers.html

• Peak and End Rule (Daniel Kahneman)http://www.nobel.se/economics/laureates/2002/kahneman-lecture.html

• Choice Overload (Sheena Iyengar)http://www.columbia.edu/~ss957/articles.html

• Delmore Effect (Paul [Whitmore] Sas)http://www-psych.stanford.edu/~wit/PhDraft.pdf

http://gsbwww.uchicago.edu/fac/christopher.hsee/vita/



http://web.mit.edu/ariely/www/papers.html

http://web.mit.edu/ariely/www/papers.html

Tales from the Lab

Documents

Transcript of Tales from the Lab