Max diff scaling for research access(4)

MaxDiff ScalingA Better Way to Understand Preference / Importance

Nico Peruzzi, PhD

October 2010

http://twitter.com/NicoPeruzziPhD [email protected]

Agenda

Copyright 2010 Outsource Research Consulting | All Rights Reserved

What is MaxDiff? Why use it? Why is it better than rating scales? How is it different than conjoint? Case studies

33

What is MaxDiff?

MaxDiff (Maximum Difference Scaling) is an approach for obtaining preference/importance scores for multiple items: Brand preferences Brand images Product/service features Messages Advertising claims Benefits

MaxDiff is also known as “best-worst scaling”


44

How many times have you tried to interpret the difference between a mean of 4.2 and 4.4 on a 5-point rating scale? You know the futility of that exercise.


Types of Data Nominal – categories (gender, political

affiliation, etc.) Ordinal – rankings (college football ranking,

survey rankings, etc.), Likert scales Interval – Temperature Fahrenheit, Likert

scales often treated as if they were interval Ratio – height, weight, temperature Kelvin,

MaxDiff Scores!


MaxDiff Design


Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

Item 8

Item 9

Item 10

Experimental Design/

Randomization

Card #1

Card #2

Card #3

Card #4

Card #5

Card #6

Card #7

Etc.

77

Research has shown that MaxDiff scores demonstrate greater discrimination among items and between respondents on the items.

• Respondents see a number of screens like this one. Each time, they simply choose the most important (or preferred) and least important item

• No scale bias or different interpretation across respondents or languages/cultures/education levels

• Any downside? Time


88

Easy to Interpret

MaxDiff scores are easy to interpret. They are commonly placed on a 0 to 100 common scale and sum to 100.

Thus, when you see a “10” it has twice as much value as a “5” You can’t do this with rating scale results

Feature Rating

Price 10.3

Ease of Use 8.0

Quality 7.2

Performance 6.0

Etc… …

Total 100Copyright 2010 Outsource Research Consulting | All Rights

Reserved

99

Why not use conjoint?

MaxDiff is related to conjoint in that it forces trade-offs, however… Conjoint is only appropriate when you want to

estimate how multiple attributes taken together affect overall preference

MaxDiff looks at a lot of “items” (levels) within a single attribute (variable) (e.g., product features)… Whereas conjoint looks at multiple attributes,

each with some number of levels; conjoint commonly examines the feature-price mix


Case Studies


Case Study: Feature Benefits

Client: Data Robotics, Inc.

Objective: Testing benefits

Method: MaxDiff with 7 items


12

Data Robotics Vision and Mission

12

Give creative professionals, prosumer, and SMB customers the best storage experience ever

BeyondRAID

Easy: Self-managing storage arrays that eliminate the difficult and confining choices associated with RAID

Affordable: Best in value upfront and over time − savings compound with “pay-as-you-grow” storage

Safe: Products offer single and dual drive redundancy with self-healing capabilities

Expandable: Instantly increase capacity by inserting a new hard drive or replacing the smallest drive – even when all hard drive bays are full

Drobo Product Family

PositioningDesktop

Storage for Everyone

Performance Desktop

Storage for Professionals

Single Server Storage for SMBs and

Professionals

Shared Storage for Everyone

Shared Storage and Data Backup for Small Business

Multiple Server Storage for SMBs and Departments

Capacity 4 Drives 5 Drives 8 Drives 5 Drives 8 Drives 8 Drives

Target Deployment

Home/ SOHO/ Mobile/ ROBO

Creative Pro/ SOHO

SMB/ Dept. Single Server

Storage / Creative Pro

Home / SOHO SMB File SharingSMB / Dept. iSCSI SAN or VMware

Cluster

Redundancy Single DriveSingle or Dual

DriveSingle or Dual

DriveSingle or Dual Drive Single or Dual Drive Single or Dual Drive

SmartVolumes Auto Auto Up to 16 Auto Auto Up to 255

Interface USB, FW800USB, FW800,

eSATAUSB, FW800, GigE iSCSI

1 x GigE Network 2 x GigE Network 2x GigE iSCS,

(USB mgmt only)

List Price $399 $799 $1,499 $699 $1,999 $3,499

FS

SDirect Attached Storage - DAS Network File Sharing iSCSI SAN

What are some benefits of Drobo? Which statements most resonate with buyers

I am constantly running out of storage. I need to instantly expand and grow my capacity without downtime or delay.

I find RAID too complex and want something easier to manage. I like the ability to buy drives when I need storage, instead buying

storage ahead of my needs. I need a storage device that consolidates my needs on a single,

easy to use array rather than multiple devices that require individual administration and create unnecessary clutter.

I want a storage device that can manage itself so that I don’t need to use consultants or hire dedicated staffing to manage it.

My data is extremely valuable - I need a system that protects me from drive failures.

The ability to mix and match hard drives of different capacities, from any drive maker is important to me.


Results


Label Average95%

Lower95%

Upper

Item 1 28.7 27.3 30.2Item 2 23.2 22.0 24.4Item 3 15.2 13.6 16.9Item 4 12.9 11.4 14.4Item 5 10.7 9.3 12.2Item 6 5.4 4.4 6.4Item 7 3.8 2.9 4.7

Item 1 is 24% more preferred than Item 2, and 89% more preferred than Item 3

100 point ratio scale

Case Study: Feature Preference

Client: A multi-billion dollar technology company

Objective: Understand which features customers most want in a product line extension

Method: MaxDiff with 11 items


Features of a XXXXXX product


Capacity of 500 Webcams Integrated phone conferencing Application sharing Have 2 presenters at the same time View a participant’s desktop Playback meeting recordings Assign permissions to those accessing recordings Publicize recordings on a public website 24/7 support Access software from smart phone

18

Results


Label Average95%

Lower95%

UpperItem 1 14.7 12.5 17.0

Item 2 14.4 12.4 16.4

Item 3 10.0 8.6 11.4

Item 4 9.9 8.1 11.8

Item 5 8.7 6.9 10.6

Item 6 7.6 6.4 8.8

Item 7 7.5 5.2 9.8

Item 8 7.4 5.7 9.1

Item 9 6.8 5.2 8.4

Item 10 6.4 4.9 8.0

Item 11 6.4 5.2 7.6

Large variability points to individual

differences. Try

segmenting to look for

groups that have similar

score patterns.

Sum to 100

Case Study: Naming

Client: Tough customers – 9 ½-year-old boy-girl twins and their parents

Objective: Name new dog

Method: MaxDiff with 27 items Guidelines for # of screens to show:

3K/k to 5K/k K = total # of items k = # of items shown per set 3(27)/4 = 20 to 5(27)/4 = 34 3(27)/5 = 16 to 5(27)/5 = 27


Results Label Average 95% Lower 95% Upper

Lucky 8.7 7.9 9.5

Zeus 8.7 8.2 9.2

Buddy 8.4 7.2 9.6

Percy 8.3 7.7 9.0

Friday 6.8 3.7 9.9

Sole 6.7 5.2 8.3

Arlo 5.7 2.0 9.5

Rocky 5.7 1.6 9.8

Neptune 5.0 3.7 6.4

Spock 5.0 1.0 9.1

Snicket 4.9 0.8 9.0

Sparky 4.6 0.0 9.7

Oliver 3.6 0.0 7.7

Shakespeare 3.2 0.1 6.3

Dodger 2.8 0.0 6.8

Sherlock 2.6 0.0 6.2

Washington 2.4 0.0 5.8

Omen 2.0 0.0 6.0

Buckbeak 1.1 0.0 2.4

Bilbo 1.0 0.0 2.9

Cesar 0.9 0.0 2.7

Thor 0.6 0.0 1.2

Clifford 0.5 0.0 0.9

Will 0.4 0.0 0.8

Romulus 0.3 0.0 0.6

Falstaff 0.0 0.0 0.1

Rescaled Scores (0 to 100 scaling)

Small sample size

= large variability

Copyright 2010 Outsource Research

Consulting | All Rights Reserved

Follow-up Take top finishers and run…

Another MaxDiff A Single select/ final choice question A Ranking question


2222

Contact: Nico Peruzzi, [email protected]://twitter.com/NicoPeruzziPhD

Questions?


Max diff scaling for research access(4)

Documents

Transcript of Max diff scaling for research access(4)