Slaying a Sacred Cow - EphMRA
Transcript of Slaying a Sacred Cow - EphMRA
When breakthrough innovation comes from where you least expected it.
Linda GrosjeanOrlando Wood
Slaying a Sacred Cow
q15
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
q15d
185 kg105 kg
200 kg 145 kg
What is the combined weight of Linda and Orlando, when dressed up as a cow?
A: B:
C: D:
a15RIGHT
£1 MILLION!
1197lb
1198lb
The Wisdom of Crowds
How can a crowd be as accurate as or even more accurate than an expert?
We are good at anticipating others’ behaviour, but the truth is we are self-deceit machines
No – not this one!
Using the Wisdom of Crowds to Slay the Sacred Cow
BrainJuicer’s Predictive Markets Technique
• A crowd of 500 respondents (invited on a nationally representative basis)
• Crowd shown up to 15 ideas and asked to judge which is most likely to work in market
• Asked as a share buying/selling game
BrainJuicer’s Predictive Markets Technique
0
10
20
30
40
50
60
-40 -30 -20 -10 0 10 20 30 400
10
20
30
40
50
60
-30 -20 -10 0 10 20 30
Correlation 0.92
Savoury Snacks Parallel Test
Household Cleaners Parallel Test
Correlation 0.79
Predictive Market Result (Net Preference)
Traditional Monadic Result
(Purchase Intention)
Concepts green-lighted in both traditional and Predictive Markets techniques were the same
Predictive Market Picks out the Same Winners as Traditional Testing
Predictive Market Result (Net Preference)
Picking out the Same Winners as Traditional Testing
Monadic Ideas Tested No. of Green Lighted Concepts
Savoury snacks 15 5 out of 5Household cleaners 10 5 out of 5Average 10 out of 10 Sequential-Monadic No. of Green Lighted Concepts
Telecoms UK 14 4 out of 5Telecoms France 14 2 out of 3Telecoms Spain 14 2 out of 3Chilled Food 11 3 out of 3Window cleaners 17 3 out of 4Chewing gum 7 4 out of 4Lighting ideas 20 9 out of 10Average 8.5 out of 10
Greater discrimination vs. traditional tests
-9 -7 -10-3
-14
-26 -30
31
19 177 7
136
22
12 7
4-24
-13
-7
-35
-25
-15
-5
5
15
25
35
% o
f res
pond
ents
dou
blin
g (p
ositi
ve) v
s se
lling
(neg
ativ
e) s
hare
s
0
10
20
30
40
50
60
70
80
90
100
Sequ
entia
l Mon
adic
% D
ef/P
rob
Scores from traditional monadic test can give a flat and unhelpful result
Predictive Market is more discriminating, separating the wheat from the chaff
The Crowd In Action – Linda & John!
Validation of Technique in Pharmaceutical Research
• A series of experiments have been undertaken between Roche and BrainJuicer. For two of these parallel monadic data is available from traditional monadic research among health professionals (a targeted sample)
– Ad concept testing for MabThera RA in a European country
– Actemra Ad concept testing in the US
• The Actemra Ad concept testing was also conducted among Roche employees (an Internal Roche ‘crowd’)
Experiment 1
Traditional480 rheumatologists in 6 countries
Face to face; followed by phone
CHF 400,000
12 weeks
Roche proprietary questions
Wisdom of the Crowd500 ordinary people in one European market
Online interview
£12,000 (25’000 CHF)
2 weeks
Standard questions
Interview flow
Traditional• Assessment of awareness/
perception of current biologics
• Glimpse evaluation
• Detailed evaluation
• Fit of concept with MabThera personality
• Comparative evaluation of each concept
• Ranking of concepts vs. positioning statement
• Link with then existing campaign
Wisdom of the Crowd • Would you probably buy or probably sell
shares in each of these ideas?
• If you owned shares in these ideas, which would you.. Double .. Definitely Sell shares in?
• Which of these faces (next slide) best expresses how doctors would feel about this ad?
• Plus other diagnostic questions
The crowd delivered a very similar result to the targeted sample of rheumatologists
Most encouragingLeast encouraging Encourage to find out moreLeast likely
Most likely ‘Passport
’
‘Boots’
‘Postcard’
‘Switch’
‘New route’
‘Making History’
Net Preference Score=Percentage doubling shares minus percentage
selling shares
‘Switch’
‘Making History’
‘New route’
‘Boots’
‘Passport’
‘Postcard’
= Same Result
Traditional Wisdom of the Crowd
Experiment 2
Traditional202 US rheumatologists
On line interview
$120’000 (122’000 CHF)
10 weeks
Roche proprietary questions
Wisdom of the Crowd500 ordinary US people +
200 Roche Employees
On line interview
£12,000 (25’000 CHF)
Took 2 weeks
Standard questions
Cement Creative Tree Creative
Experiment 2
Same results – with clear winner in Cement advertising concept
Predictive MarketNet Preference Measure
Traditional ResearchPreference Measure
82%
18%
87%
13%
Internal Crowd also returned the same overall result – Cement route is clear winner
Predictive Market External Crowd
Predictive MarketInternal Crowd
82%
18%
93%
7%
January 09 FuturePast
30 - 50 K
Average200 - 500 K
CHFCost of ad concept
marketresearch historically
90% cost reduction feasibleif parallel pilots successful
-> Crowd method as possible new standard
Cost Savings of Predictive Market Approach
3 Pharmaceutical Research Innovations in One
• Reduction of the enormous costs and time associated with ad and concept screening in pharmaceutical research
• Avoids overuse of expensive and rare doctor samples
• Ability to research sensitive subjects that should not go beyond the company walls
What next for Roche and Predictive Markets?
• Continue validation of different types of “emotional research” with non-traditional respondents, including
– Name development– Name testing– Ad concept testing– Brand icon/image testing– Consumer message development and testing