The Evolution of A/B Testing - NMHC Optech 2016
-
Upload
nicole-mintiens -
Category
Marketing
-
view
258 -
download
3
Transcript of The Evolution of A/B Testing - NMHC Optech 2016
![Page 1: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/1.jpg)
THE EVOLUTION
OF A/B TESTING
@Nicole_Mintiens
Data Analysis & Insights Manager @GetG5
![Page 2: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/2.jpg)
41 Shades of Blue
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 3: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/3.jpg)
$200 MILLION
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 4: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/4.jpg)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 5: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/5.jpg)
Advertising and Marketing• Improved ROI = Good
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 6: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/6.jpg)
Advertising and Marketing• Improved ROI = Good• Wasted Resources = Bad
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 7: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/7.jpg)
Advertising and Marketing• Improved ROI = Good• Wasted Resources = Bad• Marketing = Complex
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 8: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/8.jpg)
Advertising and Marketing• Improved ROI = Good• Wasted Resources = Bad• Marketing = Complex• Facts > Opinion
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 9: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/9.jpg)
WHY SPEND TIME TESTING?• $92 traffic /$1 optimization
Source: Econsultancy & Redeye Conversion Rate Optimization Report 2014
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 10: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/10.jpg)
WHY SPEND TIME TESTING?• $92 traffic /$1 optimization• 87% run 1-5 tests/month
Source: Econsultancy & Redeye Conversion Rate Optimization Report 2014
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 11: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/11.jpg)
WHY SPEND TIME TESTING?• $92 traffic /$1 optimization• 87% run 1-5 tests/month
– Best brands run hundreds per month
Source: Econsultancy & Redeye Conversion Rate Optimization Report 2014
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 12: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/12.jpg)
A/B TESTING FACTS• Improves conversion rate
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 13: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/13.jpg)
A/B TESTING FACTS• Improves conversion rate• Enhances effectiveness
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 14: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/14.jpg)
A/B TESTING FACTS• Improves conversion rate• Enhances effectiveness• Optimizes ROI
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 15: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/15.jpg)
A/B TESTING FACTS• Improves conversion rate• Enhances effectiveness• Optimizes ROI
So… YOU should test!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 16: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/16.jpg)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 17: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/17.jpg)
MISCONCEPTIONS• Hurt conversion rate
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 18: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/18.jpg)
MISCONCEPTIONS• Hurt conversion rate• Reduce effectiveness
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 19: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/19.jpg)
MISCONCEPTIONS• Hurt conversion rate• Reduce effectiveness• Waste resources
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 20: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/20.jpg)
MISCONCEPTIONS• Hurt conversion rate• Reduce effectiveness• Waste resources• Damage ROI
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 21: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/21.jpg)
TESTING IS NOT
OPTIMIZATION
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 22: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/22.jpg)
TESTING IS NOT
OPTIMIZATION…but it plays a role.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 23: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/23.jpg)
DON’T JUST TEST, OPTIMIZE.• Avoid mistakes
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 24: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/24.jpg)
DON’T JUST TEST, OPTIMIZE.• Avoid mistakes• Increase effectiveness
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 25: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/25.jpg)
DON’T JUST TEST, OPTIMIZE.• Avoid mistakes• Increase effectiveness• Improve ROI
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 26: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/26.jpg)
I’LL BE YOUR GUIDE• 7+ years optimizing multifamily
– PPC, SMM, SEO & CRO
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 27: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/27.jpg)
A/B TESTING BASICS• Statistical hypothesis-based
experimentation between variations of a control.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 28: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/28.jpg)
A/B TESTING BASICS• Statistical hypothesis-based
experimentation between variations of a control.
• Goal: – Isolate differences that
maximize an outcome • i.e. conversion
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 29: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/29.jpg)
CONVERSION DEFINITIONAction YOU want the user to take.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 30: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/30.jpg)
A/B/N TESTING
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 31: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/31.jpg)
A/B/N TESTING
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 32: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/32.jpg)
A/B/N TESTING
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 33: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/33.jpg)
• 2000 – Google runs first test to find optimum number of SERP
results.
A/B TESTING HISTORY
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 34: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/34.jpg)
• 2000 – Google runs first test to find optimum number of SERP
results.
• 2008 – A/B tests raise additional $60 million for Obama campaign.
A/B TESTING HISTORY
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 35: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/35.jpg)
• 2000 – Google runs first test to find optimum number of SERP
results.
• 2008 – A/B tests raise additional $60 million for Obama campaign.
• 2010– Obama campaign’s Director of Analytics founds Optimizely.
A/B TESTING HISTORY
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 36: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/36.jpg)
• 2000 – Google runs first test to find optimum number of SERP
results.
• 2008 – A/B tests raise additional $60 million for Obama campaign.
• 2010– Obama campaign’s Director of Analytics founds Optimizely.
• 2011 – Google has run over 7,000 A/B tests
A/B TESTING HISTORY
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 37: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/37.jpg)
MULTIFAMILY AVG CONVERSION RATE
Source: 2016 G5 Multifamily Google Analytics Data
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 38: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/38.jpg)
MULTIFAMILY AVG CONVERSION RATE
Source: 2016 G5 Multifamily Google Analytics Data
~2%The Evolution of A/B Testing
@Nicole_Mintiens
![Page 39: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/39.jpg)
ECOMMERCE AVG CONVERSION RATE
~3%The Evolution of A/B Testing
@Nicole_Mintiens
![Page 40: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/40.jpg)
AMAZON PRIME’S CONVERSION RATE
Source: https://www.internetretailer.com/2015/06/25/amazon-prime-members-convert-74-time
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 41: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/41.jpg)
2366%Better than
average!
AMAZON PRIME’S CONVERSION RATE
Source: https://www.internetretailer.com/2015/06/25/amazon-prime-members-convert-74-time
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 42: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/42.jpg)
AMAZON’S TECHNOLOGY
Source: Builtwith.com
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 43: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/43.jpg)
AMAZON’S TECHNOLOGY
Source: Builtwith.com
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 44: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/44.jpg)
SOFTWARE ADVANCEMENTS• Marketing automation
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 45: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/45.jpg)
SOFTWARE ADVANCEMENTS• Marketing automation • Easy to install
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 46: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/46.jpg)
SOFTWARE ADVANCEMENTS• Marketing automation • Easy to install• WYSIWYG editors
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 47: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/47.jpg)
WHAT A/B TESTING CAN DO
• Validate hypotheses
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 48: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/48.jpg)
WHAT A/B TESTING CAN DO
• Validate hypotheses• Inform choices
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 49: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/49.jpg)
WHAT A/B TESTING CAN DO
• Validate hypotheses• Inform choices • Insight for iteration
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 50: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/50.jpg)
WHAT A/B TESTING CAN’T DO• Determine YOUR
–Problem
Thanks, Boss!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 51: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/51.jpg)
WHAT A/B TESTING CAN’T DO• Determine YOUR
–Problem –User’s
needs/wantsThanks, Boss!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 52: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/52.jpg)
WHAT A/B TESTING CAN’T DO• Determine YOUR
–Problem –User’s
needs/wants
• Improve traffic–Quality
Thanks, Boss!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 53: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/53.jpg)
WHAT A/B TESTING CAN’T DO• Determine YOUR
–Problem –User’s
needs/wants
• Improve traffic–Quality–Volume
Thanks, Boss!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 54: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/54.jpg)
CAN DOES NOT EQUAL SHOULDTesting will waste your time without:• Enough traffic
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 55: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/55.jpg)
CAN DOES NOT EQUAL SHOULDTesting will waste your time without:• Enough traffic• Enough research
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 56: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/56.jpg)
CAN DOES NOT EQUAL SHOULDTesting will waste your time without:• Enough traffic• Enough research• Technical resources
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 57: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/57.jpg)
THEN HOW DO YOU IMPROVE?!
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 58: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/58.jpg)
A PROCESSto improve performance
informed by INSIGHTand defined by GOALS.
CONVERSION RATE OPTIMIZATION(CRO)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 59: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/59.jpg)
A PROCESSto improve performance
informed by INSIGHTand defined by GOALS.
GOALS
CONVERSION RATE OPTIMIZATION(CRO)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 60: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/60.jpg)
ALWAYS ROOM FOR IMPROVEMENT• Customer acquisition cost
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 61: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/61.jpg)
ALWAYS ROOM FOR IMPROVEMENT• Customer acquisition cost• Return on investment
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 62: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/62.jpg)
ALWAYS ROOM FOR IMPROVEMENT• Customer acquisition cost• Return on investment• Customer understanding
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 63: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/63.jpg)
ALWAYS ROOM FOR IMPROVEMENT• Customer acquisition cost• Return on investment• Customer understanding• Google’s perception
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 64: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/64.jpg)
GOALS & TRACKING
Source: http://borsen.dk/opinion/blogs/view/17/4159/den_svaereste_marketingdisciplin.html
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 65: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/65.jpg)
YOUR IDEAL RESIDENTS
Prospects
Valu
e
I only rent the most affordable apartments.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 66: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/66.jpg)
YOUR IDEAL RESIDENTS
Prospects
Valu
e
I only rent the most affordable apartments.
I’ll rent near my tech coworkers regardless of price.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 67: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/67.jpg)
CUSTOMER RESEARCH
Source: https://blog.kissmetrics.com/wp-content/uploads/2015/02/NYU.lowres.ifiwere.jpg
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 68: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/68.jpg)
“The EFFECTIVENESS of
any lead generation campaign
comes down to your HOOK.”Mike King @IPullRankFounder iPullRankConsultant to Fortune/INC500 brands.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 69: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/69.jpg)
VOICE OF CUSTOMER RESEARCH• What makes your
property unique?
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 70: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/70.jpg)
VOICE OF CUSTOMER RESEARCH• What makes your
property unique?
• Why you and not the competition?
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 71: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/71.jpg)
VOICE OF CUSTOMER RESEARCH• What makes your
property unique?
• Why you and not the competition?
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 72: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/72.jpg)
UNIQUE VALUE PROPOSITION (UVP):Above-the-fold EXPERIENCEthat communicates WHATyou do and WHYit’s UNIQUE.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 73: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/73.jpg)
Original (A)
UVP: FIVE SECOND TEST
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 74: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/74.jpg)
Original (A) Variation (B)
UVP: FIVE SECOND TEST
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 75: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/75.jpg)
Original (A) Variation (B)
UVP: FIVE SECOND TEST
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 76: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/76.jpg)
CUSTOMER SEGMENTATION
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 77: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/77.jpg)
CUSTOMER SEGMENTATION
User Persona Creator: https://xtensio.com/user-persona/
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 78: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/78.jpg)
Slide credit to Angie Schottmuller - Check out the full presentation:
http://www.slideshare.net/aschottmuller/content-marketing-onpage-optimization-holy-grail
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 79: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/79.jpg)
LARGEST LEAKS
Image source: https://s-media-cache-ak0.pinimg.com/564x/7e/c9/f9/7ec9f92ab4e7819c76011e432e760ece.jpg
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 80: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/80.jpg)
LARGEST LEAKS
Image source: https://s-media-cache-ak0.pinimg.com/564x/7e/c9/f9/7ec9f92ab4e7819c76011e432e760ece.jpg
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 81: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/81.jpg)
ANALYZE ENGAGEMENT PATTERNSHeatmaps
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 82: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/82.jpg)
ANALYZE ENGAGEMENT PATTERNSHeatmaps
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 83: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/83.jpg)
ANALYZE PERCEPTIONS
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 84: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/84.jpg)
ANALYSIS FRAMEWORKG5 D-LITE
Deliver Relevance - Match then exceed user expectations.D
L
I
T
EThe Evolution of A/B Testing
@Nicole_Mintiens
![Page 85: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/85.jpg)
ANALYSIS FRAMEWORKG5 D-LITE
Lower Friction - Reduce user anxiety. Simplify their process.
Deliver Relevance - Match then exceed user expectations.D
L
I
T
EThe Evolution of A/B Testing
@Nicole_Mintiens
![Page 86: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/86.jpg)
ANALYSIS FRAMEWORKG5 D-LITE
Improve Clarity - Ensure messaging is clear, concise & valued.
Lower Friction - Reduce user anxiety. Simplify their process.
Deliver Relevance - Match then exceed user expectations.D
L
I
T
EThe Evolution of A/B Testing
@Nicole_Mintiens
![Page 87: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/87.jpg)
ANALYSIS FRAMEWORKG5 D-LITE
Improve Clarity - Ensure messaging is clear, concise & valued.
Lower Friction - Reduce user anxiety. Simplify their process.
Deliver Relevance - Match then exceed user expectations.
Trigger Action - Motivate conversion. Remove distraction.
D
L
I
T
EThe Evolution of A/B Testing
@Nicole_Mintiens
![Page 88: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/88.jpg)
ANALYSIS FRAMEWORKG5 D-LITE
Improve Clarity - Ensure messaging is clear, concise & valued.
Lower Friction - Reduce user anxiety. Simplify their process.
Deliver Relevance - Match then exceed user expectations.
Experiment - Learn, test and optimize for your users.
Trigger Action - Motivate conversion. Remove distraction.
D
L
I
T
EThe Evolution of A/B Testing
@Nicole_Mintiens
![Page 89: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/89.jpg)
“A solid test hypothesis is an
INFORMED SOLUTIONto a real problem - not an arbitrary guess. ”Michael Aagaard
@ContentVerveInternational Speaker and Senior CRO @Unbounce
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 90: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/90.jpg)
HYPOTHESIS TEMPLATEIf [cause], then [effect], because [rationale].
Source: https://help.optimizely.com/Ideate_and_Hypothesize/Design_a_hypothesis_that_drives_your_business_goals
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 91: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/91.jpg)
TESTING
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 92: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/92.jpg)
TESTING Wait… do you even have enough traffic?
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 93: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/93.jpg)
Gives you CONFIDENCE that the changes you make to
your website ACTUALLYhave a positive IMPACT.
https://www.optimizely.com/optimization-glossary/statistical-significance/
STATISTICAL SIGNIFICANCE
“”
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 94: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/94.jpg)
MULTIFAMILY AVG SESSIONS/DAY
Source: 2016 G5 Multifamily Google Analytics Data
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 95: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/95.jpg)
MULTIFAMILY AVG SESSIONS/DAY
Source: 2016 G5 Multifamily Google Analytics Data
66The Evolution of A/B Testing
@Nicole_Mintiens
![Page 96: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/96.jpg)
TEST DURATION CALCULATOR
Source: https://www.convert.com/tools/ab-test-duration-calculator/
Image source: http://www.vintag.es/2016/09/the-good-old-days-12-crazy-vintage-ads.html
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 97: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/97.jpg)
TEST DURATION CALCULATOR
Source: https://www.convert.com/tools/ab-test-duration-calculator/
Image source: http://www.vintag.es/2016/09/the-good-old-days-12-crazy-vintage-ads.html
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 98: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/98.jpg)
LOW TRAFFIC OPTIMIZATION
Source: http://unbounce.com/landing-pages/i-dont-have-enough-traffic-to-ab-test-my-landing-pages/
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 99: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/99.jpg)
A/B TESTING TOOLS
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 100: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/100.jpg)
A/B TESTING TOOLS
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 101: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/101.jpg)
BUILD VARIATIONSUse technical resources to ensure proper setup and tracking
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 102: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/102.jpg)
BUILD VARIATIONSUse technical resources to ensure proper setup and tracking
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 103: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/103.jpg)
LAUNCH TESTOriginal
(A)Variation
(B)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 104: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/104.jpg)
LAUNCH TESTOriginal
(A)Variation
(B)
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 105: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/105.jpg)
MEASURE RESULTS
Image source: https://www.optimizely.com/ab-testing/
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 106: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/106.jpg)
MEASURE RESULTS
Image source: https://www.optimizely.com/ab-testing/
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 107: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/107.jpg)
DRAW INSIGHTS AND ITERATE
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 108: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/108.jpg)
Your properties aren't 1-size fits all and your strategy shouldn't be either.
DRAW INSIGHTS AND ITERATE
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 109: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/109.jpg)
MOST IMPORTANTLYResearch and
user feedback is
VITAL.
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 110: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/110.jpg)
REPEAT
GOALS
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 111: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/111.jpg)
THERE IS NO
BEST COLOR
41 Shades of Blue
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 112: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/112.jpg)
INDUSTRY BENCHMARKS
• Contextless data
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 113: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/113.jpg)
INDUSTRY BENCHMARKS
• Contextless data• Weak conclusions
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 114: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/114.jpg)
INDUSTRY BENCHMARKS
• Contextless data• Weak conclusions• Blind duplication
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 115: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/115.jpg)
INDUSTRY BENCHMARKS
• Contextless data• Weak conclusions• Blind duplication • Tactics over strategy
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 116: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/116.jpg)
USE STRATEGY,NOT JUST TACTICS
• Avoid risks
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 117: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/117.jpg)
USE STRATEGY,NOT JUST TACTICS
• Avoid risks• Use data effectively
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 118: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/118.jpg)
USE STRATEGY,NOT JUST TACTICS
• Avoid risks• Use data effectively• Discover solutions to
YOUR problems
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 119: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/119.jpg)
FREEMIUM (OR CLOSE) RESOURCES• Research:
– Personas
– Tracking
– Heatmapping
– Test Duration Calculator
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 120: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/120.jpg)
FREEMIUM (OR CLOSE) RESOURCES• Research:
– Personas
– Tracking
– Heatmapping
– Test Duration Calculator
• Analysis Tools:
G5 D-LITE CRO Framework
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 121: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/121.jpg)
FREEMIUM (OR CLOSE) RESOURCES• Research:
– Personas
– Tracking
– Heatmapping
– Test Duration Calculator
• Analysis Tools:
G5 D-LITE CRO Framework
• Hypotheses:
If [cause], then [effect], because [rationale]
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 122: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/122.jpg)
FREEMIUM (OR CLOSE) RESOURCES• Research:
– Personas
– Tracking
– Heatmapping
– Test Duration Calculator
• Analysis Tools:
G5 D-LITE CRO Framework
• Hypotheses:
If [cause], then [effect], because [rationale]
• Testing:
The Evolution of A/B Testing
@Nicole_Mintiens
![Page 123: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/123.jpg)
“CRO IS LEARNING WHYYOUR USERS aren’t
converting and FIXING IT.”The Evolution of A/B Testing
@Nicole_Mintiens
![Page 124: The Evolution of A/B Testing - NMHC Optech 2016](https://reader035.fdocuments.net/reader035/viewer/2022062906/58ecd4791a28abc22a8b4697/html5/thumbnails/124.jpg)
GET THESE SLIDES: goo.gl/TPUOgA
THANKS!
DON’T JUST TEST, OPTIMIZE.
The Evolution of A/B Testing
@Nicole_Mintiens