How to Execute and A/B Test Cross-Channel Personalization

WEBINAR: EXECUTING AND A/B TESTING CROSS-CHANNEL PERSONALIZATION

Executing and A/B Testing Cross-Channel Personalization

Andy Zimmerman, Evergage Shanelle Mullin, ConversionXL


Agenda• The Cross-Channel Imperative • Challenges • Solution Requirements • Examples • A/B Test Process • Prioritization • Duration • Results and Archiving • Q & A


The Cross-Channel Imperative


Challenge: Cross-Channel is Happening Now


70% OF WALMART.COM

TRAFFIC LAST HOLIDAY SEASON

WAS MOBILE DEVICES.

Forrester: Your Digital Experience Technology Starts With A Customer Journey Map


69%


$1.5 Trillion retail sales generated

by cross-channel interactions – Forrester, The Omnichannel Commerce Playbook for 2016


47% of bank customers engaged in

at least one cross-channel interaction over 90 day period.– Forrester, Design Better Cross-Channel Banking Journeys


Top Goal is Customer Experience


4 Key Elements of a Great Customer Experience

4

1 Remember me.

2 Understand me.

3 Help me achieve my goals.

Surprise me. Delight me.


Cross-Channel Personalization


77% of consumers have chosen, recommended, or paid more for a

brand that provides a personalized service or experience.

– Forrester’s North American Consumer Technographics® Brand Compass Survey, Q3 2015


The Cross-Channel Challenge


Challenge: Organizational Misalignment

– Forrester, The Omnichannel Commerce Playbook for 2016


Challenge: Data is SiloedMarketers are using SIX or more tools to collect performance data.

Marketing Analytics Tool

Website Analytics Tool

Marketing Automation Tool

Operational Analytics Tool

Business Intelligence System

Sales Automation Tool

Telemarketing or Online Surveys

Channel Performance/Analytics Tool

CRM System

ERP System

0% 20% 40% 60% 80%

30%

42%

43%

44%

54%

54%

57%

57%

69%

79%


Requirements for a Cross-Channel Personalization Solution


Track Behavior Across ChannelsWebsite Email SearchMobile

Web App


Synch Data with Other Systems

CRM

Marketing Automation

Email Marketing

Data Warehouse

Data Lake

Call Center


One Unified Customer Profile


Core Capabilities of Personalization Platform

Cross-channel Targeting

Algorithmic Recommendations

Real-Time Segmentation

A/B & MV Testing

Analysis & Attribution

Behavioral Tracking

Unified Customer Profile


Cross-Channel Personalization Maturity Evolution

Beginning Intermediate Part A

Intermediate Part B Advanced

Step 1 Digital Channel 2 or 3 Digital Channels

Synch with other systems

All channels, including agents

Web Mobile App Open-time Email Search

Web Mobile App Open-time Email Search

CRM System Email/MAP Data Warehouse

Call Center In-Store Kiosk Reps/Agents

Benefit One Team Multiple Teams Better data, Targeting x-channel Organization-wide

Challenge Getting Started Cross-Team coordination needed

Coordinate synch if not out-of-box, APIs

Additional Integrations


Examples of Cross-Channel Personalization


Ski Resort Example


Rue La La Example


How to A/B Test


Only 32% of small companies and 42% of large companies have a structured, documented process.


1. Research 2. Prioritize 3. Experiment 4. Analyze and Learn


Prioritization Problems

1. Ease and impact are too subjective 2. Too rigid. 3. Doesn’t encourage a data-driven culture


Prioritization Fixes

1. Binary responses 2. Customizable variables 3. Ask everyone to bring data


Grab the Template

bit.ly/pxlmodel


Determining How Long to Test

1. Calculating sample size 2. Running for full weeks, usually two cycles


Use the Calculator

bit.ly/evansample


Ensuring a Representative Sample

1. Every day of the week 2. Various traffic sources 3. Ad, email, social, content schedule 4. External factors (e.g. holidays) 5. Comparison shoppers


On External Factors...

1. Season 2. Day of the Week 3. Press 4. PPC / SEM / SEO 5. Word of Mouth


On Sample Pollution...

1. Length pollution 2. Device pollution 3. Browser pollution 4. Cookie pollution


Watch Out...

1. Regression to the mean 2. The novelty effect 3. Simultaneous testing


Creating a Hypothesis

1. Is it designed to solve a conversion problem? 2. Is it measurable / testable? 3. Will you gain insight, win or lose?


Skills That Help with Testing

1. HTML, CSS, JavaScript, jQuery 2. Design 3. Copywriting


Get the Details

bit.ly/gaabresults


No Difference? Not So Fast...

1. Maybe the implementation sucked. 2. The variation might’ve won in a segment.


Segmentation 101

1. Desktop vs. Mobile 2. New vs. Returning 3. Traffic Direct on the Page vs. Traffic via Internal Link


Insights > Wins


Why Archive?

1. To avoid repeating tests 2. To better communicate wins and insights 3. To improve the quality of future tests 4. To create an insights > wins culture


Archive Methods

1. Iridion 2. Effective Experiments 3. GrowthHackers Projects 4. Experiment Engine 5. Testing Tool 6. Excel


Conversion optimization training program.

evergage-webinar


Questions?

How to Execute and A/B Test Cross-Channel Personalization

Marketing

Transcript of How to Execute and A/B Test Cross-Channel Personalization