APPLYING OPTIMIZATION FUNDAMENTALS

F O R M A J O R G A I N S

…OR 6 TIPS TO SUPERCHARGE

Y O U R O P T I M I Z A T I O N P R O G R A M

TIP 1:FOCUS ON PROCESS…NOT ‘HACKS’

@jtrondeau

@jtrondeau

WHAT WE MUST

AVOIDREALLY?

Results:No Statistical DifferenceConfidence:N/ATakeaway:The extra field increased lead quality - so keep it!

Results:31.3% Lift In Opt-InsConfidence:95%

Source: Unbounce

@jtrondeau

WHAT WE NEED:PROCESS

TIP 2:MEASURE MULTIPLE METRICS

@jtrondeau

(THAT MATTER)

@jtrondeau

It all starts with identifying goals & KPIs!

@jtrondeau

Top Funnel

Mid Funnel

Bottom Funnel

Engagement metrics, not directlyconnected with business goals.

Robust metrics, directly connected to business goals.

Generally associated with lead generation or ‘relationship’ changing metrics.

@jtrondeau

Page Goals Business Goals

Immediate Conversion

ClicksOpt-Ins

RPVSalesAOV

@jtrondeau

@jtrondeau

Protip: Just because a top funnel metric increased at x% doesn’t mean a deep funnel metric will increase at that rate (or increase at all).

@jtrondeau

@jtrondeau

Results:2.01% Drop in Product Views4.43% Drop in Add-to-Carts10.59% Drop in Orders

BENEFITS OFTESTING MULTIPLEMETRICS

@jtrondeau

#1 GET THE FULL STORYMORE USEFUL METRICS, MORE INSIGHTS

#2 CAN GET INSIGHTS ON LOW SAMPLESIF THERE IS CONSISTENCY BETWEEN PAGES/METRICS YOU MIGHT BE ABLE TO CALL A TEST EARLIER

#3 BETTER ITERATIONSTESTING IS AN ITERATIVE PROCESS, THE MORE DATA YOU HAVE THE MORE YOU CAN IMPROVE.

@jtrondeau

NO STATISTICAL DIFFERENCE

@jtrondeau

@jtrondeau

@jtrondeau

WHAT CAN ILEARN?

@jtrondeau

DESPITE THE SMALLER SAMPLE SIZE FOR THE DEEPER FUNNEL CONVERSION……THE CONSISTENCY IN ACTIONS OVER 3 PAGES AND 8 DAYS MADE US CONFIDENT THAT THIS TEST WAS COMPLETE.

TIP 3:ANALYTICS TO IDENTIFY PAGES

@jtrondeau

@jtrondeau

HOME CATEGORY PRODUCT CART THANK YOU

EVALUATING YOUR FUNNEL WILL

HIGHLIGHT YOUR PROBLEM AREAS

DETAILED FUNNEL ANALYSIS IS YOUR

TEST’S BACKBONE

HOME CATEGORY PRODUCT CART THANK YOU0

25

50

75

100

125

Page Viewed

Uni

que

View

s (In

1,0

00’s

)

PROTIP

@jtrondeau

WE CAN SURMISE A LOT FROM QUANTITATIVE DATA……WE DEVELOP HYPOTHESES WHEN WE EVALUATE QUALITATIVE DATA.(WE’LL TALK ABOUT THIS LATER)

@jtrondeau

PICK PAGESTHAT MATTER

@jtrondeau

#1 HIGH VALUE PAGES WITH A HIGH BOUNCE RATE BOUNCES ARE BAD NEWS BEARS

#2 DEEP FUNNEL PAGESPEOPLE JUMPED THROUGH YOUR HOOPS, BRING THEM HOME

#3 FUNNEL VALLEYSCHECK THE PREVIOUS PAGE TO FIND OUT WHY NO ONE IS MOVING DOWN THE FUNNEL

OR MY PERSONALFAVORITE

@jtrondeau

High Visit, High Bounced Pages

TIP 4:FIX WHAT’S BROKEN…

@jtrondeau

TIP 4:…AND TEST WHAT’S AMBIGUOUS

@jtrondeau

@jtrondeau

@jtrondeau

Just Fix It

Test/Optimize

TIP 5:SCHEDULE A FINITE STOP TIME

@jtrondeau

@jtrondeau

LIFT LOSS NULL

@jtrondeau

@jtrondeau

Results:6.49% Increase In SalesConfidence:95%

@jtrondeau

@jtrondeau

Results:Decreased Add To Cart Clicks by 6.6% For New Visitors. Confidence:95%

@jtrondeau

NO STATISTICAL DIFFERENCE

@jtrondeau

LIFT LOSS

@jtrondeau

“Netflix considers 90% of what they try to be wrong” -- Mike Moran

“80% of the time you/we are wrong about what a customer wants” -- Avinash Kaushik

Microsoft’s statistic from thousands of controlled experiments: only 10-30% of experiments move the metrics they were

designed to improve”— Ronny Kohavi

@jtrondeau

FACTORS THATIMPACT TEST TIMING

#1 LIFT PERCENTAGEWHAT WAS THE LIFT OR LOSS

#2 CONFIDENCE RATE95% OR BETTER

#3 NUMBER OF VARIANTS2 OR MORE

#4 NUMBER OF VARIANTSTESTED POPULATION PER DAY

@jtrondeau

WORK BACKWARD

#1 PICK TIME PERIOD1-6 WEEKS

#2 RECORD UNIQUE TRAFFICWHAT’S THE TRAFFIC FOR THE TIME PERIOD

#3 NUMBER OF VARIANTSDIVIDE TRAFFIC BY VARIANTS

#4 DESIRED CONFIDENCE95%

#4 CALCULATE LIFT PERCENTAGECAN YOU GET X LIFT? IF YES, DO IT!

@jtrondeau

IF YES, THENSCHEDULE

(AND STICK TO IT!)

TIP 6:FOCUS ON THE USER

@jtrondeau

CONVERSION RATE OPTIMIZATION ISN’T ABOUTINCREASING CONVERSION

IT’S ABOUT CREATING MEETING USER EXPECTATIONS

WE NEEDDETAILEDUSERRESEARCH

@jtrondeau

#1 HEATMAPSSEE WHERE PEOPLE ARE CLICKING/SCROLLING

#2 USER SURVEYSGET INFORMATION FROM YOUR BEST USERS

#3 SESSION RECORDINGSWATCH EXACTLY HOW PEOPLE INTERACT ON YOUR SITE

#4 USABILITY TESTSGET SITE TESTERS AND ASK FOR FEEDBACK

QUALITATIVEDATA =COMPETITIVEEDGE

@jtrondeau

@jtrondeau

EXAMPLE OF AHEATMAP

@jtrondeau

EXAMPLE OF ASCROLLMAP

@jtrondeau

HOW A HEATMAPINCREASEDSALES

@jtrondeau

We Saw a 40.9% Increase In DM Lab Sales When We Added A Link To Our Sales Page

@jtrondeau

EXAMPLE OF ASESSIONRECORDING

@jtrondeau

LESSONS LEARNED

@jtrondeau

EXAMPLE OF ASURVEY

@jtrondeau

SOME SWEETSURVEYTIPS

#1 ASK A SINGLE QUESTIONIF ASKING FOR MORE, THEN SET TIME CONSTRAINTS

#2 OPEN ENDED = POWERYOU’LL GET LESS RESPONSES, BUT BETTER DATA

#3 KNOW IT’S A SEGMENTTHE TYPE OF PEOPLE WHO ANSWER SURVEYS AREN’T AN ACCURATE REPRESENTATION OF THE WHOLE

@jtrondeau

• Was it was easy to find what you were looking for?• Were the words/vocab used to define categories/sub-categories clear to you?• Do you have any suggestions to improve our website navigation?• Does our website look credible to you?• Is our pricing clear?• Is there anything else you’d like to know before signing up with us?• Will you shop with us again? Why/why not?• Do you think the form has any confusing/unnecessary input fields?

@jtrondeau

EXAMPLE OF AUSER TEST

@jtrondeau

MY FAVORITETYPE OFQUAL.DATA

COMMON CUSTOMER SERVICE QUESTIONS ANDCOMMON SALES QUESTIONS

@jtrondeau

QUICKRECAP

#1 CLEAN YOUR DATAMAKE SURE YOU TRUST WHAT’S IN GA

#2 GET A TESTING TECHVWO, OPTIMIZELY, ETC…

#3 GET A QUALITATIVE SOURCEHEATMAPS, SURVEYS, CUSTOMER DATA, ETC…

#4 I.D. PROBLEM PAGES & WHY

#5 RUN TEST & REPORT

@jtrondeau

• Email: justin.rondeau@digitalmarketer.com• Twitter: @jtrondeau• Website: digitalmarketer.com

Justin RondeauDirector of Optimization,

Digital Marketer

THANKS! QUESTIONS?