Next Era of SEO: A Guide to SEO Split-Testing

A GUIDE TO SPLIT-TESTING

THE NEXT ERA OF SEO

STANFORD STUDY

Frank George

Risk Aversity Performance

Cautious Fantastic. Recommend for awards.

Cautious Terrible. Close to getting fired.

FIREFIGHTER TRAITS

Frank

George

Students were now told:

We made it up. Entirely fictional!

!

How should a firefight treat risk?

They should be cautious!

They should embrace risk. It is a dangerous job!

STUDENTS NOW ASKED:

Frank

George

People cannot let go of old beliefs.

See what they want to see.

!

Have you recommended ALT attributes on images

for SEO reasons?

?

ALT ATTRIBUTES GOOD FOR SEO

TEST RESULT: ALT ATTRIBUTES HAD NO IMPACT

Google algorithm is too complex.

We are too biased.

!

Blog Post

https://engineering.pinterest.com/blog/demystifying-seo-experiments

Blog Post

http://www.brickmarketing.com/blog/aris-vrakas.htm

Blog PostBlog Post

http://www.brickmarketing.com/blog/aris-vrakas.htm

https://codeascraft.com/2017/01/25/optimizing-meta-descriptions-h1s-and-title-tags-lessons-from-multivariate-seo-testing-at-etsy/

HOW YOU CAN TEST — FREE TOOL

‘TRADITIONAL’ CRO A/B TESTING IS USER ORIENTED

Visitors A & B: control version

of a page

Visitors C & D: test version

of the same page

SEO A/B TESTING IS PAGE ORIENTED

Pages 1, 2, 3: Control Layout

Pages 4, 5, 6: Test Layout

All users see the same as each other

SEO A/B TESTING INCLUDES GOOGLEBOT

Googlebot sees the same as users

Pages 1, 2, 3: Control Layout

Pages 4, 5, 6: Test Layout

OVERVIEW

SEO SHOULD BE INFORMED BY DATA & TESTS

1

2009: ALGORITHM IS PRETTY WELL UNDERSTOOD

No Panda No Penguin No RankBrain No Machine Learning

We know what to do.

AMIT SINGHAL - HEAD OF SEARCH, GOOGLE

See this fascinating comment from a user called Kevin Lacker (@lacker):

https://news.ycombinator.com/item?id=11032761

https://news.ycombinator.com/user?id=lacker

I was thinking about it like it was a math puzzle and if I just thought

really hard it would all make sense.

Kevin Lacker

Hey why don't you take the square root?

Amit Singhal according to Kevin Lacker

I was like, why a square root, it doesn't make much sense

Kevin Lacker

Multiply by 2 if it helps, add 5, whatever, just make things work and

we can make it make sense later.

Amit Singhal according to Kevin Lacker

THE ALGORITHM WAS MOSTLY HANDMADE

ALGORITHM IS TOO COMPLEX TO VISUALISE

EARLY 2016: AMIT SINGHAL RETIRES

JOHN GIANNANDREA TAKES OVER

PREVIOUSLY HEAD OF MACHINE LEARNING

IMPACT OF MACHINE LEARNING


Translate

10 YEARS RESEARCH

2 MONTHS MACHINE LEARNING

Imagine the same step change in the search algo…

It will be extremely hard to understand many rankings.

Even for Google.

!

No, but I’m still pretty good at this

You’re thinking this to yourself right now.

How good are we?

?

Google Information Needs Study (NBED LINK)

http://adage.com/article/media/future-tv/303508/

SEARCH: future of tv advertising

Google Information Needs Study (NBED LINK)

http://contentmarketinginstitute.com/2016/04/television-advertising-native/


DA PA LRDs Rank

http://dis.tl/rankq-1a

91 47 2 ?

http://dis.tl/rankq-1b

86 57 9 ?


DA PA LRDs Rank


91 47 2 3


86 57 9 8


https://www.jensinkler.com/fix-your-kettlebell-swing/

SEARCH: kettlebell technique

http://www.mensfitness.com/training/pro-tips/perfect-kettlebell-swing


DA PA LRDs Rank


41 42 12 ?


76 44 20 ?


DA PA LRDs Rank


41 42 12 4


76 44 20 6


http://www.macworld.co.uk/how-to/iphone/how-to-back-up-iphone-ipad-backup-3595444/

SEARCH: do I need to backup my iphone

http://www.idownloadblog.com/2013/08/28/not-enough-icloud-storage-iphone-cannot-be-backed-up/


DA PA LRDs Rank


77 59 16 ?


67 34 3 ?


DA PA LRDs Rank


77 59 16 7


67 34 3 5


How do we compare?

?

WE SURVEYED 2000 PEOPLE FOR 10 QUERIES

% OF UK SEARCH RESULT PAIRS PREDICTED CORRECTLY

30%

40%

50%

60%

70%

Laypeople


30%

40%

50%

60%

70%

Laypeople <= 3 yrs SEO


30%

40%

50%

60%

70%

Laypeople <= 3 yrs SEO > 3 yrs SEO


30%

40%

50%

60%

70%

Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip


30%

40%

50%

60%

70%

Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip DeepRank

TESTING - STOP GUESSING WHAT WORKED

Testing the same type of change on different sites can have differing effects.

THE CASE FOR TESTING

1) Predicting effect in algo is really hard (no best practice)



2) Some sites respond differently to others for same change




3) Machine Learning impact on algo will make this worse




3) Machine Learning impact on algo will make this worse

4) Optimise further than previously possible

2

SOME RESULTS FROM OUR SEO TESTS

DistilledODN

Your Server Your Home Page

CDN

Distilled ODN

Your Optimized Home Page

2 IMPORTANT GRAPHS. THIS IS THE FIRST.

BLACK IS THE VARIANT, TREAT BLUE AS THE CONTROL

BLACK (ACTUAL) HIGHER THAN BLUE (CONTROL) - POSITIVE

THIS IS THE START DATE

SECOND GRAPH: TOTAL ADDITIONAL SESSIONS

SHADED AREA SHOWS CONFIDENCE INTERVAL

TEST IS SIGNIFICANT WHEN THE FAN CROSSES 0

IN THIS CASE IT HAPPENED HERE:

HERE THEY ARE TOGETHER.

ALT ATTRIBUTES

1

HYPOTHESIS: ALT attributes on images are good for SEO

RESULT: ALT attributes had no effect

SEO ‘CONTENT’

2

HYPOTHESIS: Text content on category pages is sort of good?

RESULT: No real impact when removed

RESULT: A 3.1% improvement in organic performance

Not every site is the same.

Identical changes can have different results.

!

JAVASCRIPT RENDERING

3

HYPOTHESIS: Javascript impacts Google’s crawling / indexation

MAY 2014 “our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on.

OCT 2015 “We are generally able to render and understand your web pages like modern browsers.”

Javascript EnabledJavascript Disabled

SITUATION: Javascript needed to render product list

RESULT: CSS fallback led to a 6.2% uplift

TITLES: HIGHEST SEARCH VOLUME

4

Title tag before: Which TV should I buy? - Argos Title tag after: Which TV to buy? - Argos

HYPOTHESIS: Title tags should match greatest search volume term

RESULT: Organic sessions decreased by average 8%

STRUCTURED MARKUP ON CATEGORY PAGES

5

SITUATION: Category pages have lots of images but little text

HYPOTHESIS: Structured Markup per product will help

RESULT: Organic sessions increase by 11%

3

HOW YOU CAN RUN TESTS YOURSELVES

(+ A FREE TOOL)

3 STEPS TO DIY SEO SPLIT-TESTS

1. Create two buckets of pages.

2. Make a change to all pages in one bucket.

3. Analyse which bucket performs better.

1. CREATE TWO BUCKETS OF PAGES.

A good DIY approach is using GA segments. Easy to make (see link). Suggestion: Create Segments by category (e.g. blog tag, product category).

https://support.google.com/analytics/answer/3124493?hl=en

At least 1,000 organic sessions per day to each bucket

!

2. MAKE A CHANGE TO ALL PAGES IN ONE BUCKET.

Control Variant

This will be your variant bucket. Utilise your CMS where possible. Ask Dev for scalable approach.

3. ANALYSE WHICH BUCKET PERFORMS BETTER.

distilled.net/diy-splittester

http://distilled.net/diy-splittester

Typically takes 2 - 3 weeks to reach significance.

!

READ THIS:

Blog Post

http://dis.tl/will-abtest

THEN READ THIS:

Blog Post

https://www.distilled.net/resources/do-it-yourself-seo-split-testing-tool-with-causal-impact/

4

SUMMARY

SPLIT-TESTING IS BECOMING MORE NECESSARY

30%

40%

50%

60%

70%

Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip

SPLIT-TESTING IS BECOMING MORE ESTABLISHED

SPLIT-TESTING MAKES YOU MORE CREDIBLE

SPLIT-TESTING IS ALREADY ACCESSIBLE

Step change in your SEO approach.

Start now.

!

WWW.DISTILLEDODN.COM

CHECK OUT OUR SEO A/B TESTING PLATFORM:

@TomAnthonySEO

@Dom_Woodman

@WillCritchlow

DIS.TL/SMXDE17GET THE DECK:

http://www.distilledodn.com

http://dis.tl/SMXDE17

Next Era of SEO: A Guide to SEO Split-Testing

Internet

Transcript of Next Era of SEO: A Guide to SEO Split-Testing