Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings

download Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings

of 132

  • date post

    22-Jan-2018
  • Category

    Marketing

  • view

    13.730
  • download

    0

Embed Size (px)

Transcript of Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings

  • Bartosz GralewiczELEPHATE

    Can Google properly crawl and index JavaScript?SEO Experiments - Results and findings@bart_goralewiczhttps://www.linkedin.com/in/goralewicz/

  • Why is JavaScript SEO such a complex topic?

  • DATANO

    Why is JavaScript SEO such a complex topic?

  • DATANO

    Making JavaScript SEO easy

  • JS SEO

  • JS SEO

  • What is the problem with JavaScript?HTML

    JS

  • What is the problem with JavaScript?HTML

    HTML

  • What is the problem with JavaScript?HTML

    HTML

  • JavaScript

  • and JavaScript

  • JavaScript & SEO

  • 50 shades of JavaScript

  • *more about it later

    not as forgiving as HTML*

  • It has all with

  • Reverse engineering algorithms is no longer an

    option.

    Times have changed. Today, as long as you're not blocking Googlebot from crawling your JavaScript or CSS files, we are generally able to render and understand your web pages like modern browsers. To reflect this improvement, we recently updated our technical Webmaster Guidelines to recommend against disallowing Googlebot from crawling your site's CSS or JS files.

    ele.ph/ajax

  • ele.ph/scotch

  • Can this backfire?

  • Hulu.com JavaScript & SEO backfire

  • -37%

  • ele.ph/hulu

    Hulu case study

  • Google and JavaScript difficult relationship

  • The only solution

    Java Script Experiment

  • KamilGrymuza

  • http://jsseo.expert

  • Content!

    OK.

  • ele.ph/AI

  • Inspect Element

    vs.

    View Source

  • Inspect element

  • View source

  • First JavaScript rendering issues

  • Timeout, GSC bug or real ?

  • /test/

  • JavaScript links were not parsed by Googlebot

    Google kept us

    waiting

  • Fail?

  • Re-fetched all URLs via GSC

  • /html/test/ (duh!)/plain-inline/test//jquery-inline/test//jquery-ajax/test//vue/test//react-inline/test/

    And thats it! /test/

  • Lets start with JS frameworks created by

  • Version 1

  • Checklist

    Check if framework/test/ URL was crawled

    Search for unique content from frameworks page

    Is URL indexed by Google?

    Fetch and render test (GSC)

  • Fetch and render test (GSC)

  • Is URL indexed by Google?

  • Search for unique content from frameworks page

  • Check if framework/test/ URL was crawled

  • V2 Inline

    GSCfetch

    URLIndexed?ContentIndexed?

    TestURL

    Inline Inline

  • GSCfetch

    URLIndexed?

    ContentIndexed?

    TestURL

    External Ajax callInline

  • GSCfetch

    URLIndexed?

    ContentIndexed?

    TestURL

    Inline External

  • GSCfetch

    URLIndexed?ContentIndexed?

    TestURL

    Inline External

  • Inline vs. External JS makes a huge difference

    Not all JavaScript frameworks are crawled and indexed in the same way

    Mindblowing findings

    JS generated links arent always crawled

  • ele.ph/angularU

    Also

    Regardless of what your developers say, Angular JS 2 always has to be server rendered

    according toAngular JS 2 community.

  • If you search for any competitive keyword terms, its always going to be server rendered sites. And the reason is because although Google does index client-siderendered HTML, its not perfect yet and other search engines dont do it as well. So if you care about SEO, you still need to have server-rendered content.

    Jeff Whelpley

    Angular U conference, June 22-25, 2015, Hyatt Regency, San Francisco AirportAngular 2 Server Rendering

    ele.ph/angularU

  • ?How about Search Engines other than

  • 63,4%

    22,8%

    11,7%

    1,3% 0,8%

    Google Bing Yahoo! Ask AOL

    Search Engine market share

    Source: https://www.statista.com/statistics/267161/market-share-of-search-engines-in-the-united-states/

  • 83,5%

    11,1%3,8% 0,8% 0,6%

    Google Bing Yahoo! MSN DuckDuckGo

    Search Engine market share

    Source: https://www.statista.com/statistics/280269/market-share-held-by-search-engines-in-the-united-kingdom/

  • ele.ph/beyondgoogle

  • ele.ph/jsforum

  • Tell me why I cant

    Clear your mind of

    questions

    *not the actual conversation

  • GSCfetch

    URLIndexed?

    ContentIndexed?

    TestURL

    External Ajax callInline

  • http://jsseo.expert/plain-external/test/

    http://jsseo.expert/react-external/test/

    http://jsseo.expert/jquery-external/test/

    we found this URL from a link ca 3 weeks ago, but we haven't crawled it

    we picked this one up ~19 days ago, and similar to above, our systems just didn't feel like crawling it yet

    picked up ~21 days ago, also not crawled yet.

  • Thanks, John!

    Youre the best,

    Bartosz!*

    *he didnt say that

    John Muellers feedback got me thinking

  • Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sitamet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuadafames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusceposuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus.Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet,consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna erosquis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proinpharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinarultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbitristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverraimperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummypede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectusmalesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus etnetus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitorcongue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusceest. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Loremipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit ametcommodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames acturpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere,magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus.Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet,consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna erosquis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proinpharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinarultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbitristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna.