TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness
Transcript of TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness
![Page 1: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/1.jpg)
Paul Shapiro | @fighto #TechSEOBoost
SEO Automation to Increase Efficiency & Effectiveness
Working Smarter
![Page 2: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/2.jpg)
Paul Shapiro | @fighto #TechSEOBoost
![Page 3: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/3.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Why Automate?
1.Work faster and free up time for the important stuff
2.Look at more data
3.Improved consistency and procedure
![Page 4: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/4.jpg)
Paul Shapiro | @fighto #TechSEOBoost
What Can You Automate?
• If you’re doing something on a routine basis, it can probably be automated.
• Any procedures that can be broken down into smaller, micro tasks that can be handled a by a computer.• Machine Learning can help with more
complex decision making.
![Page 5: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/5.jpg)
Paul Shapiro | @fighto #TechSEOBoost
How Do You Automate?
![Page 6: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/6.jpg)
Paul Shapiro | @fighto #TechSEOBoost
![Page 8: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/8.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Why KNIME?
• Fast way to put together complex analyses
• Great for prototyping
• Large library of built-in “nodes”
• Free/Open Source
• Run on Windows/Mac/Linux
• Very expandable – even compatible with R, Python, Java, JavaScript
• Easy enough for non-technical staff to grasp
![Page 9: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/9.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Other Options
• Scripting Languages• Python
• Ruby
• Node.js
• Go
• R
• Excel with VBA
• Google Sheets
![Page 10: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/10.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Cron & Windows Task Manager
are Your Friend
![Page 11: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/11.jpg)
Paul Shapiro | @fighto #TechSEOBoost
What is Cron and Why?
• *NIX system daemon used to schedule tasks and scripts.
• Windows Task Manager is the Windows equivalent of Cron.
• This way we can schedule scripts and programs that perform automated tasks on a recurring, scheduled basis.
![Page 12: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/12.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Quick How To
* * * * * command /to/executeDay of Week (0-6) (Sunday = 0)
Month (1-12)
Hour (0-23)
Day of Month (1-31)
Minute (0-59)
![Page 13: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/13.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Run Every Month at Midnight
0 0 1 * * python datacollector.py
![Page 14: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/14.jpg)
Paul Shapiro | @fighto #TechSEOBoost
The Basics of KNIME
![Page 15: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/15.jpg)
Paul Shapiro | @fighto #TechSEOBoost
What is a Node?
• Nodes are prebuilt, drag and drop modules designed perform a singular task
• Nodes are strung together like a chain to accomplish larger, more complex tasks
• Nodes can be grouped together into “meta-nodes”, which can be configured in unison
![Page 16: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/16.jpg)
Paul Shapiro | @fighto #TechSEOBoost
How Do You Add Nodes &
How Do They Connect?
How do you add nodes to your “workflow”?
How do you string nodes together?
![Page 17: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/17.jpg)
Paul Shapiro | @fighto #TechSEOBoost
How Do You Configure & Run Nodes?
Configuring Nodes
Running Workflows
OR
![Page 18: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/18.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Keyword Research
![Page 19: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/19.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Most Keyword Research Looks Like This
![Page 20: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/20.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Typical Time Investment for Keyword
Research
5
8
12
21 21
6
10
18
22
28
0
5
10
15
20
25
30
Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages)
Hours to Complete Keyword Research by Site Size
Average (Low End) Average (High End)
![Page 21: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/21.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Size of the Data Set
vs.
![Page 22: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/22.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Filtering +
Data
Manipula-
tions
One Big Keyword List
Seed Keywords
- List
- GWMT
- SEMRush Comp. KWs
- SQR Keywords
Keyword Planner Suggestions (via
GrepWords)Google Autocomplete
Semantic Keyword Recommendations (via MarketMuse)
Google Autocomplete
SEMRush Domain vs. Domain
KeywordsGoogle Autocomplete
![Page 23: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/23.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Data Manipulations / Calculations
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic Competition
Search Volume
Keyword Trends
23
![Page 24: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/24.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Data Manipulations / Calculations
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic Competition
• Get Search Volumes via SEMRush API or via GrepWords APISearch Volume
Keyword Trends
24
![Page 25: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/25.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Data Manipulations / Calculations
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic Competition
• Get Search Volumes via SEMRush API or via GrepWords APISearch Volume
• Use 2 Years of Google Trends data to calculate slope and determine growing/declining keywordsKeyword Trends
25
![Page 26: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/26.jpg)
Paul Shapiro | @fighto #TechSEOBoost
String ‘em All Together and then…
26
![Page 27: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/27.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Visualize
27
This top-right quadrant contains
keywords with:
• Low competition
• Good growth
Larger bubbles show higher
search volumes.
You can alternatively use
current rank on the x-axis to
signal organic market share like
a traditional growth-share
matrix.
![Page 28: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/28.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Option: Tableau
28
![Page 29: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/29.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Option: Python + Bokeh Charts in KNIME
29
![Page 30: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/30.jpg)
Paul Shapiro | @fighto #TechSEOBoost 30
Sample Python Code:
http://pshapi.ro/SLBokeh
![Page 31: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/31.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Learn More…
SearchLove Presentation about Automating Keyword Research:
http://pshapi.ro/2idcqYF
BrightonSEO Presentation about Semantic Keyword Research:
https://searchwilderness.com/semantic-keyword-research/
![Page 32: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/32.jpg)
Paul Shapiro | @fighto #TechSEOBoost
What to Automate for SEO?
![Page 33: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/33.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Reporting
![Page 34: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/34.jpg)
Paul Shapiro | @fighto #TechSEOBoost
![Page 35: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/35.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Data Collection
![Page 36: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/36.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Search Console
Schedule to run monthly with Cron and backup to SQL database:
https://searchwilderness.com/gwmt-data-python/
JR’s BigQuery vision:
http://pshapi.ro/2vmjDe8
![Page 37: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/37.jpg)
Paul Shapiro | @fighto #TechSEOBoost
301 Redirect Mapping
from Old URLs
![Page 38: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/38.jpg)
Paul Shapiro | @fighto #TechSEOBoost
301 Redirect Mapping from Old URLs
Crawl Current Site
Download Rendered
Pages
Extract Main Content
(BoilerPipe)
Convert to Bitvector
Get Historic URLs from Wayback Machine
API
Filter Out URLs found on Current
Site
Grab Rendered Page from Wayback Machine
Extract Main
Content (BoilerPipe)
Convert to Bitvector
Patrick Stox
https://searchengineland.com/fixing-historical-redirects-using-wayback-machine-apis-257628
Cosine
Similarity
Generate .htaccess
strings
1
2
3
![Page 39: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/39.jpg)
Paul Shapiro | @fighto #TechSEOBoost
KNIME Toolkit
![Page 40: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/40.jpg)
Paul Shapiro | @fighto #TechSEOBoost
SERP Similarity / Overlap
![Page 41: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/41.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Riyaad Edoo
1. Download ranking data via STAT API
2. Compare results from 1-10 for each query against results from 1-10 for every other query.
3. Calculate percent similarity.
4. Schedule checks and examine what changed.
![Page 42: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/42.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Test JavaScript Rendering
![Page 43: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/43.jpg)
Paul Shapiro | @fighto #TechSEOBoost
http://pshapi.ro/puppetcrawl
![Page 44: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/44.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Performance Testing with Lighthouse
![Page 45: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/45.jpg)
Paul Shapiro | @fighto #TechSEOBoost
http://pshapi.ro/perfpony
![Page 46: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/46.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Anomaly Detection
![Page 47: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/47.jpg)
Paul Shapiro | @fighto #TechSEOBoost
CTR
1. Data collection: We collect data on query, page and associated metrics via the Google Search Console Search Analytics API.
2. Round average position: I round average position to the tenths decimal place (e.g., 1.19 is rounded to 1.2).
3. Math: We identify outliers using a combination of the statistical methods for identifying outliers (modified z-score, IQR).
4. Email: If any negative outliers are identified for a keyword query and page combination at an average position, an email is sent out identifying all of this data to each of the SEOs assigned to the account to investigate.
5. Scheduling: Set your script to run on a recurring basis.
My SEL Article: http://pshapi.ro/2Ae2LYP
![Page 48: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/48.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Content Ideas
![Page 49: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/49.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Reddit Data Mining
Reddit Data Mining: Python Script
https://searchwilderness.com/reddit-python-code/
1. Enter filename for output
2. Enter a search or series of searches
3. Choose reddit sorting method. For this purpose,
choose ‘new’
4. Choose to look at all of reddit, or isolate to
particular subreddit(s).
5. Schedule with cron to find new topic ideas on a
recurring basis.
Password: fighto
![Page 50: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/50.jpg)
Paul Shapiro | @fighto #TechSEOBoost
AMP Validation
![Page 51: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/51.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Bulk Check AMP
Pages with
AMPBench API
Python Script:
http://pshapi.ro/2AHlNaE
Requires:
• Python
• Requests package
Ideally AMPBench would run locally, but can be ran off the appspot demo URL.
![Page 52: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/52.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Link Building
![Page 53: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/53.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Link Building: Prospecting with Competitors
![Page 54: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/54.jpg)
Paul Shapiro | @fighto #TechSEOBoost
Tech Audit Related Site Changes
![Page 55: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/55.jpg)
Paul Shapiro | @fighto #TechSEOBoost
• New 404s
• New redirects
• Changes to robots.txt
• Content based changes over time
• Indexation changes
• New pages
![Page 56: TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & Effectiveness](https://reader033.fdocuments.net/reader033/viewer/2022050613/5a650cab7f8b9aa2548b6497/html5/thumbnails/56.jpg)
Paul Shapiro | @fighto #TechSEOBoost
CatalystDigital.com
Paul Shapiro
https://searchwilderness.com
@fighto
Thanks!