Post on 20-Jul-2020
JPK
Gro
upBusiness Forecasting and Analytics Forum
September 19-20 • Chicago, IL
In-Depth Workshop:Digital Forecasting and Analytics
September 20, 1:15pm
Widely considered one of the leading digital measurement experts in the world, Garyleads EY’s Digital Analytics Practice. EY acquired Gary’s previous company –
Semphonic – in March of 2013. As Semphonic’s President and co-Founder, Gary ledSemphonic’s growth over a 15 year period from a 2-person practice to the one of the
leading digital analytics practices in the United States. Voted the most InfluentialIndustry Contributor by the Digital Analytics Association in 2012, Gary writes an
influential blog (http://semphonic.blogs.com/semangel), has published more thantwenty whitepapers on advanced digital analytics practice and is a frequent speaker
at industry events.
View presentation online at:https://jpkgroupsummits.com/attendee5
Gary Angel – Ernst & Young
Examine the role of exogenous variables & discusshow/whether to include them in a forecast
Page 1 Introduction to Digital Forecasting November 2015
Introduction to Digital
Forecasting
with Gary Angel
Page 2 Introduction to Digital Forecasting
Overview
Today’s Agenda:
Introduction Forecasting Techniques
Matching Technique to Problem
Digital Analytics Model
Historical Forecasting Basics
Advanced Elements
Predictive Models
Sample Conceptual Models
Summary
Page 3 Introduction to Digital Forecasting
But have you really thought about how it works?
You know what forecasting is…
Page 4 Introduction to Digital Forecasting
The three primary types of forecast
Opinion-Based
Time-Series
Predictive Models
Ask the Experts
Avaraging, Trending
and
Smoothing
Model the System
Page 5 Introduction to Digital Forecasting
With their common sub-methods
Opinion Based Methods
Market Research
Surveys
Expert Panels
Time Series
Averages & Moving Averages
Smoothing Methods
Models Regression
Econometric
Demand Signals
Simulation
Page 6 Introduction to Digital Forecasting
Opinion-Based Methods
Best when information is limited (new channels, markets, products)
Difficult to replicate and fairly low accuracy
Page 7 Introduction to Digital Forecasting
Time-Series Methods
Best when historical data points exist and are stable
Doesn’t capture levers of change
Page 8 Introduction to Digital Forecasting
Predictive Models
Provides causal insight & allows for what-if analysis
Harder to build and needs enough variation in historical data plus knowledge of exogenous factors
Page 9 Introduction to Digital Forecasting
Building a Digital Analytics Forecast
Page 10 Introduction to Digital Forecasting
Building a digital forecast
► There are countless problems in digital that might require
a forecast. For our workshop, we’re going to focus on just
one:
Forecasting Website Traffic
Page 11 Introduction to Digital Forecasting
Mathematical Techniques for Forecasting
► Outline
► Stability
► Moving Average
► Weighted Average
► Smoothing
► Break-outs
Page 12 Introduction to Digital Forecasting
Stability
► All forecasting is based on the assumption that the future
will resemble the past.
► The simplest forecast (which we use far more than we
ought) is that things will remain exactly the same:
Actual Visits in
Jan. 2016
1,763,226
Forecast Visits
Feb. 2016
1,763,226
Page 13 Introduction to Digital Forecasting
But stability from what/when?
► Even when stability is our forecast, it’s fair to ask
“Compared to what?”
► Here’s three different stable forecasts:
Actual Visits in
Jan. 2016
1,763,226
Forecast Visits
Feb. 2016
1,763,226
Actual Visits in
Feb. 2015
1,676,079
Forecast Visits
Feb. 2016
1,676,079
Visits per day in
Jan. 2016
1,676,079
Forecast Visits
Feb. 2016
1,592,591
Page 14 Introduction to Digital Forecasting
Moving Average
► But we know things don’t remain the same. There’s often
lots of noise and variation in any given measurement.
► So one common technique is to try and eliminate noise by
taking the average of our past data points.
Forecast Visits
Feb. 2016
Month Visits1/1/2015 1,919,431
2/1/2015 1,676,079
3/1/2015 1,755,637
4/1/2015 1,842,057
5/1/2015 1,790,122
6/1/2015 1,822,100
7/1/2015 1,756,784
8/1/2015 1,842,306
9/1/2015 1,744,129
10/1/2015 1,691,928
11/1/2015 1,570,476
12/1/2015 1,507,756
1/1/2016 1,763,226
1,744,772
Page 15 Introduction to Digital Forecasting
Moving Average – Can make for a slow learner
► When using an average, the number of periods
determines how quickly the measure detects changes
AND how much it is influenced by outliers:
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
Visit 12 month Average
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
Visit 12 month Average
3 Month Average
Page 16 Introduction to Digital Forecasting
Weighted Moving Average
► A weighted average is designed to provide stability AND
rapid tracking.
► Each period is given a different weight – usually biasing
the average toward more recent periods.
Forecast Visits
Feb. 2016
Month Visits1/1/2015 1,919,431
2/1/2015 1,676,079
3/1/2015 1,755,637
4/1/2015 1,842,057
5/1/2015 1,790,122
6/1/2015 1,822,100
7/1/2015 1,756,784
8/1/2015 1,842,306
9/1/2015 1,744,129
10/1/2015 1,691,928
11/1/2015 1,570,476
12/1/2015 1,507,756
1/1/2016 1,763,226
1,721,567
Page 17 Introduction to Digital Forecasting
Weighted average is designed to blend stability and sensitivity
► The stability and sensitivity of the forecast are functions of
the size of the window (# of periods used in the average)
and the weighting:
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
1/1
/20
15
2/1
/20
15
3/1
/20
15
4/1
/20
15
5/1
/20
15
6/1
/20
15
7/1
/20
15
8/1
/20
15
9/1
/20
15
10
/1/2
015
11
/1/2
015
12
/1/2
015
1/1
/20
16
Actual Weighted
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
1/1
/20
15
2/1
/20
15
3/1
/20
15
4/1
/20
15
5/1
/20
15
6/1
/20
15
7/1
/20
15
8/1
/20
15
9/1
/20
15
10
/1/2
015
11
/1/2
015
12
/1/2
015
1/1
/20
16
Actual Weighted
0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.1 0.1 0.1 0.3 0 0 0 0 0 0 0 0 0.1 0.2 0.3 0.4
Page 18 Introduction to Digital Forecasting
Exponential smoothing
► Exponential Smoothing applies a weight to each iteration,
constantly adjusting the forecast score against the
difference between the previous forecast/actual. The
weight determines the amount of adjustment:
Forecast Visits
Feb. 2016
1,671,7261,200,000
1,400,000
1,600,000
1,800,000
2,000,000
4/1
/20
14
6/1
/20
14
8/1
/20
14
10
/1/2
014
12
/1/2
014
2/1
/20
15
4/1
/20
15
6/1
/20
15
8/1
/20
15
10
/1/2
015
12
/1/2
015
Actual Exponential Forecast
1671726
.5 Weight to Actual and .5 Weight to Forecast
Page 19 Introduction to Digital Forecasting
Exponential Smoothing in Excel
► Excel includes a simple Exponential Smoothing function in
the Data Analysis Toolpack (an add-in):
Page 20 Introduction to Digital Forecasting
Double exponential smoothing (Holt)
► Double exponential smoothing adjusts both the predicted
value and the trend (weighting) with each new value.
► This makes it much better for matching trends than Single
Exponential Smoothing.
Forecast Visits
Feb. 2016
1,790,276 1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
1/1
/20
15
2/1
/20
15
3/1
/20
15
4/1
/20
15
5/1
/20
15
6/1
/20
15
7/1
/20
15
8/1
/20
15
9/1
/20
15
10
/1/2
015
11
/1/2
015
12
/1/2
015
1/1
/20
16
Visit Double Exp Forecast
Alpha 0.9
Beta 0.2
Page 21 Introduction to Digital Forecasting
Double Exponential Smoothing in Excel
► Here’s the process for double exponential smoothing in
Excel:
1 Set your alpha (exponential smoothing value) and beta (trend value) – between 0-1
2You will create three new values: Exponential Forecast, Trend Forecast, and DESmoothed Forecast (this last one is the real forecast)
3 DESmoothed Forecast is always equal to the Exponential Forecast + Trend Forecast
4For the 1st Period, the Exponential Forecast is equal to the actual value for that period and the Trend Forecast is zero.
5For every other period, the Exponential Forecast is equal to the Previous Exponential Forecast plus the alpha value times the difference between the Previous Actual and the Previous DESmoothed Forecast.
6For every other period, the Trend Forecast is equal to the Previous Trend Forecast plus the beta value times the difference between the current Exponential Forecast and the Previous DESmoothed Forecast.
Page 22 Introduction to Digital Forecasting
Double Exponential Smoothing in Excel
► It looks like this:1
2
3
3
4
5
5
Page 23 Introduction to Digital Forecasting
Double Exponential Smoothing in Excel
► It looks like this:6
6
Page 24 Introduction to Digital Forecasting
Triple exponential smoothing (Holt-Winters)
► Triple exponential smoothing adds a weight for a seasonal
cycle. The predicted value and trend value are updated
identically to the Holt method except that they are first
adjusted seasonally. The seasonal parameter can be
tuned or pre-determined.
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
Actual Forecast
► It should only be used
when your data has a
significant seasonal
component.
Page 25 Introduction to Digital Forecasting
About those pesky parameters
► You can use tools (like Excel’s Solver) to find the best
values for the parameters.
► You do this by calculating the MSE (average of the
forecast errors after squaring) and then optimizing the
parms to that value.
Page 26 Introduction to Digital Forecasting
Break-outs and Banding
► Break-outs and banding aren’t a separate forecasting
technique – they are tools for understanding whether a
movement is interesting.
► All processes have a certain amount of variation. Banding
is used to draw a band of fairly natural variation around
the trend. When an actual “breaks” the band, the variation
is usually significant.
1,200,000
1,400,000
1,600,000
1,800,000
2,000,000
2/1
/20
15
3/1
/20
15
4/1
/20
15
5/1
/20
15
6/1
/20
15
7/1
/20
15
8/1
/20
15
9/1
/20
15
10
/1/2
015
11
/1/2
015
12
/1/2
015
1/1
/20
16
Actual Upper Band Lower Band
Page 27 Introduction to Digital Forecasting
Whew…Let’s take a break and then tackle Modeling
Page 28 Introduction to Digital Forecasting
Model-based Forecasting
► Outline
► Time-Series vs. Models
► Identifying key variables
► Source, Season
► Building a Conceptual Model
► Sample Conceptual Models
► Quick Discussion of other Models
Page 29 Introduction to Digital Forecasting
Time-Series vs. Model
Forecast Model
Page 30 Introduction to Digital Forecasting
Building a Model
► The first (and maybe most important) step in building a
model is deciding what variables you might use.
► Keep in mind that there is no one right answer. The level
of variables you use needs to match the operational level
you want to understand.
► For example, if you’re trying to optimize channel
marketing decisions, it doesn’t work to use Total Marketing
Spend as your marketing variable.
Page 31 Introduction to Digital Forecasting
Throwing variables at a wall
► Don’t just toss variables into a modelling blender.
Page 32 Introduction to Digital Forecasting
Building a conceptual model
Website Traffic = Visits from last month * Repeat Visit Rate( ) + Avg. New Visits
Website Traffic = Search Visit per Dollar * Exp. Search Spend( ) +
Display Visit per Dollar * Exp. Display Spend( ) +
Avg. Direct Visits
Website Traffic = Current High Frequency Customers * Avg. Visit Propensity( ) +
Avg. New Visits
Current Med. Frequency Customers * Avg. Visit Propensity( ) +
Current Low Frequency Customers * Avg. Visit Propensity( ) +
Page 33 Introduction to Digital Forecasting
SEO
Page Ranks
Keyword Volumes
Outcomes
Open Sessions
Repeat Rates
Satisfaction
Exogenous
Econometrics
Brand Awareness
Web Growth
Device Shifts
Seasonality
Sample systems and variables
Marketing
Total Spend
Digital Spend
Channel Spend
Mix
User-Base
Active Users
Users by Cohort
New Users Last Period
Segmentation
User Types
Visit Types
Page 34 Introduction to Digital Forecasting
Deepening a conceptual model
Website Traffic = Visits sourced by marketing +
Visits sourced by our user-base +
Visits sourced by Web “Flow”
Visits sourced by marketing = Total Marketing Spend * constant
Visits sourced by marketing = Digital Marketing Spend * constant
Mass Marketing Spend * constant
+( )
( )
Visits sourced by marketing = PPC Marketing Spend * constant
Mass Marketing Spend * constant
+( )
( )
Display Marketing Spend * constant +( )
Page 35 Introduction to Digital Forecasting
Deepening a conceptual model II
Visits sourced by marketing = PPC Marketing Spend * constant
Mass Marketing Spend * constant
+( )
( )
Display Marketing Spend * constant +( )
► Implies that the impact of increasing PPC marketing
spend will be constant. That’s rarely the case for any
variable except over a narrow band.
► We could further break out PPC Marketing spend into
categories (like brand, non-brand) but eventually we’ll run
into the same problem.
Page 36 Introduction to Digital Forecasting
Modeling change in a variable
PPC Marketing Visits Sourced = PPC Marketing Spend * function(saturation)( )
► Media buying typically
decays both in terms of
impact per incremental
buy after a saturation
point and in terms of
time. 0 10 20 30 40 50 60 70 80 90 100
Constant
PPC Marketing Visits Sourced = PPC Marketing Spend * function(saturation)( ) +
Prior PPC Marketing Spend * function(saturation) * function(timedecay)( )
Page 37 Introduction to Digital Forecasting
Paid Marketing: Outline Model
► Break-downs by Channel
► Inside Channel by Ad Group / Campaign
► National vs. Local
► Web / Mobile
► Make sure (at minimum) that brand and non-brand are separated
► Build decay model to capture time lag of spending
► Build saturation model to capture reduced incrementality of spend
by channel
► Think about integrating a full mix/attribution model
► Consider treating as a separate customer type or sub-type within
each customer segment
Page 38 Introduction to Digital Forecasting
SEO: Outline Model
► Break-out Brand vs. Non-Brand
► Break out Long-Tail as a single entity
► For short-tail
► Track positional impact for major keywords
► Consider treating as a separate customer type or sub-type
within each customer segment
Page 39 Introduction to Digital Forecasting
Mass Media: Outline Model
► Spend at highest level of temporal granularity possible
(Monthly / Weekly / Daily)
► Channel and GRPs (Planned or Delivered)
► National / Local
► Think about integrating a full mix model
► Consider use of pre-qualification survey to evaluate mass-
media source quality
Page 40 Introduction to Digital Forecasting
Repeat Visitors: Return Model
► Two (potentially complementary) approaches:
► Customer-based by expected return frequency
► Visit-based by previous visit type
Advisors
Plan Managers
High-Wealth Investors
General Investors
1.8
1.5
1.3
1.2
1.3
1.2
1.3
1.1
1.6
1.4
1.8
1.3
1.9
1.4
1.6
1.3
Research SupportSpecific
FundGeneral
Fund
Page 41 Introduction to Digital Forecasting
Oh the places you’ll go
► Outline – Other kinds of digital models
► Conversion Rate
► Site Alerting (Problem Identification)
► Marketing Spend
► Content Performance
► Brand impact
Page 42 Introduction to Digital Forecasting
Take another deep breath and we’ll wrap up
Page 43 Introduction to Digital Forecasting
When do we use what
► Factors in selecting an approach:
► Do you have historical data?
► Is the situation inherently unpredictable or chaotic?
► Do you need to understand WHY a system changed or WHAT you
can adjust? (you don’t always)
► How much data do you actually have?
► How much work are you willing to do?
Page 44 Introduction to Digital Forecasting
Thank you
► gary.angel@ey.com
► +1 415 894 8255
► LinkedIn:
www.linkedin.com/pub/gary-
angel/0/176/a43/
Gary Angel