Methods, Diagnostics, and Practices for Seasonal Adjustment Catherine C. H. Hood Introductory...
-
Upload
wyatt-lyon -
Category
Documents
-
view
219 -
download
0
Transcript of Methods, Diagnostics, and Practices for Seasonal Adjustment Catherine C. H. Hood Introductory...
Methods, Diagnostics, andPractices for Seasonal
Adjustment
Catherine C. H. Hood Introductory Overview Lecture: Seasonal Adjustment
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
2© 2007 Catherine C.H. Hood
Acknowledgements
Many thanks to ● David Findley, Brian Monsell, Kathy
McDonald-Johnson, Roxanne Feldpausch● Agustín Maravall
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
3© 2007 Catherine C.H. Hood
Outline
Basic concepts Software packages for seasonal
adjustment production● Mechanics of X-12 and SEATS
Overview of current practices Recent developments in research areas
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
4© 2007 Catherine C.H. Hood
Time Series
A time series is a set of observations ordered in time● Usually most helpful if collected at regular
intervals
In other words, a sequence of repeated measurements of the same concept over regular, consecutive time intervals
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
5© 2007 Catherine C.H. Hood
Time Series Data Occurs in many areas: economics, finance,
environment, medicine Methods for time series are older than those
for general stochastic processes and Markov Chains
The aims of time series analysis are to describe and summarize time series data, fit models, and make forecasts
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
6© 2007 Catherine C.H. Hood
Why are time series data different from other data?
Data are not independent● Much of the statistical theory relies on the data
being independent and identically distributed
Large samples sizes are good, but long time series are not always the best● Series often change with time, so bigger isn’t
always better
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
7© 2007 Catherine C.H. Hood
What Are Our Users Looking for in an Economic Time Series? Important features of economic
indicator series include● Direction● Turning points
□ In addition, we want to see if the series is increasing/decreasing more slowly than it was before
● Consistency between indicators
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
8© 2007 Catherine C.H. Hood
Why Do Users Want Seasonally Adjusted Data?
Seasonal movements can make features difficult or impossible to see
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
9© 2007 Catherine C.H. Hood
Classical Decomposition One method of describing a time series Decompose the series into various components
● Trend – long term movements in the level of the series● Seasonal effects – cyclical fluctuations reasonably
stable in terms of annual timing (including moving holidays and working day effects)
● Cycles – cyclical fluctuations longer than a year● Irregular – other random or short-term unpredictable
fluctuations
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
10© 2007 Catherine C.H. Hood
Causes of Seasonal Effects
Possible causes are● Natural factors● Administrative or legal measures● Social/cultural/religious traditions
(e.g., fixed holidays, timing of vacations)
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
11© 2007 Catherine C.H. Hood
Causes of Irregular Effects
Possible causes● Unseasonable weather/natural disasters● Strikes● Sampling error● Nonsampling error
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
12© 2007 Catherine C.H. Hood
Other Effects
Trading Day: The number of working or trading days in a period
Moving Holidays: Events which occur at regular intervals but not at exactly the same time each year
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
13© 2007 Catherine C.H. Hood
May 2007
S M T W T F S
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
14© 2007 Catherine C.H. Hood
June 2007
S M T W T F S
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
15© 2007 Catherine C.H. Hood
Moving Holiday Effects
Holidays not at exactly the same time each year ● Easter● Labor Day● Thanksgiving
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
16© 2007 Catherine C.H. Hood
“Combined” Effects
Trading day and moving holiday effects are both persistent, predictable, calendar-related effects, so trading day and holiday effects often included with the seasonal effects
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
22© 2007 Catherine C.H. Hood
The Simple Case
The time series would have ● No growth or decline from year to year,
only rather repetitive within-year movements about an unchanging level
● No trading day or moving holidays
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
23© 2007 Catherine C.H. Hood
Change in Variations
What if the magnitude of seasonal fluctuations is proportional to level of series?● take logarithms
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
26© 2007 Catherine C.H. Hood
Log Transformations
Appropriate when the variability in a series increases as its level increases, and when all values of the series are positive
Change multiplicative relationships into additive relationships
Increases/decreases can be thought of in terms of percentages
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
27© 2007 Catherine C.H. Hood
Problem: Extreme Values
Solution:● These effects can be estimated also, but
they can be difficult to estimate when seasonality and trend are present
Which of these values are outliers (extreme values)?
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
31© 2007 Catherine C.H. Hood
Trading Day and Other Effects What if trading day and/or other effects
(holiday, outliers) are present?● X-11: TD, holiday regression on the irregular
component, extreme value modifications● SEATS: RegARIMA models for a regression on
the original series ● X-12: Use X-11 methods or RegARIMA models
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
32© 2007 Catherine C.H. Hood
Models
Multiplicative model:
Yt = St´ × Tt × It
= St´ × Nt
where
St´ = St × TDt × Ht
Additive model:
Yt = St´ + Tt + It
= St´ + Nt
where
St´ = St + TDt + Ht
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
33© 2007 Catherine C.H. Hood
Objectives
Estimate Nt (remove effects of St ) for seasonal adjustment
Estimate Tt (remove effects of St and It) for trend estimation
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
34© 2007 Catherine C.H. Hood
How Do We Estimate the Components?
Seasonal adjustment is normally done with off-the-shelf programs such as:● X-11 or X-12-ARIMA (Census Bureau), ● X-11-ARIMA (Statistics Canada),● Decomp, SABL, STAMP, ● TRAMO/SEATS (Bank of Spain)
Seasonal Adjustment
RegARIMA Models(Forecasts, Backcasts, and Preadjustments)
Modeling and Model Comparison Diagnostics and Graphs
Seasonal Adjustment Diagnostics and Graphs
RegARIMA Model
log = ´ Xt + Zt
transformations ARIMA process
Xt = Regressor for trading day and holiday or calendar effects, additive outliers,
temporarychanges, level shifts, ramps, and user-defined effects
Dt = Leap-year adjustment, or “subjective” prior adjustment
( )Yt
Dt
Catherine Hood Consulting
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
37© 2007 Catherine C.H. Hood
ARIMA Models and Forecasting
If we can describe the way the points in the series are related to each other (the autocorrelations), then we can describe the series using the relationships that we’ve found
AutoRegressive Integrated Moving Average Models (ARIMA) are mathematical models of the autocorrelation in a time series
One way to describe time series
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
38© 2007 Catherine C.H. Hood
Autocorrelation The major statistical tool for ARIMA
models is the sample autocorrelation coefficient
rk =
( Yt – Y )2
t=k+1
n
t=1
n
__ __
__
( Yt-k – Y )( Yt – Y )
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
39© 2007 Catherine C.H. Hood
Autocorrelations
r1 indicates how successive values of Y relate to each other,
r2 indicates how Y values two periods apart relate to each other,
and so on.
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
40© 2007 Catherine C.H. Hood
ACF
Together, the autocorrelations at lags 1, 2, 3, etc. make up the autocorrelation function or ACF and then we plot the autocorrelations by the lags
The ACF values reflect how strongly the series is related to its past values over time
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
41© 2007 Catherine C.H. Hood
Autoregressive Processes The autoregressive process of order p is
denoted AR(p), and defined by
Zt = r Zt-r + wt
where 1 , . . . , p are fixed constants and {wt} white noise, a sequence of independent (or uncorrelated) random variables with mean 0 and variance 2
r=1
p
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
42© 2007 Catherine C.H. Hood
Moving Average Processes
The moving average process of order q, denoted MA(q), includes lagged error terms t–1 to t–q, written as
Zt = wt – r wt-r
where 1 , 2 , … , q are the MA parameters and wt is white noise
r=1
q
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
43© 2007 Catherine C.H. Hood
Random Walk Constrained AR Model
Zt = Zt-1 + wt with 1 = 1 First differenced model
Zt = Zt-1 + wt Zt – Zt-1 = wt (1 – B) Zt = wt
Seasonal difference modelZt – Zt-12 = wt
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
44© 2007 Catherine C.H. Hood
ARMA processes
The autoregressive moving average process, ARMA(p,q) is defined by
Zt – r Zt–r = r wt–r
where again wt is white noise
qp
r=1 r=0
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
45© 2007 Catherine C.H. Hood
Seasonal Processes A seasonal AR process
Zt = r Zt-Sr + wt
A seasonal MA process
Zt = wt – Θr wt-r
where 1 , . . . , P and Θ1 , … , ΘQ are fixed constants, {wt} is white noise, and S is the frequency of the series (12 for monthly or 4 for quarterly)
p
RegARIMA Model
log = ´ Xt + Zt
transformations ARIMA process
Xt = Regressor for trading day and holiday or calendar effects, additive outliers,
temporarychanges, level shifts, ramps, and user-defined effects
Dt = Leap-year adjustment, or “subjective” prior adjustment
( )Yt
Dt
Catherine Hood Consulting
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
47© 2007 Catherine C.H. Hood
RegARIMA Model Uses Extend the series with forecasts (or possibly
backcasts) Detect and adjust for outliers to improve the
forecasts and seasonal adjustments Estimate missing data Detect and directly estimate trading day
effects and other effects (e.g. moving holiday effects, user-defined effects)
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
48© 2007 Catherine C.H. Hood
Automatic Procedures
Both X-12-ARIMA and SEATS have procedures for the automatic identification of● ARIMA model● Outliers● Trading Day effects● Easter effects
Seasonal Adjustment
RegARIMA Models(Forecasts, Backcasts, and Preadjustments)
Modeling and Model Comparison Diagnostics and Graphs
Seasonal Adjustment Diagnostics and Graphs
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
50© 2007 Catherine C.H. Hood
How are component estimates formed?
X-11, X-12: limited set of fixed filters ARIMA Model-based (AMB):
● Fit ARIMA model to series● This model, plus assumptions, determine
component models● Signal extraction to produce component
estimates and mean squared errors (MSE)
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
51© 2007 Catherine C.H. Hood
Example Trend Filter from X-12-ARIMA
A centered 12-term moving average
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
52© 2007 Catherine C.H. Hood
Example: 3x3 Filters
3 x 3 filter for Qtr 1, 1990 (or Jan 1990)
1988.1 + 1989.1 + 1990.1 + 1989.1 + 1990.1 + 1991.1 + 1990.1 + 1991.1 + 1992.1 9
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
53© 2007 Catherine C.H. Hood
Example Seasonal Filter from X-12-ARIMA: 3x3 Filter
Recall that Y = TSI, so SI = Y/T, i.e., the detrended series
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
54© 2007 Catherine C.H. Hood
AMB Approach
Fit RegARIMA model yt = x´t + Zt
Given an ARIMA model for series Zt,
(B) (B) Zt = Θ (B) (B) wt
and the model Yt = St + Nt , determine models for components St and Nt
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
55© 2007 Catherine C.H. Hood
Where . . . St independent of Tt independent of It
( St independent of Nt ) St , Tt , It follow ARIMA models
consistent with the model for Zt
(hence so does Nt) It is white noise (or low order MA)
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
56© 2007 Catherine C.H. Hood
Canonical Decomposition
Problem: There is more than one admissible decomposition
Solution: Use the canonical decomposition, the decomposition that corresponds to minimizing the white noise in the seasonal component
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
57© 2007 Catherine C.H. Hood
Properties of the Canonical Decomposition
Unique (and usually exists) Minimizes innovation variances of
seasonal and trend; maximizes irregular variance
Forecasts of St follow a fixed seasonal pattern
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
58© 2007 Catherine C.H. Hood
Advantages of AMB Seasonal Adjustment Flexible approach with a wide range of
models and parameter values Model selection can be guided by
accepted statistical principals Filters are tailored to individual series
through parameter estimation, and are “optimal” given
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
59© 2007 Catherine C.H. Hood
Advantages of AMB Seasonal Adjustment (2) Signal extraction calculations provide
error variances of component estimates with MSE based on the model● Approach easily extends (in principle) to
accommodate a sampling error component
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
60© 2007 Catherine C.H. Hood
At the End of the Series
X-11: asymmetric filters (from ad-hoc modifications to symmetric filters)
X-11-ARIMA, X-12: one year (optionally longer) forecast extension
AMB: full forecast extension
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
61© 2007 Catherine C.H. Hood
Issues Relating to Current Practices
X-12 versus SEATS Use of RegARIMA models, for
forecasting, trading day, holidays, etc. Diagnostics
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
62© 2007 Catherine C.H. Hood
Agreement in Current Practices
Compute the concurrent factors (running the seasonal adjustment software every month with the most recent data) instead of projected factors
Use regARIMA models whenever possible (ARIMA models required for SEATS)
Continue to publish the original series along with the seasonal adjustment
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
63© 2007 Catherine C.H. Hood
X-12 vs SEATS Eurostat recommends use of either program US Census Bureau recommends use of X-
12-ARIMA● According to research, X-12 is more accurate
than SEATS for most series● X-12 works better for short series (4 to 7 years)
and for longer series (over 15 years)● X-12 has better diagnostics
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
64© 2007 Catherine C.H. Hood
Setting Options To reduce revisions, best to set certain
options for production● Most agencies let the software choose the
options and then fix the settings for production Problems come with SEATS because model
used is not always the model specified, and model coefficients also are not always the ones specified
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
65© 2007 Catherine C.H. Hood
Trading Day and Moving Holiday Settings In Europe, there has been a lot of work on “user-
defined” variables that include trading days and moving holidays to incorporate country-specific holidays
Most agencies in the U.S. use built-in trading day and built-in moving holidays from X-12-ARIMA● Unfortunately, not all the built-in variables are useful
for every situation● Some agencies avoid trading day altogether
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
66© 2007 Catherine C.H. Hood
Outlier Settings At the Australia Bureau of Statistics, they have a
very rigorous procedure of outlier identification, including meta data on certain unusual events
Most other agencies use the automatic outlier selection procedure
At the U.S. Census Bureau● Choose new outliers with every run● At annual review time, set outliers for current data and
set a high critical value for the new data coming in
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
67© 2007 Catherine C.H. Hood
Direct/Indirect Definitions
If a time series is a sum (or other composite) of component series● Direct adjustment – a seasonal adjustment of
the aggregate series obtained by seasonally adjusting the sum of the component series
● Indirect adjustment – a seasonal adjustment of the aggregate series obtained from the sum of the seasonally adjusted component series
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
68© 2007 Catherine C.H. Hood
Example – Direct and Indirect Adjustment
US = NE + MW + SO + WE
Indirect seasonal adjustment of US:
SA(NE) + SA(MW) + SA(SO) + SA(WE) Direct seasonal adjustment of US:
SA( NE + MW + SO + WE )
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
69© 2007 Catherine C.H. Hood
Comment on Yearly Totals
When do yearly totals of the original series and the seasonally adjusted series coincide?
When the series has● An additive decomposition● A seasonal pattern that is fixed from one year to
the next ● No trading adjustments
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
70© 2007 Catherine C.H. Hood
Areas for Improvement in Current Practices
Concurrent adjustment Use of regARIMA models
● Moving holidays and other user-defined effects Setting options (to reduce revisions) and
checking the options regularly Software to make it easier to check
diagnostics regularly● Training in ARIMA modeling and diagnostics
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
71© 2007 Catherine C.H. Hood
Recent Developments and Research Areas
X-13 (X-13-SEATS) Improved and new diagnostics (for both
X-12 and SEATS) New filters for X-12 and new, more
flexible models for SEATS Supplemental and utility software Documentation and training
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
72© 2007 Catherine C.H. Hood
Newest X-12
Version 0.3 includes a new automatic ARIMA-modeling procedure based on the program TRAMO from the Bank of Spain
The next release (X-13) will include ARIMA-model-based seasonal adjustment options
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
73© 2007 Catherine C.H. Hood
Model-based Adjustment
SEATS, developed by Agustín Maravall at the Bank of Spain
REGCMPT, developed by Bill Bell at the Census Bureau
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
74© 2007 Catherine C.H. Hood
SEATS
Disadvantages● No diagnostics for the adjustment● No methods for series with different
variability in different months● No user-defined regressors● Not very flexible ARIMA models
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
75© 2007 Catherine C.H. Hood
REGCMPT
Advantages● Methods for different variability in
different months● Can build very flexible regARIMA
models Still being tested
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
76© 2007 Catherine C.H. Hood
X-13-SEATS
Advantages● Would combine the model-based
adjustments from SEATS with diagnostics from X-12, and keep the ability to use X-11-type adjustments also
Disadvantage● ????
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
77© 2007 Catherine C.H. Hood
Running in Windows
TRAMO/SEATS for Windows Windows Interface to X-12-ARIMA
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
78© 2007 Catherine C.H. Hood
Supplemental Software X-12-Graph in SAS and in R X-12-Data and X-12-Rvw Programs to help write user-defined
variables for custom trading day and moving holidays
Excel interfaces to run SEATS and X-12 from Excel● Interfaces to other software are available
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
79© 2007 Catherine C.H. Hood
Documentation and Training Documentation
● “Getting Started” papers to use with the Windows version, written for novice users
● Documentation on commonly used options for both X-12 and SEATS
Training● Advanced Diagnostics● RegARIMA Modeling
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
80© 2007 Catherine C.H. Hood
Resources X-12-ARIMA website
www.census.gov/srd/www/x12a Seasonal adjustment papers pages TRAMO/SEATS website
www.bde.es/english/ Papers and course information
www.catherinechhood.net
Methods, Diagnostics, and Practices for Seasonal Adjustment---June 2007
Catherine Hood Consulting
81© 2007 Catherine C.H. Hood
Contact Information
Catherine HoodCatherine Hood Consulting1090 Kennedy Creek RoadAuburntown, TN 37016-9614
Telephone: (615) 408-5021 Email: [email protected]
Web: www.catherinechhood.net