J. Khélif Insee July 2008 A quality report for seasonally and trading day adjusted French IIP.

J. KhélifInsee

July 2008

A quality report for seasonally and trading day adjusted French IIP

Quality report on SAJ. Khélif July 2008

Scheme of the presentation

1. General background

2. Building a quality report for IIP

3. Is the tool useful ? How can it be improved ?


Background

When analyzing short term economical indicators, users comment on:

– seasonally adjusted (SA) and/or trading day (TD) adjusted series;

– these series are expected to reflect recent economical evolutions;

– producing good quality SA-TD adjusted series is an important issue.


Background

› Different problems have to be faced by the producer:1. Criteria: there are no universal criteria for “good quality” SA-TD

adjusted series;

2. Timeliness : time for analysis is very short, poor quality problems have to quickly be identified;

3. Know-how : Producers are not necessarily SA experts.

› Our aim: build a tool that helps us judge and improve the quality of our SA-TD adjusted series

– The tool had to be easy and quick to run;– The tool had to propose synthetic analysis tables;– The tool had to be based on widely accepted quality criteria.


2. Developing a tool to evaluate the quality of SA-TD series

› Select criteria among many:– There is no unique way of SA series (unobserved

components);– Each SA adjustment method comes with a series of widely

accepted tests;– We made a selection according to our own local

preoccupations.

› Synthesize the information :– our aim: check for quality with a top-down approach;– Find a way of judging the quality of aggregates;– Target those series that are “really” problematic.

› Our solution : – give grades to elementary series based on the technical

criteria chosen; – aggregate them according to their weight in the economy.


2. Developing a tool to evaluate the quality of SA-TD series

› We decided on controlling 3 main elements:

1. The quality of pre-adjustments and the Reg-Arima estimation of the raw data;

2. Idempotency : SA and TD adjusted series should not show seasonal and/or trading day patterns.

› Our notation system :

1. Quality of pre-adjustments and Reg-Arima modeling

Maximum mark : 20

2. Idempotency Maximum mark : 20


2.1 Measuring the quality of pre-adjustments and Reg-Arima modeling

5 items are checked using Tramo-Seats in Demetra output:– the quality of pre-adjustments : if outliers represent more than

5% of all the observations the series gets a mark of 4 (0 otherwise);

– White noise tests on residuals after Arima modeling: if residuals pass the tests (4 of them) we consider that the modeling is ok (16 points if all 4 are ok).

Possible solutions when the quality is poor:– When there are too many outliers : change the time span;– When the residuals are not white noise : change the model,

change the time span.


2.2 Idempotency

2 items are checked in order to make sure that SA-TD adjusted series show no residual seasonality (TD effects):

– Global TD effects detection test (maximum mark=8);– The X11-Arima F-test for identifiable seasonality (maximum

mark=8);– The weight of the irregular component (maximum mark=4).

Possible solutions when the quality is poor:– The series should not have been TD adjusted;– The pre-adjustments and Reg-Arima modeling is problematic

and should be re-run;– The seasonality is changing and nothing can be done at first.


3. The IIP quality report for April 2007

The quality of SA & TD adjustments seems ok for the IIP:- Idempotency is almost always ok- Whereas the Reg-Arima modeling is of poorer quality.

WeightQuality of the reg-Arima modeling

Idempotency

(mark on 20 points) (mark on 20 points)

210 524 17.5 18.0 17.7 17.7

163 949 18.1 19.0 18.5 14.4

18 577 16.0 16.7 16.4 1.4

30 706 16.7 20.0 18.3 2.7

4 650 12.0 20.0 16.0 0.4

8 323 17.5 20.0 18.8 0.7

12 200 17.1 20.0 18.6 1.1

5 533 18.6 19.9 19.3 0.5

19 144 19.1 20.0 19.6 1.8

40 166 18.2 17.5 17.9 3.4

7 931 19.4 20.0 19.7 0.7

18 076 17.0 15.6 16.3 1.4

14 159 19.2 18.4 18.8 1.3

73 932 18.4 19.1 18.8 6.6

8 134 17.8 19.2 18.5 0.7

4 303 16.6 19.7 18.2 0.4

7 603 19.5 15.7 17.6 0.6

22 151 17.0 19.8 18.4 1.9

21 282 19.3 19.3 19.3 2.0

10 459 19.9 19.4 19.7 1.0

27 999 14.7 13.1 13.9 1.8

33 797 16.0 18.3 17.2 -

EB Agricultural and food industries

EC Consumer-goods industries

C2 Printing and reproduction of recorded media

C3 Pharmacy & perfumes

F4 Manufacture of chemicals

C1 Manufacture of wearing apparel

E2 Machinery and equipement

E3 Electrical & electronic equipment

EF Intermediate-goods industries

F1 Manufacture of non-metallic mineral products

C4 Durable consumer goods

ED Automotive industry

EE Producer-durables industries

E1 Ships, railways & aerospace equipment

ContributionAverage

mark

EH Construction

Manufacturing Industry

Industry

F5 Manufacture metal products,

F6 Manufacture of electrical equipment

EG Energy

F2 Manufacture of textiles

F3 Manufacture of wood, paper & paper products



› 33 elementary indices (out of 120) didn’t pass at least one white-noise test;

› The quality results weren’t satisfactory for wearing apparel & machinery equipment (decreasing production, new surveys);

› An example of what happens when you don’t change your Arima models often enough: the automotive industry in November 2005.

IPI de la branche automobile

90

95

100

105

110

115

120

janv-

00

mar

s-00

mai-

00

juil-0

0

sept

-00

nov-

00

janv-

01

mar

s-01

mai-

01

juil-0

1

sept

-01

nov-

01

janv-

02

mar

s-02

mai-

02

juil-0

2

sept

-02

nov-

02

janv-

03

mar

s-03

mai-

03

juil-0

3

sept

-03

nov-

03

janv-

04

mar

s-04

mai-

04

juil-0

4

sept

-04

nov-

04

janv-

05

mar

s-05

mai-

05

juil-0

5

sept

-05

Série CVSCJO avec anciens paramètres

Série CVSCJO avec nouveaux paramètres



› Idempotency: checking certain properties

– The SA-TD adjusted series must not have seasonal/trading day patterns ;

– Same criterion for the irregular component;– The irregular component should not explain most of the series

evolutions.

› In April 2007, for the IIP :– 25 times out of 120 we detected residual seasonal and trading day effects

‐ The series are unstable (recent surveys)‐ This can happen when particular events close to TD effects happen (some firms close during the summer, change in dates of school vacations, long week-ends…).

– The average weight of the irregular component is almost always very low (inf. to 80%)

– Examples : electricity production was not adjusted for TD effects whereas it should have been ; unstable elementary indexes in “machinery & equipment” ; problems with firms closing in august since 2004 in the automobile industry.


3. Is the tool useful ? How can it be improved ?

› In September 2007, the report was implemented again– we improved the global quality of the IIP;– we corrected some options (electricity production).

› But the tool can be improved– It should be easier to implement ;– It should make more use of X12 quality tests;– It should be harsher on certain criteria (the outlier problem);– It should take other criteria into account (revisions);– It should be made useful to users.

J. Khélif Insee July 2008 A quality report for seasonally and trading day adjusted French IIP.

Documents

Transcript of J. Khélif Insee July 2008 A quality report for seasonally and trading day adjusted French IIP.