Week9 Annotated

ACTL2002/ACTL5101 Probability and Statistics: Week 8

ACTL2002/ACTL5101 Probability and Statistics

c© Katja Ignatieva

School of Risk and Actuarial StudiesAustralian School of Business

University of New South Wales

[email protected]

Week 8Probability: Week 1 Week 2 Week 3 Week 4

Estimation: Week 5 Week 6 Review

Hypothesis testing: Week 7 Week 8

Linear regression: Week 10 Week 11 Week 12

Video lectures: Week 1 VL Week 2 VL Week 3 VL Week 4 VL Week 5 VL

mailto:[email protected]


Last eight weeks

Introduction to probability;

Moments: (non)-central moments, mean, variance (standarddeviation), skewness & kurtosis;

Special univariate (parametric) distributions (discrete &continue);

Joint distributions;

Convergence; with applications LLN & CLT;

Estimators (MME, MLE, and Bayesian);

Evaluation of estimators;

Interval estimation;

Hypothesis testing.2301/2350


This week

Special hypothesis tests:

- Fisher exact test;

- Contingency tests:

One sample Binomial.

One sample Multinomial;

r -Sample Multinomial.

- Application contingency tests:

Goodness of fit test;

Test for homogeneity;

Test for independence.

- Non-parametric tests:

Sign test.

Binomial test (test on quantiles);

- Goodness of fit tests.2302/2350


Fisher’s exact test


Hypothesis testing

Fisher’s exact testFisher’s exact test

Contingency tablesOne sample BinomialOne sample Multinomialr -Sample MultinomialExamplesExercises

Non-parametric testsTwo sample testTest on quantilesExercises

Goodness of fit testsAn overviewExercises




Introduction

Last two weeks we have seen hypothesis testing;

One part of it was the test statistic;

To do so, we assumed a distribution for X .

However, there are cases where we don’t know the distributionof X .

Use non-parametric methods: e.g., Fisher exact test, Wilcoxonpaired sample signed rank test.

The non-parametric methods do not rely on the estimation ofthe parameters describing the distribution.

2303/2350




Hypergeometric distribution

Take n draws from a sample without replacement;

Let the total number of objects be N(≥ n);

Have two possible different outcomes (e.g. “blue ball” and“red ball”), one of them (say blue balls) has M(≤ N) objects.

Let X be the random variable of blue balls selected.

This provides us the Hypergeometric distribution:X ∼ HYP(n,M,N) with n = 1, 2, . . . ,N, M = 0, 1, . . . ,N.The probability mass function of a Hypergeometricdistribution is given by:

pX (x) =

(Mx

)·(N−Mn−x

)(Nn

) .

2304/2350




Hypergeometric distribution

0 1 2 3 4 50

0.1

0.2

0.3

0.4

x

probab

ility ma

ss fun

ction Hypergeometric(5,4,10) p.m.f.

0 1 2 3 4 50

0.5

1

xcumula

tive de

nsity f

unction Hypergeometric(5,4,10) c.d.f.

2305/2350




Example

Example: Male bank supervisors were used as the subjects forthe following experiment. They had to make decisions onitems in an in-basket. One experiment was to decide whetherto promote or not.

By random selection, 24 of the supervisors examined a filelabeled as being that of a male employee and 24 examined afile labeled as being that of a female employee. Apart fromthe label, the files were identical. The results were:

Number Male Female

Promote x =21 14 M =35Hold file 3 10 13

n =24 24 N =48

Note: Given margins X ≤ 24, more on slide 2309.2306/2350




Example

It appears there is evidence of sex bias. But could this haveoccurred by chance? That is, even if there were no bias andthe supervisors were oblivious to gender, discrepancies likethose observed could occur with fairly large probability bychance alone.

Let us represent counts as (2× 2 contingency table):

N11 N12 n1·N21 N22 n2·n·1 n·2 n··

Question: How to test relationship between two randomvariables?

Solution: One way: Chi-squared test, approximate for large n.

Alternative: Fisher’s exact test.2307/2350




Fisher’s exact test of independence

The Fisher exact test is given by:

Test (H1 can be two-sided or one-sided):

H0 :N11

n·1=

N12

n·2v.s. H1 :

N11

n·16= N12

n·2, for a given α.

H0 implies no sex bias, i.e., differences due to randomization.

Margins (n1·, n2·, n·1, n·2, and n··) are known constants.

Test statistic: We have N11 is hypergeometric with p.m.f.:

pN11(n11) =

(n1·n11

)(n2·n21

)(n··n·1

) ,

i.e., N = n··, n = n·1, M = n1·, x = n11, n2· = N −M, and n21 = n− x .2308/2350




Determine rejection region:One sided test: C = min|n1· − n·2|, |n2· − n·1|, . . . , ul orC = ll , . . . ,minn1·, n·1. Note, the min is due toconstraint on marginals.

We have for the one-sided test that (see slide 2306)

ul = argminx

∑xy=min|n1·−n·2|,|n2·−n·1| pN11(y) ≤ α

, or

ll = argmaxx

∑minn1·,n·1x pN11(y) ≤ α

.

Two sided test: C =min|n1·−n·2|, |n2·−n·1|, . . . , ul∪ll , . . . ,minn1·, n·1,where ll and ul are given using α/2.

Rejection region for a two-sided test with α =0.05C = 11, . . . , 14, 21, . . . , 24. Observed value of falls in thisregion, thus reject H0.

pN11(11) = 1.29 · 10−5, pN11(12) = 3.36 · 10−4, pN11(13) = 0.0035,pN11(14) = 0.021. Thus Pr(N11 ≤ 14) = Pr(N11 ≥ 21) = 0.0245.2309/2350


Contingency tables

One sample Binomial

Hypothesis testing






Contingency tables

One sample Binomial

First, consider c = 2, then O1 ∼ Bin(n,E1/n). We have:

T =(O1 − E1)2

E1 · (1− E1/n)∗=

(O1 − E1)2

E1+

(O1 − E1)2

n − E1

=(O1 − E1)2

E1+

((n − O1)− (n − E1))2

n − E1

∗∗=

(O1 − E1)2

E1+

(O2 − E2)2

E2

** O2 ≡ n − O1 and E2 ≡ n − E1; * where:

1

E1(1− E1/n)=

n

E1 · (n − E1)=

n−E1 + E1

E1 · (n − E1)

=n − E1

E1 · (n − E1)+

E1

E1 · (n − E1)=

1

E1+

1

n − E1.

Note that√T equals o1−n·p√

n·p·(1−p)hence is approximately

standard normal distributed. Thus, T is approximately χ2(1)distributed ⇒ appropriate test statistic.

2310/2350


Contingency tables

One sample Multinomial

Hypothesis testing






Contingency tables


One sample MultinomialWe specify c cell probabilities of one variable: One sampleMultinomial.

This gives us a c by one contingency table.

We have n outcomes A1, . . . ,Ac , with specified probabilitiesPr(A1), . . . ,Pr(Ac).

Example: a group of individuals is asked whether they preferto invest their super in:

1) riskfree asset;2) partly risky asset, partly riskfree asset;3) risky asset.

The null hypothesis is:

H0 :p1 = p2 = p3 = 1/3

v.s. H1 :p1 6= p2 or p1 6= p3 or p2 6= p3.2311/2350


Contingency tables


χ2 densities

0 2 4 6 8 100

0.5

1

xprob

abilit

y den

sity f

uncti

on χ2(ν) p.d.f.

ν = 1ν = 3ν = 10

0 2 4 6 8 100

0.5

1

xcum

ulativ

e de

nsity

func

tion χ2(ν) c.d.f.

2312/2350


Contingency tables


One sample MultinomialWe perform an experiment with 90 individuals, and find thefollowing:

# times selected Expected Observed (Oi − Ei )2/Ei

1) Riskless 23 30 23 49/302) Combination 36 30 36 36/303) Risky 31 30 31 1/30

Total 90 90 90 86/30

Note: 31 = 90− 23− 36.

Do the data present sufficient evidence to warrant rejection ofthe hypothesis of no preference?

Test statistic: T =c∑

i=1

(Oi−Ei )2

Ei, reject for large values of T .

T is chi-squared distributed with c−1 degrees of freedom(last cell is determined by total and values of other cells).2313/2350


Contingency tables


Example: (Chi-squared) Goodness of fit testUse one sample multinomial case.

Adopt this test for a complete specified distribution, i.e.,H0 : X ∼ FX (x) v.s. H1 : X has another distribution.

Divide the sample space in c cells, A1, . . . ,Ac and letEj = n · Pj , where Pj = Pr(X ∈ Aj), and X ∼ FX (x).

We have Oj ∼ Bin(n,Pj)→ N(Ej ,Ej · (1− Pj)) ≈ N(Ej ,Ej).

Hence, we have a multinominal test:

T =∑c

j=1(Oj−Ej )

2

Ej∼ χ2(df ), with df = c − 1 degrees of

freedom.

Note: MLE for k parameters: df = c − 1−k.

General principal for selecting cells: as many (⇒ Pj → 1),such that (np =)Ej≥ 5, i.e., highest degree of freedom, butchi-squared approximation is fairly accurate.

2314/2350


Contingency tables


Example: bombs. The following data come from an article byR.D. Clarke in the 1946 Journal of the Institute of Actuaries. Itdescribes the pattern of damage by German V-2 bombs during theSecond World War.

No. of flying bombs Actual no. of

per square squares Expected (Oi−Ei )2

Ei

0 229 Pr(X = 0) · n =228 0.00951 211 Pr(X = 1) · n =211 0.00052 93 Pr(X = 2) · n =98 0.26983 35 Pr(X = 3) · n =30 0.7004 7 Pr(X = 4) · n =7 0.0004

5 and over (7 bombs) 1 Pr(X ≥ 5) · n =2 0.1917

Total 576 576 1.17Let Xi |H0 ∼ Poi(λ) be the number of bombs on square i .

λ = x =∑n

i=1 xin = 0·229+1·211+...+7·4+1·7

576 = 537576 .

n = 576 number of squares; Pr(Xi = j) = e−λ·λjj! .

2315/2350


Contingency tables


Example: Goodness of fit test: bombs

Clarke noted: ‘During the flying bomb attack on London, frequentassertions were made that the points of impact of the bombstended to be grouped in clusters. . . ’, rather than displaying arandom distribution over the metropolis. It was as if certain areaswere specifically targeted, or as if bombs tended to fall together forsome mechanical reason. Both these explanations appearedunlikely. To determine whether clustering really took place, ‘. . . 144square kilometres of south London [were] . . . divided into 576squares of 1/4 square kilometre each, and a count was made of thenumber of squares containing 0, 1, 2, 3, . . . , etc. flying bombs.Over the period considered the total number of bombs within thearea involved was 537. The expected numbers of squarescorresponding to the actual number yielded by the count were thencalculated from the Poisson formula. . . .’

2316/2350


Contingency tables

r -Sample Multinomial

Hypothesis testing






Contingency tables



Suppose we have now r independent samples of a Multinomialpopulation (gives a c by r contingency table).

For j = 1, . . . , r define c outcomes A1j , . . . ,Acj ;

Denote, under the null pi |j = Eij /∑c

i=1 Eij , we have:∑ci=1 pi |j = 1.

Test H0 : pi |j = p0i |j , we have Eij = nip

0i |j .

Test statistic:

T =r∑

j=1

c∑i=1

(Oij − Eij)2

Eij∼ χ2(df )

with df = (c−1) · (r−1) degrees of freedom (recall: sum ofchi-squared r.v.)

2317/2350


Contingency tables

Examples

Hypothesis testing






Contingency tables

Examples

Example homogeneity: We consider three different healthinsurers with the same number of insured and the number oftreatments of six different group (times 1000). Is there a differencebetween the insureds of the different insurers?

Treatment Insurer A Insurer B Insurer C Total

Pharmaceutic 147 186 101 434Intensive care 25 26 11 62Podiatry 32 39 15 86Ambulance 94 105 37 236Hospital 59 74 28 161Chiropractic 18 10 10 38

total 375 440 202 1017

Test whether all cell probabilities are equal.Note: r = 3 independent samples; c = 6 treatments.18 = 375− 147− . . .− 59 and 37 = 236− 94− 105. (table withexpected: see slide 2320)2318/2350


Contingency tables

Examples

Example: Goodness of fit: homogeneity

We test:

H0 : πi1 = πi2 = . . . = πiJ , for i = 1, . . . , r ,i.e., H0 : πi1 = πi2 = πi3, for i = 1, . . . , 6.

We can use Pearson’s chi-square statistic:

T =r∑

i=1

c∑j=1

(Oij − Eij)2

Eij∼ χ2(df )

with degrees of freedom given by:

df = (c−1) · (r−1).

2319/2350


Contingency tables

Examples

Example: Goodness of fit: homogeneity

Expected Insurer A Insurer B Insurer C Total πMLi

Pharmaceutic 160 188 86 434 0.427Intensive care 23 27 12 62 0.061Podiatry 32 37 17 86 0.085Ambulance 87 102 47 236 0.232Hospital 59 70 32 161 0.158Chiropractic 14 16 8 38 0.037

Total 375 440 202 1017πMLj = 375

1017440

1017202

1017 1

We have t =∑r

j=1

∑ci=1

(Eij−Oij )2

Eij= 12.27, χ2

0.95(10) = 18.5, thus

we can not reject H0. Table with observed: see slide: 2318.where 59 = n · 0.158 · 375

1017 ; 12 = n · 0.061 · 2021017 and 0.037 = 38

1017 .2320/2350


Contingency tables

Examples

Example: Chi-square test of independence

Example: A death benefit insurer is interested in whether theprobability of reaching the age of 80 is dependent on themarital status.Question: why this of interest?

Married Not married Total

LE > 80 550 61 611LE ≤ 80 681 144 825

Total 1 231 205 1 436

“If I were to take a sample of 1 436 people cross-classified intotwo categories which were in fact unrelated in the populationfrom which the sample was drawn, I might find associations asstrong or stronger than those observed in this table. Whyshould I believe that there is any real association in yourtable?”

2321/2350


Contingency tables

Examples


The joint distribution of the counts nij , where i = 1, . . . , r andj = 1, . . . , c is Multinomial with cell probabilities denoted byπij .

Let:

πi · =c∑

j=1

πij and π·j =r∑

i=1

πij

denote the marginal probabilities that an observation will fallin the i th row and j th column, respectively.

If the row and column classifications are independent of eachother

πij = πi ·π·j .2322/2350


Contingency tables

Examples


We thus test:

H0 : πij = πi ·π·j , for i = 1, . . . , r , j = 1, . . . , cv.s. H1 : at least one πij 6= πi ·π·j .

You can show, under H0 (independence), the MLE of πij is:

πij = πi ·π·j =ni ·n·n·jn.

Under the alternative, the MLE is πij = nij/n.These estimates can be used to form a LRT or anasymptotically equivalent Pearson’s χ2 test:

r∑i=1

c∑j=1

(Oij − Eij)2

Eij∼ χ2(df ),

where Eij = n · πij =ni·n·jn and Oij = n · πij = nij .

2323/2350


Contingency tables

Examples


with degrees of freedom:

df = r · c − 1− (r − 1)︸︷︷︸# par πi

− (c − 1)︸︷︷︸# par πj

= (r − 1) · (c − 1).

We would reject the null hypothesis of independence andconclude that there is a relationship between marital statusand the probability of reaching age 80.

The chi-square statistic used here to test independence isidentical in form and degrees of freedom to that used earlierto test homogeneity/goodness of fit. The hypotheses aredifferent and the sampling schemes are different.

2324/2350


Contingency tables

Exercises

Hypothesis testing






Contingency tables

Exercises

Exercise: Are storm claims independent

Two insurers, SC and RS, have provided some data abouttheir storm claims to an institute.

Unfortunately you are not bale to obtain more detailed datathan the following:

C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 total# claims SC 1214 434 155 168 29 2000# claims RS 1430 320 113 126 11 2000Total 2644 754 268 294 40 4000

Question: Are the claims of SC and RS dependent?

2325/2350


Contingency tables

Exercises

Solution: Are storm claims independent

Solution: T =∑ (Ei−Oi )

2

Ei∼ χ2(4), χ2

0.95(4) = 9.49

χ20.99(4) = 13.28. We have t = 55.56, thus reject H0.

C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 TotalE# claims SC 1322 377 134 147 20 2000E# claims RS 1322 377 134 147 20 2000proportion 2644

4000754

4000268

4000294

400040

4000 4000(Oi − Ei ) SC 108 −57 −21 −21 −9(Oi − Ei ) RS −108 57 21 21 9(Oi−Ei )

2

EiSC 8.82 8.62 3.29 3 4.05 27.78

(Oi−Ei )2

EiRS 8.82 8.62 3.29 3 4.05 27.78

Question: Insurer SC currently uses a Pareto(α = 2, λ = 100)(with F (x) = 1− (1 + x/λ)−α) for the distribution of theclaim size when an insured issues a claim. Is that the correctdistribution?

2326/2350


Contingency tables

Exercises

Solution: Are storm claims Pareto(2,100) distributedSolution:

C = 0 C < 50 50 < C < 100 100 < C < 500 C > 500 total# claims SC 1214 434 155 168 29 2000F (X ) 5/9 3/4 35/36 1E# claims SC 436.67 152.83 174.67 21.83 786(Oi − Ei ) −2.67 2.167 −6.67 7.167(Oi−Ei )

2

Ei0.016 0.0307 0.2544 2.35 2.65

Note: Discard C = 0, because only interested in when aclaim is made.

T =∑ (Ei−Oi )

2

Ei∼ χ2(3), χ2

0.95(3) = 7.81 χ20.99(3) = 11.34.

We have t = 2.65, thus accept H0.

Note: if parameters where estimated: df = 3− 2 = 1,χ2

0.95(1) = 3.84 χ20.99(1) = 6.6. Still accept H0.

2327/2350


Non-parametric tests

Two sample test

Hypothesis testing







Two sample test

One sample sign test

Consider a random variable X ∼ F (x), let m be the median.We observe xi , . . . , xn.

Test: H0 : m = m0 v.s. H1 : m < m0.

Let ti = 1xi−m0<0 =

1, if xi < m0;0, else.

.

Under H0, each ti is a Bernoulli experiment, with p = 0.5.

Test statistic: T =∑n

i=1 ti ∼ Bin(n, p).

Reject H0 if B(t; n, 0.5) ≤ α (the c.d.f. of a Bin(n, p) in t)Example: test whether mean claim size is $24.5 thousand,sample of 20 observations:

10.8 12.7 13.9 18.1 19.4 21.3 23.5 24.0 24.6 25.025.4 27.7 30.1 30.6 32.3 33.3 34.7 38.8 40.3 55.5

Solution: B(t = 8; 20, 0.5) = 0.2517, thus cannot reject H0.2328/2350



Two sample test

Two sample sign testWe have paired data (xi , yi ). Are X and Y independent?

Test: H0 : Pr(X < Y ) = 1/2 v.s. H1 : Pr(X − Y < 0) < 1/2.

Let ti = 1xi−yi<0 =

1, if xi < yi ;0, else.

.

Under H0, each ti is a Bernoulli experiment, with p = 0.5.

Test statistic: T =∑n

i=1 ti ∼ Bin(n, p).

Reject H0 if B(t; n, 0.5) ≤ αExample: A MVI offers contracts in fifteen regions, elevenhad an increase in mean claim size and four had a decrease.Did the claim size increase?Solution: p-value: B(t = 4; 15, 0.5) = 0.0592, at α = 6% itis significant that the mean claim size had increased.

2329/2350



Two sample test

Wilcoxon paired sample signed-rank test: We have paireddata (xi , yi ). Do X and Y have the same distribution?

Let di = xi − yi denote the difference of the pairs.Rank absolute differences: u1 = min|di |, . . . , un = max|di |.Test H0 : FX = FY v.s. H1 : FX 6= FY .

Let T+ (T−) the sum of the positive (negative) ranks.

Let Vi = 1ui>0, under H0 we have Vi ∼ Ber(1/2) then

E [T+] = E [∑n

i=1 i · Vi ] =∑n

i=1 i · E [Vi ]∗= n(n + 1)/4, *

using E[Vi ] = p = 1/2 and∑n

i=1 i = n(n+1)2 and

Var (T+) = Var (∑n

i=1 i · Vi ) =∑n

i=1 i2 · Var (Vi )

∗∗=∑n

i=1 i2/4

∗∗∗= n(n + 1)(2n + 1)/24. ** using Var(Vi ) =

p(1− p) = 1/4 and *** using∑n

i=1 i2 = n(n+1)(2n+1)

6 .

Test statistic: T = (T+ − E [T+])/√

Var (T+) ∼ N(0, 1) orS = (T− − E [T−])/

√Var (T−) ∼ N(0, 1), using CLT;

and in case of a two-sided test: max(T ,S).2330/2350



Two sample test

Example: Wilcoxon paired sample signed-rank test

Test whether the distribution of the number of deaths in caraccidents with a female driver is lower than with a maledriver. Data 7 months:

Month 1 2 3 4 5 6 7

Females 103 118 164 103 146 150 137Males 112 139 111 186 207 154 131di 9 21 -53 83 61 4 -6i =Rank 3 4 5 7 6 1 2

E [T+] = n · (n + 1)/4 = 14, with n = 7 months andVar (T+) = n · (n + 1) · (2n + 1)/24 = 35.

t = (t+ − E [T+])/√

Var (T+) = (21− 14)/√

35.

Φ(

21−14√35

)= 0.12. Cannot reject H0 at 5% confidence level.

2331/2350



Test on quantiles

Hypothesis testing







Test on quantiles

Test on quantiles

If we have an empirical (simulated) distribution, how todetermine the CI of a quantile of the population distributionFX (x)?

Test H0 : xq = x(i) v.s. H1 : xq > x(i) at α, for a quantile q.

Let t be the number of observations in the sample that aresmaller than the qth quantile xq. Then we have:T ∼ Bin(n, q).

Reject H0 if B(t; n, q) ≤ α.

Thus: Pr(xq < x(t)) = 1− α, where t is

argmaxt

∑tj=0

(nj

)· pj0 · (1− p0)n−j ≤ 1− α

.

2332/2350



Test on quantiles

Example: Test on quantiles

We are interested in the reserves for a mortgage life insurance(with a large pool).

The contract implies that you pay a fixed premium for thenext 40 years.

When you die before the end of the contract the insurer paysthe principal.

Contract has an expected profit in the first years and anexpected loss in the last years.

An insurer should hold enough capital in order to pay theexpected loss in the future.

2333/2350



Test on quantiles

Example: Test on quantiles

Reserve equals the 97.5% quantile.

Value does depend on evolution of mortality: unknown & noclosed form (special) distribution.

Use N = 1000 simulations.

Ordered simulations: x(975) = 23.05, x(982) = 23.81, andx(983) = 24.03.

T ∼ Bin(1000, 0.975), with Pr(T ≤ 983) = 0.964 andPr(T ≤ 982) = 0.942.

Hence, we can reject that the 97.5% quantile is larger than24.03 with probability 95%.

2334/2350



Exercises

Hypothesis testing







Exercises

Exercise: find the mean of GHI and BHIWe have the average claim size of health insurers GHI andBHI over time. The average claim sizes are:

Year 2011 2010 2009 2008 2007

GHI 13.73 1.13 5.35 3.28 9.31BHI 16.04 4.92 3.51 18.65 4.11

Year 2006 2005 2004 2003 2002

GHI 2.67 6.42 1.50 6.10 3.75BHI 4.26 3.46 5.48 2.19 9.35

Assume that the average claim size of health insurance has asymmetric distribution.

Question: Find the price, which is set equal to the 75%quantile of the mean claim size for both GHI and BHI.

Solution: T ∼ Bin(10, 0.5). Hence Pr(T ≤ 6) = 0.8281.Thus P[GHI ] = 5.35 and P[BHI ] = 4.92.2335/2350



Exercises

Exercise: Capital requirement GHIA consultancy firm has made some simulations for GHI for theaverage claim size next year.

Provided are 10,000 simulations of the average claim size.

A subset of their (ordered) simulations are:

Simulation 9950 9951 9952 9953 9954 9955 9956 9957 9958

GHI 21.68 21.84 21.94 21.97 22.04 22.07 22.11 22.12 22.13

Simulation 9959 9960 9961 9962 9963 9964 9965 9966 9967

GHI 22.26 22.49 22.86 22.86 22.96 23.56 23.64 23.86 24.02

Question: What is the required reserve for next year (the99.5% quantile) with probability 90% (one sided).

Solution: T ∼ Bin(10000, 0.995) ≈ N(9950, 9950 · 0.005 = 49.75).

Pr(

X−µσ < z0.95

)= 95%⇒ x(9950+1.96·

√49.75) = x(9963.8) = 23.56.

Recommendation: more simulations required!2336/2350



Exercises

Exercise: Are GHI and BHI independentGHI has made lots of profits the past decade, whereas BHIhas made some losses over the last decade.GHI is in the process of taking over BHI. This was initiated bythe board of directors, who thought that the claim sizes ofBHI and GHI are independent.If so, this might reduce their price, leading to more attractivepolicies and thus more insured.Question: The BoD has asked you to verify whether they arecorrect? They are convinced at a 90% significance.Solution: When testing H0 :BHI and GHI are independentv.s. dependent one obtains: T ∼ Bin(10, 0.5). HencePr(T ≤ 7) = 0.9453⇒ C = 7, . . . , 10. We observe T = 6,thus cannot reject H0, hence independent.Question: Comment on the test performed.Solution: This assumes H0, but α = 0.1, does not imply β = 0.1!

2337/2350



Exercises

Exercise: Is GHI “better” than BHIGHI is still in the process of taking over BHI.

The consultant argues that the claim distribution of BHI isdifferent than GHI.

Question: Can you verify wether the consultant is correct?

Solution: Year 2011 2010 2009 2008 2007

di -2.31 -3.79 1.84 -15.37 5.2i 3 5 2 10 8

Year 2006 2005 2004 2003 2002

di -1.59 2.96 -3.98 3.91 -5.6i 1 4 7 6 9

Hence T− = 3 + 5 + 10 + 1 + 7 + 9 = 35Φ((35− 27.5)/

√96.25) = Φ(0.76) = 0.72.

Conclusion: Do not pay the bill!2338/2350


Goodness of fit tests

An overview

Hypothesis testing







An overview

Tests already seen

Last week: Quantile-Quantile test (bootstrap test version ofCW test).

Likelihood ratio test (bootstrap test (week 8) & approximatetest (week 7)):

T (X ) = −2 log (Λ) (= −2 (log (f0(x))− log (fA(x))))

has -under the null hypothesis- asymptotically a chi-squareddistribution with degrees of freedom dim (Ω)− dim (Ω0).

This week: Pearson chi-squared test.

Other test: AD, CV, KS, and K: based on empirical c.d.f. andhypothesized c.d.f. (note: quantile function is uniformlydistributed, see exercise 3a week 6).

2339/2350



An overview

Anderson-Darling & Cramer-von Mises test OPTIONALTest H0 : X ∼ F (X ; θ) v.s. H1 : X F (X ; θ). Test statistic:

T =n ·∫ ∞−∞

(Fn(x)− F (x))2 · w(x)dF (x)

where w(x) =

1, if T = CV , Cramer-von Mises test;

(F (x) · (1− F (x)))−1, if T = AD, Anderson-Darling test.

AD =n∑

i=1

1− 2i

n·(log(Fn(x(i))) + log(1− Fn(x(n+1−i)))

)− n

CM =1

12n+

n∑i=1

(Fn(x(i))−

i − 0.5

n

)2

, (discrete sample).

When parameters are not estimated, we have that Fn(x) =U(i), i.e., distribution of T does not depend on FX (x).

If parameters are estimated, the distribution does not dependon the parameter values, but will depend on form of FX (x).

2340/2350



An overview

Figure Anderson-Darling & Cramer-von Mises test

0 0.4 1.6 30

0.3297

0.5

0.7981

1

Exp(1) c.d.f.e.c.d.f.

2341/2350



An overview

Critical values Anderson-Darling & Cramer-von Mises test

H0 Statistic α = 0.1 0.05 0.025 0.01

F (x) CM 0.347 0.461 0.581 0.743

EXP(λ) (1 + 0.16/n) · CM 0.177 0.224 0.273 0.337

N(µ, σ2) (1 + 0.5/n) · CM 0.104 0.126 0.148 0.178

WEI(θ, β)(1 + 0.2/

√n)· CM 0.102 0.124 0.146 0.175

F (x) AD 1.933 2.492 3.070 3.857

EXP(λ) (1 + 0.6/n) · AD 1.078 1.341 1.606 1.957

N(µ, σ2)(1 + 0.75/n − 2.25/n2

)· AD 0.631 0.752 0.873 1.035

WEI(θ, β)(1 + 0.2/

√n)· AD 0.637 0.757 0.877 1.038

Anderson and Darling (1954), Pettitt and Stephens (1976),Stephens (1977), D’Agostino (1986), Stephens (1986), Pearson &Hartley (1972)CV test for Poisson: Spinelli and Stephens (1997)2342/2350

http://www.jstor.org/stable/pdfplus/2281537.pdf?acceptTC=true

http://biomet.oxfordjournals.org/content/63/2/291.full.pdf+html


http://onlinelibrary.wiley.com/doi/10.2307/3315735/pdf



An overview

Kolmogorov-Smirnoff/Kuiper test OPTIONALThe KS test statistic is based on the maximum differencebetween the sample CDF and the hypothesized CDF.H0 : X ∼ FX (x) v.s. H1 : X FX (x). Let:

D+ =maxi

i

n− F (x(i))

D− =max

i

F (x(i))− i − 1

n

D︸︷︷︸

KS-statistic

= maxD+,D−

V︸︷︷︸

Kuiper-statistic

= D+ + D−

Distribution of statistics does not depend on FX (x).If parameters are estimated, the distribution does not dependon the parameter values, but will depend on the form ofFX (x).Note: approximate test, but good for small n.

2343/2350



An overview

Figure Kolmogorov-Smirnoff/Kuiper test

D− = 0.117

D+ = 0.072

Exp(1/3.7) c.d.f.e.c.d.f.

2344/2350



An overview

Critical values Kolmogorov-Smirnoff/Kuiper testH0 Statistic α = 0.1 0.05 0.025 0.01

F (x)(√

n + 0.12 + 0.11/√n)· D 1.224 1.358 1.480 1.628

EXP(λ)(√

n + 0.26 + 0.5/√n)· (D − 0.2/n) 0.995 1.094 1.184 1.298

N(µ, σ2)(√

n − 0.1 + 0.85/√n)· D 0.819 0.895 0.955 1.035

WEI(θ, β)√n · D 0.803 0.874 0.939 1.007

F (x)(√

n + 0.155 + 0.24/√n)· V 1.620 1.747 1.862 2.001

EXP(λ)(√

n + 0.24 + 0.35/√n)· (V − 0.2/n) 0.527 1.655 1.774 1.910

N(µ, σ2)(√

n + 0.05 + 0.82/√n)· V 1.386 1.489 1.585 1.693

WEI(θ, β)√n · V 1.372 1.477 1.557 1.671

Chandra et al. (1981): uses extreme value distribution, which isrelated to Weibull distributionCritical values see: Stephens (1974, 1977, 1979), Aho et al.(1983), Dufour and Maag (1978), Koziol (1989), and Bain andEngelhardt (1983).2345/2350

http://www.jstor.org/stable/pdfplus/2287539.pdf



http://www.tandfonline.com/doi/pdf/10.1080/00949658308810677


http://www.jstor.org/stable/pdfplus/1268157.pdf






An overview

Comparison1. Plot ecdf and cdf in a graph, is it a good fit?

2. Compare the characteristics of the distributions (mean,variance, skewness, kurtosis).

3. Plot QQ-plot ⇒ is it a good fit.

1-3 Graphical methods give an idea of the fit (good/bad,tail/around median etc), interpret it in your setting!

Chi-squared test: easy to adjust for estimated number ofparameters, approximate test.

KS/Kuiper test: good power, hard to work with; moresensitive near center of the distribution;

AD/CM test: good power, hard to work with. CM/AD testplaces higher weight on the tails.2346/2350



Exercises

Hypothesis testing







Exercises

Exercise: CM OPTIONALConsider the sample from slide 2341 with only twoobservations. Do the following tests at a level of significanceof 5%.

Question: Perform the CM test if the the parameter was notestimated.

Solution: StatisticCM = 1/24 + (0.3297− 0.25)2 + (0.7981− 0.75)2 = 0.050,critical value: 0.461, Thus do not reject H0.

Question: Perform the CM test if the the parameter wasestimated.

Solution:CM = 1/24 + (0.3297− 0.25)2 + (0.7981− 0.75)2 = 0.050;statistic (1 + 0.08) · 0.050 = 0.054 critical value: 0.224, Thusdo not reject H0.2347/2350



Exercises

Exercise: AD OPTIONALConsider the sample from slide 2341 with only twoobservations. Do the following tests at a level of significanceof 5%.

Question: Perform the AD test if the the parameter was notestimated.

Solution: Statistic AD = −1/2 · (log(0.3297) + log(1−0.7981)) +−3/2 · (log(0.7981) + log(1− 0.3297))− 2 = 0.293,critical value: 2.492, Thus do not reject H0.

Question: Perform the AD test if the the parameter wasestimated.

Solution: AD = −1/2 · (log(0.3297) + log(1− 0.7981)) +−3/2 · (log(0.7981) + log(1− 0.3297))− 2 = 0.293; statistic(1 + 0.3) · 0.293 = 0.381 critical value: 1.341, Thus do notreject H0.2348/2350



Exercises

Exercise: KS/Kuiper OPTIONAL

Consider the sample from slide 2341 with only twoobservations. Perform the KS and Kuiper tests at a level ofsignificance of 5%.

We have:

i F (x(i)) in

i−1n

in − F (x(i)) F (x(i))− i−1

n

1 0.3297 1/2 0/2 0.1703 0.32972 0.7981 2/2 1/2 0.2019 0.2981

D+ = 0.2019 D− = 0.3297

Question: Perform the KS test if the parameter was notestimated.

Solution: D = 0.3297; statistic:(√

2 + 0.12 + 0.11/√

2) · 0.3297 = 0.531 critical value: 1.358,thus do not reject H0.

2349/2350



Exercises

Exercise: KS/Kuiper (cont.) OPTIONALQuestion: Perform KS test if the parameter was estimated.

Solution: D = 0.3297; statistic:(√

2 + 0.26 + 0.5/√

2) · (0.3297− 0.1) = 0.466 critical value:1.094, thus do not reject H0.

Question: Perform the Kuiper test if the parameter was notestimated.

Solution: V = 0.5316; statistic:(√

2 + 0.115 + 0.24/√

2) · 0.5316 = 0.903 critical value:1.747, thus do not reject H0.

Question: Perform Kuiper test if the parameter wasestimated.

Solution: V = 0.5316; statistic:(√

2 + 0.24 + 0.35/√

2) · (0.5316− 0.1) = 0.821 critical value:1.655, thus do not reject H0.2350/2350

Week9 Annotated

Documents

Transcript of Week9 Annotated