Analysis of Variance andAnalysis of Variance andAnalysis of Variance and Analysis of Variance and Design Design of Experimentsof Experiments--II

MODULE MODULE –– IIIIII

gg pp

EXPERIMENTAL DESIGNEXPERIMENTAL DESIGNLECTURE LECTURE -- 12 12

EXPERIMENTAL DESIGN EXPERIMENTAL DESIGN MODELSMODELS

Dr. ShalabhDepartment of Mathematics and Statistics

Indian Institute of Technology Kanpur

We consider the models which are used in designing an experiment.

2

The experimental conditions, experimental setup and the objective of the study essentially determine that what type of

design is to be used and hence which type of design model can be used for the further statistical analysis to conclude

about the decisions. These models are based on one-way classification, two way classifications (with or without

interactions), etc. We discuss them now in detail in few setups which can be extended further to any order of

classification We discuss them now under the set up of one-way and two-way classificationsclassification. We discuss them now under the set up of one-way and two-way classifications.

It may be noted that it has already been described how to develop the likelihood ratio tests for the testing the hypothesis

of equality of more than two means from normal distributions and now we will concentrate more on deriving the same

tests through the least squares principle under the setup of linear regression model. The design matrix is assumed to be

not necessarily of full rank and consists of 0’s and 1’s only.

One-way classification

3

One-way classification

Let p random samples from p normal populations with same variances but different means and different sample sizes

have been independently drawn.

Let the observations follow the linear regression model setup and

Yij: denotes the jth observation of dependent variable Y when effect of ith level of factor is present.

Then Yij are independently normally distributed withij p y y

2

( ) , 1, 2,..., , 1, 2,...,

( )ij i i

ij

E Y i p j n

Var Y

μ α

σ

= + = =

=

where

- is the general mean effect.

- is fixed.

i id b t th l diti f th i t l it d t t t

μ

- gives an idea about the general conditions of the experimental units and treatments.

- is the effect of ith level of the factor.

- can be fixed or random.iα

Example Consider a medicine experiment in which are three different dosages of medicines - 2 mg., 5 mg., 10 mg.

which are given to the patients having fever These are the 3 levels of medicines Let Y denotes the time taken by the

4

which are given to the patients having fever. These are the 3 levels of medicines. Let Y denotes the time taken by the

medicine to reduce the body temperature from high to normal. Suppose two patients have been given 2 mg. of dosage,

so and will denote their responses. So we can write that when is given to the two patients, then1 2mg.α =1 jY 12Y

Similarly, if 5 mg. and 10 mg. of dosages are given to 4 and 7 patients respectively then the responses follow the model

1 1( ) ; 1, 2.jE Y jμ α= + =

Here will denote the general mean effect which may be thought as follows: The human body has tendency to fight against

2 2

3 3

( ) ; 1,2,3,4

( ) ; 1,2,3,4,5,6,7.j

j

E Y j

E Y j

μ α

μ α

= + =

= + =

μHere will denote the general mean effect which may be thought as follows: The human body has tendency to fight against

the fever, so the time taken by the medicine to bring down the temperature depends on many factors like body weight, height

etc. So denotes the general effect of all these factors which are present in all the observations.

μ

μ

In the terminology of linear regression model, denotes the intercept term which is the value of the response variable when

all the independent variables are set to take value zero. In experimental designs, the models with intercept term are more

commonly used and so generally we consider these types of models.

μ

y g y yp

Also, we can express

5

; 1, 2,..., , 1, 2,...,ij i ij iY i p j nμ α ε= + + = =

where is the random error component in . It indicates the variations due to uncontrolled causes which can

influence the observations. We assume that are identically and independently distributed as withijε ijY

'ij sε 2(0, )N σ2

Note that the general linear model considered is

( )E Y X β=

for which can be written asY

2( ) 0, ( ) .ij ijE Varε ε σ= =

for which can be written asijY

( )ij iE Y β=

when all the entries in X are 0’s or 1’s. This model can also be re-expressed in the form of

( ) .ij iE Y μ α= +( ) .ij iE Y μ α+

This gives rise to some more issues.

Consider( )

( )

ij i

i

i

E Y β

β β βμ α

=

= + −

= +where

1

1

.

p

ii

i i

pμ β β

α β β=

≡ =

≡ −

∑

Now let us see the changes in the structure of design matrix and the vector of regression coefficients

6

Now let us see the changes in the structure of design matrix and the vector of regression coefficients.

The model can be now written as

h i 1 t d* ( )β ′

( )ij i iE Y β μ α= = +

2

( ) * *( )

E Y XCov Y I

β

σ

=

=

11

*X

X

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥

where is a p x 1 vector and

is a matrix, and X denotes the earlier defined design matrix with

1 2* ( , , , ..., )pβ μ α α α ′=

( 1)n p× +

• first n1 rows as (1,0,0,…,0),

• second n2 rows as (0,1,0,…,0)

1⎢ ⎥⎢ ⎥⎣ ⎦

We earlier assumed that rank(X) = p but can we also say that rank(X*) = p in the present case?

• …, and

• last np rows as (0,0,0,…,1).

It is thus apparent that all the linear parametric functions of are not estimable. The question now arises

Since the first column of X* is the vector sum of all its remaining p columns, so

( )*rank X p=

1 2, , ..., pα α αpp p q

that what kind of linear parametric functions are estimable?1 2, , , p

Consider any linear estimator with1 1

inp

ij iji j

L a Y=∑∑1

.in

i ijC a=∑

7

Now

1 1

( ) ( )inp

ij iji j

np

E L a E Y= =

= ∑∑

1 1i j= = 1j=

1 1

1 1 1 1

( )

i

i i

np

ij ii j

n np p

ij ij ii j i j

a

a a

μ α

μ α

= =

= = = =

= +

= +

∑∑

∑∑ ∑∑

Thus is estimable if and only if

1 1

( ) .

j j

p p

i i ii i

C Cμ α= =

= +∑ ∑p

Cα∑Thus is estimable if and only if1

i ii

Cα=∑

10

i e is a contrast

,p

iip

C

Cα

=

=∑

∑

Thus, in general neither nor any is estimable. If it is a contrast, then it is estimable.

1

,i.e. is a contrast.i ii

Cα=∑

1

p

ii

α=∑ 1 2, , , ..., pμ α α α

This effect and outcome can also be seen from the following explanation based on the estimation of parameters 1 2, , , ..., .pμ α α α

Consider the least squares estimation of respectively.1 2ˆ ˆ ˆˆ , , , ..., pμ α α α 1 1 2, , , ..., ,pμ α α α

8

2 2

1 1 1 1

t bt i

i in np p

ij ij ii j i j

S ( y )

ˆ ˆˆ

ε μ α= = = =

= = − −∑∑ ∑∑

Minimize the sum of squares due to s 'ijε

1

1 10 0

0 0 1 2

to obtain

(a)

(b)

i

i

p

np

ij ii j

n

ˆ ˆˆ , ,..., .

S ( y )

S ( ) i

μ α α

μ αμ = =

∂= ⇒ − − =

∂

∂⇒

∑∑

∑

Note that (a) can be obtained from (b) or vice versa. So (a) and (b) are linearly dependent in the sense that there are

(p +1) unknowns and p linearly independent equations. Consequently do not have a unique solution.

10 0 1 2(b) ij i

ji

( y ) , i , ,..., p.μ αα =

= ⇒ − − = =∂ ∑

1ˆ ˆˆ , , ..., pμ α αSame applies to the maximum likelihood estimation of

If a side condition that 1 ., , ... .pμ α α

1 1

ˆ 0 0orp p

i i i ii i

n nα α= =

= =∑ ∑

( ) ( )

ˆ , ,1 1 1

is imposed then a and b have a unique solution as

1 say whereinp p

ij oo ii j i

y y n nn

μ= = =

= = =∑∑ ∑

ˆ ˆ

.1

1 in

i ijjp

io oo

yn

y y

α μ=

= −

= −

∑

9

So the model needs to be rewritten so that all the parameters can be uniquely estimated. Thusij i ijy μ α ε= + +

, , ˆ ˆ .1 1 1 1

In case all the sample sizes are the same then the condition or reduces to or0 0 0 0p p p p

i i i i i ii i i i

n nα α α α= = = =

= = = =∑ ∑ ∑ ∑

ij i ij

i ij

* *i ij

Y

( ) ( )

μ α ε

μ α α α ε

μ α ε

= + +

= + + − +

= + +

where

*

*i i

μ μ α

α α α

= +

= −

1

1

and

p

iip

α α=

= ∑

10

p*i

i.α

=

=∑

This is a reparameterized form of the linear model.

Thus in a linear model, when X is not of full rank, then the parameters do not have unique estimates. A restriction

10

p

(or equivalently in case all ni’s are not same) can be added and then the least squares (or

maximum likelihood) estimators obtained are unique. The model1

0p

ii

α=

=∑1

0p

i ii

nα=

=∑

* *1

1

( ) * ; 0p

ij ii

E Y μ α α=

= + =∑

is called a reparametrization of the original linear model.