€¦ · Web view1. Cross-sectional m ultilevel s ituations. Suppose you’re interested in the...

L1 - 1

Multilevel Situations - Situations in which data values are in several random groups.

1. Cross-sectional multilevel situations.

Suppose you’re interested in the relationship of Math achievement test scores to individual student SES level – the social economic status of the individual student.

Easy analysis - a correlation or regression analysis

Math is dependent variableSES is independent variable

Simplest possible data editor . . . (an extraneous variable has been blocked out)

Of course, other variables might be involved in the analysis.

The dataset, which we’ll analyze later in the course, consists of 6800+ students.

Potential problem – which this course is about.

The kids in the study are most certainly members of essentially random groups – their schools.

The assignment to school is a function of many factors. It might be considered to be essentially random assignment or it might not.

The schools represented are probably randomly or at least haphazardly selected from a population of schools.

Randomly here means that if the experiment were repeated, different groups, i.e., schools, would be included.

The groups in the population will have different compositions. There may exist specific characteristics of the groups, such as group size, group proportion of males, number of years experience of the teacher of each group. These group characteristics will likely influence all the kids in that specific group in the same way, although differently from the influences of other groups on the kids in those other groups.

In addition to the different influences of each group on its individuals, the individuals within each group would probably be more like each other than they were like individuals in other groups.

L1 - 2

So, the data editor, really looks like the following . . .(ch3multilevel.sav in mdbT\Heck text . . .)

The students in this example are grouped by school, with from 10 to almost 30 students from each school. More than 400 schools are represented in this particular sample of 6800+ students.

That grouping of students, which 50 years ago would have been noticed but then ignored, has been found to alter how the simple regression of Math Achievement onto SES must be conducted.

Reason for this course: The results taking the grouping into account are different from those not taking them into account.

Multilevel analyses are analyses which take the effects of such grouping into account.

So, a cross-sectional multilevel modeling situation is any regression situation involving different individuals in which the individuals are grouped into random groups.

L1 - 3

2. Longitudinal Multilevel Situations.

Observations are taken at different time points from the same individuals.

Typically, our interest is on mean change across time periods.

Traditionally, Repeated Measures Analysis of Variance has been used to assess significance of mean change across time points. Repeated measures analyses were introduced in PSY 5130 and we had a week of Repeated Measures Analysis in the Fall course.

But many years ago, researchers noticed that each person gives us a group of scores – all of the measures for that person across the several time points.

Amazingly, they discovered that the analysis of this type of data is the SAME as the analysis for the data illustrated above in which persons are in groups.

That is: Scores at different time periods from a specific person are analogous to scores of different persons within a group, such as a school or classroom.

Thus, the same type of analysis – the same computer program and procedure within that program – is appropriate for two seemingly different research situations – analyses of between-subjects relationships involving persons collected within groups and analyses of repeated measures relationships involving scores collected within persons.

Multilevel analyses bring together two diverse collections of analyses . . .

1. Cross-sectional data in which individuals are grouped2. Longitudinal data.

L1 - 4

So what?

Technical

1. Random and systematic differences between the groups and differences in variability of scores within groups will affect the estimates of regression coefficients – the Bs and Betas - relating dependent variable characteristics of the individuals to independent variable characteristics. We’ll get different estimates of effects when we take grouping into account.

2. The standard errors of the estimates, used for tests of significance, are often different when grouping is taken into account, often leading to differences in patterns of significant relationships than would be reached if the grouping were ignored.

It is now believed by pretty much everyone who has studied the issues involved that the estimates and significance tests from the grouped analyses are the ones that should be reported and used.

Substantive

3. There may be group characteristics that systematically influence the dependent variable, accounting for variance that would not be accounted for if grouping were not acknowledged. If we ignore the grouping, we ignore the effects that groups might have on our dependent variables. We can make better predictions if we take grouping into account.

4. The existence of or amount of variation in estimates associated with differences between groups, e.g., the amount of variation in slopes or intercepts, may be of interest in their own right.

All of these are reasons to take grouping into account, if it’s present, when performing analyses.

Concepts important for Multilevel AnalysesContextual variables

The groups may have characteristics that are important for the analyses. For that reason, group characteristics are often called contextual variables. So, a contextual variable is a characteristic of a group (not the group, but a characteristic of the group.)

Examples: If individuals were grouped in schools, school size would be a contextual variable.

If individuals were grouped in organizations, organization verticality might be a contextual variable.

They’re called contextual because each group creates a context in which the relationships of the dependent variable to independent variable(s) for the individuals within the groups exist. Those contexts vary from group to group.

L1 - 5

Random vs. Fixed factors

Fixed Factors.

Most characteristics we’ve previously studied are fixed from one study to the next. Training Program: If the experiment were replicated, two training programs, for example, would be the same two training program.

Sex: If the experiment were replicated, Maleness and Femaleness do not change. These characteristics are fixed from one experiment to the next.Graduate program: I/O vs ResearchTraining program: Computerized vs. Lecture

These are called fixed factors. If the experiment is repeated, the key characteristics associated with the two or more groups would be the same, i.e., fixed.

Random Factors.

On the other hand, if we’re dealing with groups randomly selected from a population of groups, then the characteristics of the groups will likely change randomly if the experiment were repeated.

School: If an experiment were repeated, the specific schools in the experiment would change.

Organizational Unit: If the experiment were repeated the specific org units included would change.

Factors whose values change randomly from one experiment to the next are random factors.

To Summarize

A fixed factor is one whose values, e.g., male and female, would remain the same if the experiment were repeated.

A random factor is one whose values, e.g., the schools and their characteristics, would change randomly if the experiment were repeated.

L1 - 6

Multilevel modeling

Multilevel modeling is a collection of analyses designed to 1) to take into account the nesting of observations within randomly selected groups and 2) to take into account the randomness of the grouping variables typically found and3) to allow the analysis of the effects of group characteristics on the dependent variables.

It is also called . . .

hierarchical linear models (HLM), random coefficients models, mixed-effect models, multilevel regression models, multilevel covariance structure or multilevel structural equation models. (Text – p. 1).

It is a collection of techniques that requires the data analyst to

1. Understand multiple regression analysis.

2. Understand the use of group-coding variables to compare means of groups.

3. Understand interaction effects, i.e., moderation.

4. Understand repeated measures analysis and what they’re designed to accomplish.

After an undergraduate education and three semesters of graduate statistics, you’re now ready.

L1 - 7

Example of a cross sectional multilevel data file: ch3multilevel from the text’s data files.

It looks like a regular SPSS file and it is.

Individual Characteristics: Gender, ses, femses, and math are individual variables, outlined in red. In this data set math, representing Math Achievement, is the dependent variable. Its relationship to other variables, such as student SES is being investigated. FEMSES is a product variable. More on it later.

The variables whose values name the groups is schcode. This is what makes this file a multilevel file.

Group characteristics (outlined in blue) are . . .ses_mean = Mean SES of the students in a randomly selected school, a group characteristic,per4yrc = percentage of students in the school going on to a 4-year school, a group char. public = whether school is public or private, also a group characteristic.

Note that groups are not required to have characteristics in a multilevel analysis. If they don’t, the analysis should still take the grouping into account.

But if the groups do have characteristics, the beauty of multilevel analyses is that those characteristics can be used to possibly explain some of the variance in the dependent variable that would be unexplained in an analysis that did not acknowledge grouping.

L1 - 8

Here is the basic relationship in which researchers are interested . . .

As is apparent from the scatterplot, there is an anomaly in the data. For a couple hundred students, the Math value appears to have been “phoned in” in some way.

I wish the authors had provided a dataset with no anomalies. I’ve never had the time to try to identify the “flat line” data points and remove them from the dataset.

Moreover, eight students had math scores of 100. Really?

More on that later.

L1 - 9

Levels in multilevel analyses

The concept of “level” is an important concept associated with the analyses we’ll be conducting. It refers to the relationship to the grouping.

Cross-sectional Multilevel analyses

Level 1 – Values inside the groups.

The “lowest” level in multilevel analyses, usually containing individual persons, is called Level 1.

Level 1 characteristics are individual person characteristics. Values of these characteristics may vary from person to person both within a group and from person to person between different groups. So, the values of Level 1 variables are specific to the individual unit of analysis and cut across any groupings that might exist.

Level 2 – the groups.

The groups in which the level 1 individuals are nested, e.g., schools, are said to be at Level 2.

Level 2 characteristics are characteristics of the group, such as group size, for example.

The majority of cross-sectional multilevel analyses are comprised of only two levels.

Level 3, 4, etc. – groups of groups (Argh!!)

However, three and higher level analyses are possible. For example, in a cross-sectional analysis, individuals could be nested within classes; classes could be nested within schools; schools could be nested within school districts. In this example, individuals would be Level 1. Classes would be Level 2. Schools would be Level 3. School districts would be Level 4. We’ll focus on two-level data here – Level 1 and Level 2.

Longitudinal Multilevel analyses

Level 1: In a longitudinal analysis individual scores are at Level 1.

Level 2: The persons generating those scores are Level 2. In longitudinal analyses, characteristics of persons are Level 2 charateristics.

Level 3, 4, etc.: Groups of persons, if there are any, would be Level 3.

L1 - 10

Cross-sectional Level 2 characteristics: Composite vs. Pure

Remember that one characteristic of multilevel analyses is that the groups themselves may have characteristics that affect the dependent variable along with the Level 1 characteristics.

1. Level 2 characteristics that are composites of Level 1 characteristics for each group:

A composite of characteristics of individuals may be treated as a group characteristic.

In a cross-sectional design, Mean of SESs of individual students within a school can be used as a group characteristic. This is a composite – the arithmetic average of individual SESs.

In a cross-sectional design, the percentage of students going on to a 4-year college within a school can be used as a group characteristic. Again, this is a composite, a group percentage of individual students who had the individual characteristic of “going to a 4-year college”.

2. Level 2 characteristics that are pure group characteristics:

Pure group characteristics are characteristics of the group that are a property of the group entity only, not “built up” by combining scores of the elements.

Whether a school is public or private in a cross-sectional design, as in the above data set, is an example.

The number of years experience of the principal of the school is another example.Gross sales of an organization if organizations were groups.Debt rating of an organization if organizations were groups.

In longitudinal designs, in which each person is a “group”, a characteristic of the person would be a pure “group” characteristic.

L1 - 11

Conceptualizing the difference between cross-sectional and longitudinal analyses

In cross-sectional analyses, persons are grouped.

In longitudinal analyses, scores at different time periods are grouped within persons.

The first type of situation – cross sectional multilevel

The second type of situation – longitudinal multilevel

The key difference: Note that for the longitudinal situation, the order of observations within each group (person) is the same across persons – red -> green -> blue in the above example.

This is a difference between the cross sectional multilevel analyses and longitudinal multilevel analyses that we’ll consider and have to deal with when we cover longitudinal analyses.

Person 1 Person 2 Person 3

Persons: Level 1

Time periods: Level 1

Groups: Level 2

Persons: Level 2

Classroom A Classroom B Classroom C

L1 - 12

Examples of Research with level 1 and level 2 data – Cross-sectional only, not longitudinal

1. Bickel, p. 10. Assessing the relationship of income of the head of the household to race of the head of the household and education of the head of the household and to percentage of Blacks in the state and the state-level mean educational attainment.

Level 1 individual elements are heads of householdsLevel 1 DV INCOME Income of head of householdLevel 1 IV BLACK1 Whether head of household is Black or notLevel 1 IV EDUCATION1 Level of education of head of household

Level 2 group elements are StatesLevel 2 IV BLACK2 Percentage of Blacks in the state in which the household

resides. (Composite.)Level 2 IV EDUCATION2 Mean education level in the state in which the household.

(Also composite.)

2. Bickel p. 28. Assessing the relationship of child math achievement to child’s SES, whether the child is in a public or private school and mean SES of all the kids in the school. (Much like the Ch3multilevel.sav example. Hmm.)

Level 1 individual elements are individual childrenLevel 1 DV Math Achievement Child’s Math AchievementLevel 1 IV SES1 Child’s SESLevel 2 group elements are schoolsLevel 2 IV PRIVATE2 Whether child is in a public or private school. (Pure.)Level 2 IV SES2 Mean SES of kids in the child’s school. (A composite.)

3. Assessing the relationship of child reading achievement to income of the child’s family, quality of the individual child’s home neighborhood, amount of education of the parents of the child, child’s ethnicity, Head Start participation, mean vocabulary of the school in which groups of children reside, and quality of the neighborhood in which the school resides. NOTE: The assumption here is that neighborhood, parents, and family differed from child to child, making them all Level 1 characteristics.

Level 1 individual elements are individual children.Level 1 DV Vocab Achievement Child reading achievementLevel 1 IV INCOME1 Income level of individual child’s familyLevel 1 IV NEIGHBORHOOD1 Quality of individual child’s home neighborhoodLevel 1 IV ETHNICITY1 Individual child’s ethnicityLevel 1 IV HEADSTART1 Whether individual child participated in Head StartLevle 2 grouping elements are schoolsLevel 2 IV VOCABULARY2 Mean of vocabulary achievement levels of kids in the

school attended by the child. (A composite group characteristic.)

Level 2 IV NEIGHBORHOOD2 Quality of neighborhood in which the school resides (A pure group characteristic.)

L1 - 13

Review of Ordinary Least Squares (OLS) analyses.

Data: ch3multilevel.sav (analyzed without taking grouping into account)

Suppose our interest is on explanation of differences in math achievement – math in the above dataset.

First, we are interested in determining how math achievement of students is related to the student’s ses.

Later we’ll add some other factors that might be related to student math achievement.

L1 - 14

1. Simple Linear OLS Regression analysis.

First, a scatterplot, to identify possible problems . . .

The horizontal band is certainly a potential problem. We will stick our heads in the sand and pretend that we didn’t see that. The few outlying math values of 100 should be deleted. I won’t, in the belief that those values won’t materially affect the results based on several thousand observations.

SPSS OLS analysis using the SPSS REGRESSION procedure

GET FILE='G:\MDBO\html2\p5520\p5520 Data\ch3multilevel.sav'.

regression variables = math ses /sta = default zpp /dep=math /enter.

Regression[DataSet1] G:\MDBO\html2\p5520\p5520 Data\ch3multilevel.sav

Variables Entered/Removeda

Model Variables EnteredVariables Removed Method

1 sesb . Enter

a. Dependent Variable: mathb. All requested variables entered.

Model Summary

Model R R SquareAdjusted R

SquareStd. Error of the

Estimate1 .378a .143 .143 8.13220

a. Predictors: (Constant), ses

Coefficientsa

Model

Unstandardized Coefficients

Standardized Coefficients

t Sig.Correlations

B Std. Error Beta Zero-order Partial Part1 (Constant) 57.598 .098 586.608 .000

ses 4.255 .126 .378 33.858 .000 .378 .378 .378a. Dependent Variable: math

L1 - 15

The results show that we would expect a 4.255 increase in math achievement score for every one-unit difference in ses level. The relation is statistically significant.SPSS OLS analysis using the SPSS MIXED procedure

MIXED math WITH ses/FIXED = ses | SSTYPE(3)/METHOD = REML/PRINT = SOLUTION TESTCOV.

Mixed Model Analysis

Model Dimensiona

Number of LevelsNumber of Parameters

Fixed Effects Intercept 1 1ses 1 1

Residual 1

Total 2 3a. Dependent Variable: math.

Information Criteriaa

-2 Restricted Log Likelihood 48303.088Akaike's Information Criterion (AIC)

48305.088

Hurvich and Tsai's Criterion (AICC)

48305.088

Bozdogan's Criterion (CAIC) 48312.923Schwarz's Bayesian Criterion (BIC)

48311.923

The information criteria are displayed in smaller-is-better form.a. Dependent Variable: math.

Fixed EffectsType III Tests of Fixed Effectsa

Source Numerator df Denominator df F Sig.Intercept 1 6869 344109.335 .000

ses 1 6869 1146.382 .000

a. Dependent Variable: math.

Estimates of Fixed Effectsa

Parameter Estimate Std. Error df t Sig.95% Confidence Interval

Lower Bound Upper BoundIntercept 57.598169 .098188 6869 586.608 .000 57.405690 57.790649ses 4.254675 .125661 6869 33.858 .000 4.008340 4.501011a. Dependent Variable: math.Covariance Parameters

Estimates of Covariance Parametersa

Parameter Estimate Std. Error Wald Z Sig.95% Confidence Interval

Lower Bound Upper BoundResidual 66.132698 1.128456 58.605 .000 63.957541 68.381830a. Dependent Variable: math.

Technical information that will be ignored in most of what follows.

L1 - 16

The MIXED output is designed for multilevel analyses, but it can be used for regular OLS regression. It does not print R or R2!! It prints the variance of the residuals, rather than the standard deviation of the residuals.Simple OLS Regression in R

Run R.Load rcmdr package.Data -> Import data -> from SPSS dataset . . .

> library(foreign, pos=15)

> ch3multilevel <- + read.spss("G:/MDBO/html2/p5520/p5520 Data/ch3multilevel.sav", + use.value.labels=TRUE, max.value.labels=Inf, to.data.frame=TRUE)

> colnames(ch3multilevel) <- tolower(colnames(ch3multilevel))

This page was just devoted to getting the data into R.

L1 - 17

The OLS regression from within rcmdr

Statistics -> Fit Models -> Linear Regression

The output

> RegModel.1 <- lm(math~ses, data=ch3multilevel)

> summary(RegModel.1)

Call:lm(formula = math ~ ses, data = ch3multilevel)

Residuals: Min 1Q Median 3Q Max -31.459 -4.678 1.144 5.355 47.560

Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 57.59817 0.09819 586.61 <2e-16 ***ses 4.25468 0.12566 33.86 <2e-16 ***---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.132 on 6869 degrees of freedomMultiple R-squared: 0.143, Adjusted R-squared: 0.1429 F-statistic: 1146 on 1 and 6869 DF, p-value: < 2.2e-16

Note that R’s output gives the same values as does SPSS’s. Thankfully, there is only ONE God of Statistics.

These commands were sent to R by rcmdr. You could enter them at the R prompt.

L1 - 18

Adding predictors

Suppose we wish to know whether math achievement is related to whether the school is public or private in addition to its relation to individual student ses.

This creates a multiple regression OLS regression problem.

L1 - 19

In SPSS

regression variables = math ses public /sta=default zpp /dep=math /enter.

Regression


Model Variables EnteredVariables Removed Method

1 public, sesb . Enter

a. Dependent Variable: mathb. All requested variables entered.

Model Summary



Estimate1 .378a .143 .143 8.13268

a. Predictors: (Constant), public, ses

ANOVAa

Model Sum of Squares df Mean Square F Sig.1 Regression 75826.239 2 37913.120 573.221 .000b

Residual 454252.577 6868 66.140

Total 530078.816 6870

a. Dependent Variable: mathb. Predictors: (Constant), public, ses

Coefficientsa

Model



t Sig.Correlations


ses 4.257 .126 .378 33.846 .000 .378 .378 .378public -.098 .221 -.005 -.442 .658 .011 -.005 -.005

a. Dependent Variable: math

So, among students equal in ses, there was no difference in math achievement between those in public and those in private schools.

L1 - 20

In R

Statistics -> Fit Models -> Linear Regression

> RegModel.2 <- lm(math~public+ses, data=ch3multilevel)

> summary(RegModel.2)

Call:lm(formula = math ~ public + ses, data = ch3multilevel)


Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 57.66958 0.18905 305.053 <2e-16 ***public -0.09784 0.22134 -0.442 0.658 ses 4.25696 0.12577 33.846 <2e-16 ***---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.133 on 6868 degrees of freedomMultiple R-squared: 0.143, Adjusted R-squared: 0.1428 F-statistic: 573.2 on 2 and 6868 DF, p-value: < 2.2e-16

So, R gives the same results as SPSS – no significant difference in math achievement between public and private school students holding ses constant.

L1 - 21

Does public/private moderate the ses -> Math relation?

In SPSScompute pxs = public * ses.regression variables = math ses public pxs /sta=default zpp /dep=math /enter.


ModelVariables Entered

Variables Removed Method

1 pxs, public, sesb . Entera. Dependent Variable: mathb. All requested variables entered.

Model Summary



Estimate1 .379a .143 .143 8.13121a. Predictors: (Constant), pxs, public, ses

Coefficientsa

Model



t Sig.Correlations


ses 4.647 .244 .413 19.045 .000 .378 .224 .213public -.099 .221 -.005 -.446 .656 .011 -.005 -.005pxs -.531 .285 -.040 -1.866 .062 .313 -.023 -.021

a. Dependent Variable: math

The relation is slightly steeper for private (O) than for public (O) school students.

L1 - 22

In rcmdr

Statistics -> Fit Models -> Linear Model . . .

> LinearModel.3 <- lm(math ~ public + ses +public*ses, data=ch3multilevel)

> summary(LinearModel.3)

Call:lm(formula = math ~ public + ses + public * ses, data = ch3multilevel)


Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 57.67770 0.18906 305.070 <2e-16 ***public -0.09872 0.22130 -0.446 0.656 ses 4.64728 0.24402 19.045 <2e-16 ***public:ses -0.53146 0.28474 -1.866 0.062 . ---Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.131 on 6867 degrees of freedomMultiple R-squared: 0.1435, Adjusted R-squared: 0.1431 F-statistic: 383.4 on 3 and 6867 DF, p-value: < 2.2e-16

Note that when using the “Linear Model” procedure in rcmdr, you are allowed to enter product terms to represent interactions.

L1 - 23

Getting a plot in rcmdr

Graphs -> Scatterplot . . .

The variable, publicfac, was created as a factor from the variable, public.

Data -> Manage variables in active dataset -> Convert numeric variables to factors

L1 - 24

In conclusion, the relation of math achievement to ses was slightly steeper for private school students than it was for public school students.

€¦ · Web view1. Cross-sectional m ultilevel s ituations. Suppose you’re interested in the...

Documents

Transcript of €¦ · Web view1. Cross-sectional m ultilevel s ituations. Suppose you’re interested in the...