SEM Model Fit:Introduction

David A. Kenny

January 12, 2014

2

DefinitionFit refers to the ability of a model to reproduce the data (i.e., usually the variance-covariance matrix). 

A good-fitting model is one that is reasonably consistent with the data and so does not necessarily require any major respecification. 

A good-fitting measurement model is required before interpreting the causal paths of the structural model.

3

However…

• A good fitting model does not necessarily mean a good model.

• For instance, a model all of whose parameters are zero is likely a "good-fitting" model.

• Additionally, models with nonsensical results (e.g., paths that are clearly the wrong sign) and models with poor discriminant validity or Heywood cases can be “good-fitting” models.  Parameter estimates must be carefully examined to determine if one has a reasonable model as well as the fit statistics.

4

Additionally• A good fitting model can still be

improved.

• You can make it a better fitting model.

• Also too you can make the model simpler and not appreciably change the fit.

• A good fitting model is not necessarily the best model.

5

How Big a Sample Size?

• Rules of Thumb

–Ratio of N to the Number of Free Parameters

–Absolute N• Power Analysis

         

6

Ratio of N to Number of Free Parameters

• Tanaka (1987): 20 to 1, but that is unrealistically high.

• Bentler & Chou (1987): 5 to 1

• Many published studies fail to meet this goal!

7

Absolute N• 200 is seen as a goal for SEM research• Lower sample sizes can be used for

– Models with no latent variables– Models where all loadings are fixed (usually to

one)– Models with strong correlations– Simpler models – Models for which there is an upper limit on N

(e.g., countries or years as the unit)

8

Power Analysis• The best way to determine if you have a

large enough sample is to conduct a power analysis.

• Either use the Sattora and Saris (1985) method or conduct a simulation.

• To determine the power to detect a poor fitting model, you can use Preacher and Coffman’s web-based calculator.

9

2 Test• For models with about 75 to 200 cases, the chi square

test is a reasonable measure of fit.  But for models with more cases (e.g., 400 or more), the 2 test is almost always statistically significant. 

• This is why fit indices were invented. They provide a way to claim that one has a good model, despite the fact that the 2 test is statistically significant.

• Sometimes chi square is more interpretable if it is transformed into a Z value.  The following approximation can be used:

Z =  √(2χ2) - √(2df - 1)

10

More on 2

An old measure of fit is the chi square to df ratio or 2/df.  A problem with this fit index is that there is no universally agreed upon standard as to what is a good- and a bad-fitting model.  Note, however, that two very popular fit indices,  TLI and RMSEA, are largely based on this old-fashioned ratio.

The chi square test is too liberal (i.e., too many Type 1) errors when variables have non-normal distributions, especially distributions with kurtosis.  Moreover, with small sample sizes, there are too many Type I errors. Note the 2 test is asymptotic test, and so it works best with large sample sizes.

11

Typology of Fit Indices

• This typology is very different from the mainstream definitions. Please note differences.

• Typology–Incremental–Absolute–Comparative

12

Incremental Fit IndicesAn incremental (sometimes called

relative) fit index is analogous to R2 , and so a value of zero indicates having the worst possible model and a value of one indicates having the best possible. 

So my model is placed on a continuum. In terms of a formula, it is

        Worst Possible Model – My Model           Worst Possible Model – Best Possible Model

13

The Best Model• The standard definition of best possible

model is one in which equaled it degrees of freedom (the expected value of chi-square given the null hypothesis of perfect fit).

• An older definition (Bentler-Bonnet) is assume the best model has a chi square of zero, but that definition ignores sampling error.

14

The Worst Model• The worst possible model is called the null or

independence model and the usual convention is to allow all the variables in the model to have variation but no correlation. 

• The usual null model is to allow the means to equal their actual value.  However, for growth curve models, the null model should set the means as equal, i.e., no growth. 

15

df of the Null Model• The degrees of freedom of the null model

are k(k – 1)/2 where k is the number of variables in the model.

• If the null model sets the means equal, as in a growth-curve model, its df are (k + 2)(k – 1)/2.

16

Alternative Null Models• Alternative null models might be considered (but

almost never done).  • One alternative null model is that all latent variable

correlations are zero and another is that all exogenous variables are correlated but the endogenous variables are uncorrelated with each other and the exogenous variables. 

• O’Boyle and Williams (2011) suggest two different null models for the measurement and structural models.

17

Absolute Fit Indices

• An absolute measure of fit presumes that the best fitting model has a fit of zero.  The measure of fit then determines how far the model is from perfect fit. 

• These measures of fit are typically “badness” measure of fit in that a bigger number implies worse fit.

• Common absolute fit indices are SRMR and RMSEA.

18

Comparative Fit Indices• A comparative measure of fit is only interpretable

when comparing two different models.  This term is unique to this PowerPoint in that these measures are more commonly called absolute fit indices.  However, it is helpful to distinguish absolute indices that do not require a comparison between two models. 

• One advantage of comparative fit indices is that often they can be computed for models that are just-identified.

• Examples are AIC, BIC, and SABIC.  

19

Controversy about Fit Indices

• There is considerable controversy about fit indices. 

• Some researchers do not believe that fit indices add anything to the analysis (e.g., Barrett, 2007) and only the chi square should be interpreted.  The worry is that fit indices allow researchers to claim that a miss-specified model is not a bad model. 

• Others (e.g., Hayduk, Cummings, Boadu, Pazderka-Robinson, & Boulianne, 2007) argue that cutoffs for a fit index can be misleading and subject to misuse. 

20

Consensus View• Most analysts believe in the value of fit indices,

but caution against strict reliance on cutoffs.  • Particularly, problematic is the “cherry picking”

a fit index.  That is, you compute many fit indices and you pick the one index that allows you to make the point that you want to make.  If you decide not to report a popular index (e.g., the TLI or the RMSEA), you need to give a good reason why you are not.

21

Compute a Fit Index?• Kenny, Kaniskan, and McCoach (2014) have

argued that fit indices should not even be computed for small degrees of freedom models. 

• Rather for these models, the researcher should locate the source of specification error by determining what parameter should be added to the model and then test that parameter.

22

Additional Powerpoints

• Standard measures of fit

–CFI and TLI

–RMSEA

–SRMR

• Non-standard measures of fit

23

References

Click here to go to references.