R Package nlsem - uni-bielefeld.de · 2016-05-31 · R Package nlsem Nora Umbach 1, Katharina...

R Package nlsem

Nora Umbach 1, Katharina Naumann 1, David Hoppe 2, HolgerBrandt 1, Augustin Kelava 1, Bernhard Schmitz 2

1Hector Research Institute, University of Tubingen, Germany

2Technische Universitat Darmstadt, Germany

February 27, 2015

Introduction Model foundation of nlsem Usage of package Outlook

Outline

IntroductionLinear SEMNonlinear SEMIntegration of parametric and semiparametric approaches

Model foundation of nlsem

Usage of packageSpecify modelsEstimate modelsExample

Outlook

2 | Nora Umbach





Outlook

3 | Nora Umbach


What is the nlsem package for?

... estimating Structural Equation Models with nonlinear relationships(e. g., parametric interaction effects) between latent variables

4 | Nora Umbach


Structural Equation Models (SEM)

general case

I measurement model

y = νy + λyη + ε

x = νx + λxξ + δ

I structural model

Bη = α+ Γξ + ζ

simple example

y1

y2

y3

η

ξ1

x1

x2

x3

ξ2

x4

x5

x6

γ11

γ12

η = α + γ11ξ1 + γ12ξ2 + ζ

5 | Nora Umbach


Two classes of models for nonlinear relationships in SEM

1. parametric modelsI product indicator approachesI distribution analytic approachesI bayesian approachesI method of moments approaches

2. semiparametric modelsI structural equation mixture modelsI nonlinear structural equation mixture models

6 | Nora Umbach


1. Parametric models: distribution analytic approaches

Latent Moderated Structural Equations approach(LMS; Klein & Moosbrugger, 2000)

I structural model

η = α + Γξ + ξ′Ωξ + ζ

I variable distributions of parametric models

ξ ∼ N (µξ,Φ), ζ ∼ N (0,Ψ)

I η is nonnormally distributed (becauseof the latent product variables)

I therefore y is also nonnormallydistributed

η = α + γ11ξ1 + γ12ξ2 + ω12ξ1ξ2

+ ω11ξ21 + ω22ξ

22 + ζ

y1

y2

y3

η

ξ1

x1

x2

x3

ξ2

x4

x5

x6

γ11

γ12

ξ1ξ2

ω12

ξ21

ω11

ξ22

ω22

7 | Nora Umbach


2. Semiparametric Models: Structural Equation MixtureModels (SEMM)(Bauer, 2005; Jedidi, Jagpal, & DeSarbo, 1997)

I simultaneous approximization of nonlinearity and nonnormality in thelatent variables ...

I ... by approximating the distribution of their indicators by a mixtureof normal distributions(

xy

)∼

G∑g=1

ωgN(µg ,Σg )

I mixture components can be interpreted as subgroups in the sampleI structural model (SEM with G latent classes)

Bgηg = αg + Γgξg + ζg

I no parameters for the nonlinear effects

8 | Nora Umbach


Integration of parametric and semiparametric approaches:NSEMM

Nonlinear Structural Equation Mixture Model approach(NSEMM; Kelava, Nagengast, & Brandt, 2014)

I structural model

ηg = Bgηg + αg + Γ1gξg + Γ2gh(ξg ) + ζg

I nonlinear effects in the structural model

I ξ ∼∑G

g=1 ωgN (µξg ,Φξg )

I accounts for nonnormality of the latent and observed variables

9 | Nora Umbach





Outlook

10 | Nora Umbach



measurement model structural model assumptions

LMSy = νy + λyη + ε

η = α + Γξ + ξ′Ωξ + ζnonnormality of η

x = νx + λxξ + δ ξ ∼ N (µξ,Φ)

SEMMy|g = νg

y + λgyη

g + εg Bgηg = αg + Γgξg + ζg

nonnormalityx|g = νg

x + λgxξ

g + δg of η’s and ξ’s

NSEMMy|g = νg

y + λgyη

g + εg ηg = αg + Γ1gξg nonnormalityx|g = νg

x + λgxξ

g + δg +Γ2gh(ξg ) + ζg of η’s and ξ’s

11 | Nora Umbach





Outlook

12 | Nora Umbach


specify sem

I specify model matrices

I arguments (defaults)

R> model <- specify_sem(

num.x, (numeric)

num.y, (numeric)

num.xi, (numeric)

num.eta, (numeric)

xi, (character)

eta, (character)

num.classes=1, (numeric)

interaction="none", (character)

interc.obs=TRUE, (boolean)

interc.lat=TRUE, (boolean)

relation.lat="default" (character)

)

13 | Nora Umbach


specify sem example

R> model <- specify_sem(

num.x=6,

num.y=3,

num.xi=2,

num.eta=1,

xi="x1-x3,x4-x6",

eta="y1-y3",

num.classes=2,

interaction="xi1:xi1,

xi1:xi2,

xi2:xi2",

interc.obs=TRUE,

interc.lat=TRUE,

relation.lat="eta1~xi1+xi2"

)

y1

y2

y3

η

ξ1

x1

x2

x3

ξ2

x4

x5

x6

γ11

γ12

ξ1ξ2

ω12

ξ21

ω11

ξ22

ω22

14 | Nora Umbach


relation.lat exampley1 y2 y3 y4

η1 η2

ξ1

x1 x2

ξ2

x3 x4

ξ3

x6x5

relation.lat="eta1~xi1+xi2, eta2~xi3, eta2~eta1"

structural model:

Bη = α + Γξ + ζ(1 0β21 1

)(η1η2

)=

(α1

α2

)+

(γ11 γ12 00 0 γ23

)ξ1ξ2ξ3

+

(ζ1ζ2

)15 | Nora Umbach


specify sem example

library(nlsem)

# LMS

R> model1 <- specify_sem(num.x=4, num.y=1, num.xi=2, num.eta=1,

xi="x1-x2,x3-x4", eta="y1", num.classes=1,

interaction="xi1:xi2", interc.obs=FALSE)

R> class(model1)

[1] "lms"

# SEMM


xi="x1-x3,x4-x6", eta="y1-y3,y4-y6", num.classes=2,

interaction="none")

R> class(model2)

[1] "semm"

# NSEMM


xi="x1-x3,x4-x6", eta="y1-y3", num.classes=2,

interaction="xi1:xi1, xi1:xi2, xi2:xi2")

R> class(model3)

[1] "nsemm"

16 | Nora Umbach


em

I estimating models with Expectation-Maximization algorithm (EM)

I depending on the model’s class, EM for that class is executed

I arguments (defaults)

R> est1 <- em(

model, (’lms’,’semm’,’nsemm’)

data, (matrix)

start, (numeric)

logger=TRUE, (boolean)

convergence=0.01, (numeric)

max.iter=100, (numeric)

m=16, (numeric)

optimizer=c("nlminb", "optim"), (’nlminb’,’optim’)

max.mstep=1, (numeric)

neg.hessian=TRUE, (boolean)

... (optimizer arguments)

)

17 | Nora Umbach


Example

I load dataset

data(PoliticalDemocracy, package = "lavaan")

dat <- as.matrix(PoliticalDemocracy[ ,c(9:11,1:8)])

I specify model of class SEMM

model <- specify_sem(num.x = 3, num.y = 8, num.xi = 1, num.eta = 2,

xi = "x1-x3", eta = "y1-y4,y5-y8",

relation.lat = "eta1~xi1,eta2~xi1,eta2~eta1",

num.classes = 2)

I fit model

set.seed(911)

start <- runif(count_free_parameters(model))

res <- em(model, dat, start, convergence = 0.1, max.iter = 200)

18 | Nora Umbach


Example: emR> res <- em(model1, dat, pars.start)

-----------------------------------

Starting EM-algorithm for semm

Convergence: 0.1

-----------------------------------

-----------------------------------

Iteration 1

Doing expectation-step

Class weights: 0.079 0.921

Doing maximization-step

Results of maximization

Loglikelihood: -4344.16

Convergence: 1

Number of iterations: 1

-----------------------------------

Iteration 2

...

Doing maximization-step

Results of maximization

Loglikelihood: -862.168

Convergence: 1

Number of iterations: 1

-----------------------------------

-----------------------------------

EM completed

-----------------------------------

-----------------------------------

Computing negative Hessian

-----------------------------------

19 | Nora Umbach


Example: summary I

R> summary(res)

Summary for model of class semm

Estimates:

Estimate Std. Error t value Pr(>|z|)

class1.Lambda.x2 0.97142 0.04858 19.997 < 2e-16 ***

class1.Lambda.x3 0.94432 0.09330 10.121 < 2e-16 ***

class1.Lambda.y2 1.00738 0.28924 3.483 0.000496 ***

class1.Lambda.y3 1.09139 0.20883 5.226 1.73e-07 ***

class1.Lambda.y4 1.02985 0.23604 4.363 1.28e-05 ***

class1.Lambda.y14 0.96315 0.26402 3.648 0.000264 ***

class1.Lambda.y15 1.04334 0.20545 5.078 3.81e-07 ***

class1.Lambda.y16 1.09537 0.22512 4.866 1.14e-06 ***

class1.Gamma1 1.06047 NA NA NA

class1.Gamma2 0.81969 0.15491 5.291 1.21e-07 ***

class1.Beta2 -0.26362 NA NA NA

class1.Theta.d1 1.18205 NA NA NA

class1.Theta.d5 0.06568 0.12121 0.542 0.587900

class1.Theta.d9 0.19616 0.13058 1.502 0.133047

class1.Theta.e1 1.20361 0.43391 2.774 0.005540 **

...

class2.alpha1 0.69934 NA NA NA

class2.alpha2 -0.03629 0.47821 -0.076 0.939503

class2.tau 4.57297 0.23537 19.429 < 2e-16 ***

---

Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1

20 | Nora Umbach


Example: summary IINumber of iterations: 163

Final loglikelihood: -862.168

Final weights: 0.377 0.623

Likelihoods:

loglikelihood difference relative change

1 -4344.160 0.00000e+00 0.00000

2 -2467.248 1.87691e+03 0.43205

3 -1971.991 4.95257e+02 0.20073

4 -1703.436 2.68555e+02 0.13618

5 -1530.465 1.72972e+02 0.10154

6 -1413.241 1.17224e+02 0.07659

7 -1344.222 6.90190e+01 0.04884

8 -1300.677 4.35449e+01 0.03239

9 -1268.044 3.26329e+01 0.02509

10 -1242.807 2.52367e+01 0.01990

11 -1220.141 2.26668e+01 0.01824

...

156 -864.128 2.69674e-01 0.00031

157 -863.842 2.85736e-01 0.00033

158 -863.554 2.88017e-01 0.00033

159 -863.244 3.10405e-01 0.00036

160 -862.932 3.12287e-01 0.00036

161 -862.587 3.44222e-01 0.00040

162 -862.242 3.45633e-01 0.00040

163 -862.168 7.39285e-02 0.00009

21 | Nora Umbach


Example: extractor functions

R> plot(res)

Number of iterations

log

likel

ihoo

d

1 9 18 28 38 48 58 68 78 88 98 110 123 136 149 162

-4000

-3500

-3000

-2500

-2000

-1500

-1000

R> coef(res)

...

R> logLik(res)

’log Lik.’ -862.1679 (df=74)

R> AIC(res)

[1] 1728.336

R> BIC(res)

[1] 2043.83

R> anova(res, res2)

# only for two nested LMS models

22 | Nora Umbach





Outlook

23 | Nora Umbach


Outlook

I improve speed

I run more test simulations

I more than one η for LMS and NSEMM

I compute standard errors for NSEMM

I standardization of parameter estimates

I constraining parameters across classes (indirect approach)(e. g., Ψc1 = Ψc2 for two classes)

I Quasi-Maximum Likelihood approach (Klein & Muthen, 2007)

Version 0.2: feedback and bug reports are highly appreciated

24 | Nora Umbach

Thank you References

Thank you!

Questions?

[email protected]

http://CRAN.r-project.org/package=nlsem

25 | Nora Umbach

Thank you References

References

Bauer, D. J. (2005). A semiparametric approach to modeling nonlinearrelations among latent variables. Structural Equation Modeling, 12,513–535.

Jedidi, K., Jagpal, H. S., & DeSarbo, W. S. (1997). STEMM: A generalfinite mixture structural equation model. Journal of Classification,14, 23–50.

Kelava, A., Nagengast, B., & Brandt, H. (2014). A nonlinear structuralequation mixture modeling approach for nonnormally distributedlatent predictor variables. Structural Equation Modeling: AMultidisciplinary Journal, 21, 468-481.

Klein, A., & Moosbrugger, H. (2000). Maximum likelihood estimation oflatent interaction effects with the LMS method. Psychometrika, 65,457–474.

Klein, A., & Muthen, B. O. (2007). Quasi maximum likelihood estimationof structural equation models with multiple interaction andquadratic effects. Multivariate Behavioral Research, 42, 647–674.

26 | Nora Umbach

R Package nlsem - uni-bielefeld.de · 2016-05-31 · R Package nlsem Nora Umbach 1, Katharina...

Documents

Transcript of R Package nlsem - uni-bielefeld.de · 2016-05-31 · R Package nlsem Nora Umbach 1, Katharina...