CRICOS No. 00213J a university for the world real R Reliable and valid survey design for structural...

CRICOS No. 00213Ja university for the worldrealR

Reliable and valid survey design for structural equation modelling

Guanglun (Michael) MuCenter for Learning Innovation, QUT Faculty of Education Supervisory team:Dr. Karen Dooley, Paul Shield & Dr. Catherine Doherty

16 February, 2011


• RQ1. Does Chinese Australians’ “Chineseness” have positive impacts on their CHL proficiency through the investment of their various resources?

• RQ2. Does Chinese Australians’ CHL proficiency positively influence their “Chineseness” and further, influence their embodied cultural capital in the CHL field?

Research questions


Paradigm / Worldview: Post-positivism

Theoretical lens: Structuralism

Methodological approach: Quantitative research

Research method: Survey design

Population SamplingInstrument Analysis

Overview of research design


Population: Chinese Australian urban young adults

Target population:

Chinese Australian urban young adults, living in Sydney, Melbourne, and

Brisbane, ranging in age from 18 to 30

Sample

• Why “Sydney, Melbourne, and Brisbane”?• Why “18-30”?

• Why “urban”?• Why “young adults”?


• Online questionnaire Key Survey• Measurement scale Nominal, ordinal, interval & ratio 7-point Likert scale as interval scale• Operationalization Validity: Accuracy Face validity & Content validity Reliability: Consistency Test-retest reliability & Inter-item reliability • Strategies

Instrument development: Questionnaire design


• ‘Latent variable’ here refers to a variable that is not directly measurable but related to several variables that can be measured directly (Field, 2005).

• These directly measurable variables are called ‘indicators’ or ‘measurements’ and can be operationalized through corresponding items in a questionnaire (Field, 2005).

Theoretical concepts as latent variables

Construct items as indicators

Operationalization


• Use filtering techniques

• Offer enough variance

• Understand whose opinions are under scrutiny

• Avoid double barreled questions

• Restate negatively worded questions

• Language

• Team work

Questionnaire design strategies


• Except Australia and your country of origin, have you ever lived in any other countries?

• Have you ever formally studied Chinese?

• Where were you born?

Participants will be filtered towards branching or contingency questions applicable only to them (Creswell, 2008).

Branching or contingency questions


Handwriting tells you a lot about a person’s character.

It is really important to get as much academic education as possible for one’s career.

Compared to other families, mine is very wealthy.

I feel completely comfortable at Chinese cultural events.

I always go to venues, such as libraries, galleries, museums, theatres, and concerts, if they feature Chinese culture.

I tend to mix exclusively with Chinese social groups.

I can read Chinese newspapers and magazines easily.

I can describe my past experiences in detail in Chinese.

Children must be taught to paint, dance, or play musical instruments.

Participants are “pushed” and split to select different responses.


Family members should live close to one another.

I always prefer to live close to my family members.

People always prefer to say yes to save face.

To save face I always prefer to say yes.

I am very popular among my Chinese peers.

I am considered very popular among my Chinese peers.

The answers to a question depend on, to a certain extent, whose opinions are under scrutiny.


I always keep up to date with current Chinese affairs by watching TV, listening to the radio, reading newspaper, and surfing online.

I always keep up to date with current Chinese affairs by watching TV, listening to the radio, reading newspaper, or surfing online.

I am well regarded because of my Chinese language competency.

My Chinese competency is well regarded.

Double barreled questions actually measure two things within one item.


I won’t have good time if I don’t celebrate Chinese festivals.

I will have good time if I celebrate Chinese festivals.

I feel very sad if I miss Chinese festivals.

Negatives in a statement may lead to errors in responses.


Pilot study: Reliability and validity of the instrument

• Sample size

When significance level (α) and power value were set at .05 and .80 with a medium effect size of .30 as appropriate for a wide range of behavioral research (Hair, Black, Babin, Anderson, & Tatham, 2006), G-Power gave a sample size of approximately 60 to satisfy the Cronbach α test.

• Sampling

Convenience sampling


Main study

• Sample size Although there is no single criterion that dictates the necessary

sample size for Structural Equation Model, the sample size can be five times the number of indicators but no less than 200 in order to provide reliable results (Bentler & Chou, 1987). 46×5=230

• Sampling

Snowball sampling (why not random sampling, systematic sampling, stratified sampling, cluster sampling?)

• Data analysis

Structural Equation Modelling


Key features of structural equation modelling

• SEM is theory driven.

• SEM is capable to assess how questionnaire items define their respective theoretical concept.

• SEM improves statistical estimation by accounting for measurement error.

• SEM is able to examine complex interdependence amongst theoretical concepts.


I1 I2 I3 …

e e e

Symbolic capital

I1 I2 …

e e e

Social capital

I1 I2 I3 …

e e e

Cultural capital

I1 I2 I3 …

e e e

Economic capital

I1 I2 I3 …

e e e

CHL proficiency

I1 I2 I3 …

e e e

Chineseness


I1 I2 I3 …

e e e

CHL proficiency

I1 I2 I3 …

e e e

Chineseness

I1 I2 I3 …

e e e

Cultural capital

CRICOS No. 00213J a university for the world real R Reliable and valid survey design for structural...

Documents

Transcript of CRICOS No. 00213J a university for the world real R Reliable and valid survey design for structural...