Quasi-experimental Quasi-experimental Designs & SurveyDesigns & Survey
ResearchResearch
Hedrick, Bickman & Rog (1993) “a Hedrick, Bickman & Rog (1993) “a quasi-experimental design is not the quasi-experimental design is not the method of choice, but rather a fallback method of choice, but rather a fallback strategy”strategy”
Cannot infer cause and effect, Cannot infer cause and effect, BUTBUT well well designed quasi-experiments enable designed quasi-experiments enable you to demonstrate that rival you to demonstrate that rival interpretations are rendered unlikelyinterpretations are rendered unlikely
Quasi-experimentsQuasi-experiments
Non-equivalent Group DesignsNon-equivalent Group Designs
Posttest only nonequivalent control group Posttest only nonequivalent control group design (aka static group comparison)design (aka static group comparison)
X O X O (treatment grp)(treatment grp) O O (nonequivalent control grp)(nonequivalent control grp)
X=treatmentX=treatment O=measurement/observation O=measurement/observation
Because there is no random assignment to Because there is no random assignment to groups, confounding variables may explain groups, confounding variables may explain any difference observedany difference observed
Pre-test/Post-test Non-equivalent Pre-test/Post-test Non-equivalent Control GroupControl Group
O X O treatment group
O O non-equivalent control group
Pre Post
Example: Research Methods Example: Research Methods and Reasoning Abilityand Reasoning Ability
0
1
2
3
4
5
Pre Post
Mea
n R
easo
nin
g S
core
Develop Methods
Intervention: critical thinking seminar
Research Methods students receive the intervention (i.e., participate in the critical thinking seminar)
Developmental Psychology students are used as a nonequivalent control group
(i.e., do not attend the seminar)
Interrupted Time Series DesignsInterrupted Time Series Designs
Extension of the simple one group pre Extension of the simple one group pre
and post designand post design
Participants are pre-tested a number of Participants are pre-tested a number of
times and then post-tested a number of times and then post-tested a number of
times after being exposed to the times after being exposed to the
treatment interventiontreatment intervention
OO11 O O2 2 OO33 O O44 X O X O5 5 O O66 O O77 O O88
Useful when: Useful when: cannot randomize participantscannot randomize participants
it is possible to obtain a series of it is possible to obtain a series of
assessments of the DV before and after assessments of the DV before and after
treatmenttreatment
Interrupted Time Series DesignsInterrupted Time Series Designs
Example: Interrupted Time Series Example: Interrupted Time Series DesignsDesigns
Intervention: Course to change students’ study habits, implemented during the summer (after semester 4).
DV: semester GPA0
0.5
1
1.5
2
2.5
3
3.5
4
1 2 3 4 5 6 7 8
Mea
n G
PA
X
Baseline
Post-tx
How Many Measurements Are How Many Measurements Are Needed for a Time Series Design?Needed for a Time Series Design?
Depends on:Depends on: amount of random fluctuation (noise) that amount of random fluctuation (noise) that
may occur in the outcome being measuredmay occur in the outcome being measured
how much of an impact the intervention is how much of an impact the intervention is
expected to haveexpected to have
Multiple Time Series DesignsMultiple Time Series Designs Add a comparison group to the simple interrupted time Add a comparison group to the simple interrupted time
series design:series design:
OO11 O O2 2 OO33 O O44 X O X O5 5 O O66 O O77 O O88
------------------------------------------------------------------------------------------------------------------------------------
OO11 O O2 2 OO33 O O44 O O5 5 O O66 O O77 O O88
Campbell & Stanley consider this an excellent quasi-Campbell & Stanley consider this an excellent quasi-experimental designexperimental design
Survey Research Survey Research
SurveysSurveys
Items are composed of 2 parts: stem & Items are composed of 2 parts: stem & response optionsresponse optionsThe university should not raise tuition under any The university should not raise tuition under any
circumstances circumstances ((stemstem)) Agree Agree DisagreeDisagree ((response optionsresponse options))
Question WordingQuestion Wording Important consideration when writing Important consideration when writing
questions:questions: SimplicitySimplicity Avoid jargon and technical termsAvoid jargon and technical terms Avoid abbreviationsAvoid abbreviations Be specificBe specific Avoid double-barreled questions -Avoid double-barreled questions -
Responses to QuestionsResponses to Questions Open-endedOpen-ended
Closed-endedClosed-ended
“…closed questions should be used where alternative replies are known, are limited in number, and are clear cut. Open ended questions are used where the issue is complex, where relevant dimensions are not known, and where process is being explored.”
Stacey (1969)
Open vs. Closed QuestionsOpen vs. Closed Questions
Rating ScalesRating Scales
Likert Scales Likert Scales Semantic DifferentialSemantic Differential Graphic Rating ScaleGraphic Rating Scale Non-verbal scaleNon-verbal scale
Question PlacementQuestion Placement
Very importantVery important
Confusing questionnaire = Confusing questionnaire = demoralizing to respondentsdemoralizing to respondents
Can affect people’s responses to Can affect people’s responses to themthem
Layout of QuestionnaireLayout of Questionnaire
Use closed-ended questionsUse closed-ended questions
Use a consistent item formatUse a consistent item format
Don’t crowd questions togetherDon’t crowd questions together
Don’t split questions or response options between Don’t split questions or response options between pagespages
It is most likely to be completed if easy on the eye, It is most likely to be completed if easy on the eye, relevant, logical and as short as possible.relevant, logical and as short as possible.
Response BiasesResponse Biases Social desirabilitySocial desirability Acquiescence response - tendency either to Acquiescence response - tendency either to
agree/disagree with statementsagree/disagree with statements Yea-SayingYea-Saying Nay-SayingNay-Saying
CChecklisthecklist for for SSurvey urvey UUsese1.1. Do the items provide a representative sample Do the items provide a representative sample
of the behaviour under study?of the behaviour under study?
2.2. Do the items ask for irrelevant information?Do the items ask for irrelevant information?
3.3. Does the question have the same meaning for Does the question have the same meaning for
researcher and respondent?researcher and respondent?
4.4. Is the question worded clearly?Is the question worded clearly? Are the words Are the words
too difficult?too difficult?
5.5. Does the question concern a sensitive issue?Does the question concern a sensitive issue?
CChecklisthecklist for for SSurvey urvey UUsese
6.6. Does the question suggest a socially Does the question suggest a socially desirable answer?desirable answer?
7.7. Open vs closed questions?Open vs closed questions?
8.8. How many questions are necessary to How many questions are necessary to obtain the required information?obtain the required information?
9.9. How might the questions be arranged to How might the questions be arranged to encourage the respondent’s encourage the respondent’s participation?participation?
Top Related