Causal Designs

Chapter 9

Understanding when (and why)X Y

Theories and Hypotheses

• Theory

• A body of interconnected propositions about how a phenomenon works (recall animosity model)

• Hypothesis Testing

• Null (Dull) Hypothesis (Ho): • Nothing interesting is going on• Any differences we are observing are completely due to chance

• Alternative Hypothesis (H1)• Something interesting is going on• Differences in DV are due to IV

Experiments: Some Key Definitions• Independent Variable (X, the cause, the predictor)

• The variable you “manipulate” (good vs. bad aroma in store)

• Dependent Variable (Y, the outcome, the criterion)• What happens after you manipulate the IV (sales of a product)

• Control variables• Variables that you don’t allow to vary along with the IV• If any variable covaries with the IV, then there is a confound (e.g., if

music systematically varies along with aroma, then you can’t tell if it’s the aroma or the music that influences sales)

• Extraneous variables (or noise)• “Stuff happens” during an experiment, but it evens out across the levels

of the independent variable (e.g., different music at different times, but it doesn’t systematically vary by the level of the IV)

Factorial Designs• Factorial Design

• When the researcher is examining the impact of two independent (or predictor) variables on a DV

• Can have two main effects (overall impact of each IV) and an interaction (combined effect of two IVs)

No ServiceFailure

ServiceFailure Row Means

Low Hostility 2 3 2.5High Hostility 2 5 3.5

Column Means 2 4

Main Effect ofHostility

Main Effect ofService Failure

Interaction (Moderation)

Moderation• Moderation (under what conditions is a relationship stronger/weaker)

• When the effect of one IV (service failure) on the DV (negative word of mouth) depends on the level of another IV (trait hostility)

Service Failure NWOM

TraitHostility

Mediation• Mediation (like a combination shot in pool)

• When the effect of one IV on the DV occurs through an “intermediary” variable (think cue ball hits one ball hits eight ball)

• For example, assume a person experiences a service failure

• They infer a negative motive, feel angry, and spread NWOM

• Here, anger is the mediator between Inference of Negative Motive and NWOM

Inference ofNegative Motive

Anger Negative WordOf Mouth

Factorial Designs – Practice Problem• Factorial Design

• A store is interested in determining the effect of music and aroma on purchase amount during the holiday season

• Data are shown below• Summarize and graph the effects based on this data…

No Music MusicNo Aroma $20 $30Pleasant Aroma $30 $60

Practice Problem Answer• Factorial Design

• There is a main effect of aroma (pleasant aroma leads to higher sales), a main effect of music (music leads to higher sales)

• If we look at the graph, there is also an interaction (lines not parallel); the nature of that interaction is such that music has a bigger impact on sales when there is also a pleasant aroma in the store

No Music Music Row MeansNo Aroma $20 $30 $25Pleasant Aroma $30 $60 $45

Column Means $25 $45

Main Effect ofAroma

Main Effect ofMusic

Practice Problem Graph

Another Practice Problem• According to the animosity model of foreign product purchase, consumer

ethnocentrism leads to a reduction in willingness to pay for foreign products because it is negatively associated with perceptions of foreign product quality.

• Draw a diagram that illustrates this model• If this is a mediation model, identify the mediator

Another Practice Problem - Answer• According to the animosity model of foreign product purchase, consumer

ethnocentrism leads to a reduction in willingness to pay for foreign products because it is negatively associated with perceptions of foreign product quality.

• Draw a diagram that illustrates this model• If this is a mediation model, identify the mediator• Mediator = perceptions of quality

ConsumerEthnocentrism

PerceptionsOf Quality

Willingness to Purchase

Validity: Some Key Definitions• Validity (in general)

• The extent to which conclusions drawn from a study are true

• Internal Validity• When a researcher can clearly identify cause and effect relationships (i.e.,

there are no confounds)

• External Validity• The extent to which what you find in your study can be generalized to your

target population

• Construct Validity• Extent to which your constructs of interest (e.g., sensation seeking) are

accurately and completely identified (measured)• In other words, the extent to which you are actually measuring what you say

you are measuring (your sensation seeking scale really does measure the true construct of sensation seeking)

Threats to Internal Validity• History Effect

• When something (an historical event) happens during the course of a study that affects the dependent variable

• Maturation Effect• Similar to a history effect; something happens over time (changes in the individual) that

affects the DV• Testing Effect

• In a pretest-posttest design, you affect the time 2 DV by pretesting at time 1; the simple act of measuring the DV at time 1 changes the DV at time 2

• Instrumentation Effect• The mere fact that you are measuring something (e.g., observing behavior) changes the

behavior• Statistical Regression

• When you select groups based on extreme scores, they regress toward the mean, changing your groups

• Selection Bias• When groups (control, experimental) differ before experimental manipulation; creates

unequal groups (a confound)• Mortality

• Some drop out or die (attrition), and these drop-outs change scores in the condition; those who stick around may be different than those who drop out

Experimental Research Designs• Terminology

• X = subjects are exposed to a treatment (independent variable)• O = the outcome (dependent variable)• [R] = random assignment of subjects to conditions• EG = experimental group• CG = control group• = time

• Pre-experimental (“Crude” experimental) designs• Either have no control group or non-random assignment to groups• Suffer from low internal validity because it is not possible to compare

groups without the possibility of confounding factors• Types: One-shot, One-group, and Static Group comparisons

Pre-Experimental Research Designs - 1• One Shot Study

• X O1

• How do customers respond (O) to a single product like gatorade (X)?

• Problem? No control group. Response could be driven by many factors that covary along with the product (e.g., lighting, context). That is, there are many opportunities for extraneous variables to “confound” the manipulation of the IV.

• Internal validity low

Pre-Experimental Research Designs - 2• One Group Pre-test Post-Test

• O1 X O2

• How do sales of sweaters at time 1 (O1) change at time 2 (to O2) after the introduction of a new product display (X)?

• Problem? No control group. History and Testing Effects. Response could again be driven by many factors that covary along with the manipulation of the product display, that change over time with the introduction of the new product display (e.g., changes in store music or changes in the economy), or are related to testing at time 1. In other words, there are many opportunities for extraneous variables to “confound” the manipulation of the IV.

• Internal validity low

Pre-Experimental Research Designs - 3• Static Group Comparison (Non-random assignment to groups)

• Experimental Group (EG): X O1

• Control Group (CG): O2

• Compare two stores. In Store 1 (EG), use a promo display for nose strips. In Store 2 (CG), don’t use a promo display. Compare sales.

• Problem? Non-random assignment to groups. This again allows factors other than the promo display to affect sales. For example, in Store 1, it could be that the pharmacists are more friendly and more likely to recommend nose strips to their weary-eyed customers.

• Internal validity low

True Experimental Research Designs - 1• Pre-test, Post-test Control Group (subjects randomly assigned to groups)• Also called a mixed design (one within-subject variable, time; and one

between-subject variable, the experimental manipulation)

• Experimental Group (EG): [R] O1 X O2

• Control Group (CG): [R] O3 …….. O4

• Treatment effect = (O2-O1) – (O4-O3): The difference between differences

• Test all subjects (O1, O3), then randomly assign to experimental or control group, then test again (O2, O4)

• Eliminates testing effects, maturation, and (with good control over experimental conditions) confounding factors.

• Internal validity higher than earlier designs, but if not careful (low control over conditions), internal validity could be threatened

True Experimental Research Designs - 2• Post-test Only (but with subjects randomly assigned to groups)

• Experimental Group (EG): [R] X O1

• Control Group (CG): [R] O2

• Randomly assign to experimental group or control group, then compare levels of dependent variable (O)

• No testing effects. With good control over experimental conditions, eliminates confounding factors.

• Internal validity higher than static group comparison, but if not careful (low control over conditions), internal validity could be threatened.

True Experimental Research Designs - 3• Solomon Four Group Design

• Design 1 (Pre-test, post-test control group design)• Experimental Group (EG): [R] O1 X O2

• Control Group (CG): [R] O3 …….. O4

• Design 2 (Post-test only design)• Experimental Group (EG): [R] X O5

• Control Group (CG): [R] O6

• Let’s say that O = anger with waiting in line, and X = pleasant fragrances.

• If [O2 < O1], [O2 < O4], [O5 < O6], [O5 < O3], strong internal validity!

Quasi-Experimental Research Designs - 1• Non-equivalent control group• Like pre-test, post-test control group, but it is groups of subjects (not

individual subjects) who are randomly assigned to conditions. Hence there is no [R] shown below. For example, you could randomly assign stores to conditions (experimental vs. control), but you can’t randomly assign people to conditions, and you can’t control everything about the stores that may be confounded with the experimental manipulation.

• Experimental Group (EG): O1 X O2

• Control Group (CG): O3 …….. O4

• Treatment effect = (O2-O1) – (O4-O3): The difference between differences• Equivalence of groups prior to treatment: (O3-O1)

• Eliminates testing effects, maturation, and (with good control over experimental conditions) confounding factors. Scores at pre-test (O1 and O3) can be used as a control variable in data analysis. If in field, external validity is heightened over straight lab studies.

Quasi-Experimental Research Designs - 2• Separate sample pretest-posttest• Some folks (Sample 1) are tested before an advertising campaign (O1)• Then an advertising campaign occurs (X)• Then another group is tested after the campaign (O2)• Can’t be sure people were exposed to treatment (X), which is why it’s in

parentheses

• Sample 1: O1 (X)• Sample 2: (X) O2

• Some problems with internal validity (history, maturation), but external validity is high due to its naturalistic setting