Chapter 3: Research DesignResearch Design – a plan that shows how (methods and

data) and why (literature review and theory background) a researcher intends to study an empirical question.

I. Selection of RDs shaped by:A. Purpose – exploratory, descriptive, or explanatory?B. Practical limitations – unethical, lack of data, time,

or moneyC. RDs to testing hypotheses have at least one of the

following objectives:1. Establish relationships between _______________2. Demonstrate that the results are ___________ true

in the real world3. Reveal the time order of phenomena4. Eliminate __________ hypotheses or explanations

II. Causality and Controlled Experiments A. Correlation or Causation? Example: Negative Ads and

Turnout (Table 3-1)1. There is a relationship; is it causal? Could be that the

unexposed are more educated and have a stronger sense of civic duty. If true, then we observe a noncausal relationship only (outlawing negative ads would have no effect on turnout).

2. ____________ relationships (joint causation) – when two things are both affected by a third factor and only appear to be related. When the third factor is taken into account (controlled for), the relationship weakens (example: # of fire trucks causes greater fire damage?)

B. Requirements for Causal Design1. Covariation: X (cause) does in fact vary with Y (effect) in

either a positive or negative direction. 2. Time order: X occurs before Y in time (cause must always

precede effect).3. Elimination of alternative causes (confounding factors and

spurious relationships). How we design research helps us establish causal relationships.

III. Randomized Controlled ExperimentsA. Experimentation (greater certainty of causality) – research

that allows the researcher to control exposure to an experimental variable (test factor or independent variable), the assignment of subjects to different groups, and the observation or measurement of responses and behavior. Not common in Political Science, but adapted in survey research

1. Subjects:a. Experimental group (exposed or tested to “treatment”

variable).b. Control group – group not exposed to treatment/program2. Randomization – chance process of assigning subjects to

each group (or using precision matching to assure group similarities). Goal = groups are same except for treatment.

3. Researcher determines when, where and under what conditions treatment stimulus occurs.

4. Researcher measures the treatment impact or response (often referred to as pre and post-test results). Shows if and how much of an experimental effect there is.

5. Research controls environment and is able to exclude or control the effect of extraneous factors (contamination).

B. Application Questions: how could we conduct this type of research on negative ads and voting? (p. 56-57).

1. Identification of variables (dependent and independent)2. Randomized group assignment (coin flipping)3. Questionnaire for demographic and political factors as well as

dv question (averages should be same)4. Pretest – both groups view 15 minute news report. 5. Treatment – Experimental group views negative commercials6. Posttest – all subjects view another 15 minute news report. 7. Experimental effect = change in experimental group’s

intention to vote % (E = Mexp2 – Mexp1)How has this satisfied our requirements for causality?

C. Problems with Internal Validity of Experimentation (Internal Validity is the extent to which the research procedure demonstrated a true cause and effect relationship and was not due to some other factor).

1. History – allowing events or conditions to change in between pre and post testing.

2. Maturation – change in the subjects themselves in between pre and post testing (may change reaction to treatment too).

3. Testing – sometimes the treatment itself may change the predisposition of the subject (e.g., effect of a political debate on political awareness; pretest question may sensitize them to politics suddenly).

4. Selection biases – subjects may be picked according to some nonrandom process (e.g., smoking program).

5. Experimental mortality or attrition – subjects may leave one group or another and change the comparability of the groups.

D. Problems with ___________ Validity and Experimentation (External Validity is the extent to which the obtained results can be generalized to other populations, times, and settings).

1. Population from which pool is drawn may only be in one area, from one age group, from one race, etc.

2. Artificial treatments may not reflect “real world” (e.g., are groups exposed to one negative ad or hundreds? In the context of a room or filtered through comments of friends).

IV. Nonexperimental Designs (use of survey data, focus groups, aggregate data or collectivities, content or document analysis, case studies)

A. Interrupted Time Series Analysis – Repeated measures of a dependent variable (e.g., # of conservative SC Decisions). Then, an event in time is identified (say change in party control of Congress). Observe changes in dv

B. Continuous Time Series Analysis – repeated measures of both dv and iv (conservative SC decisions and “public mood” p. 79).

C. Cross-sectional design – measurements of dv and iv at a particular point in time (e.g., Survey and question: were evangelicals more likely to vote Republican in the 2004 election? Or compare two school districts science test scores; one with no instruction in evolutionary theory). No randomization and no trend observation.

D. Panel Studies – cross-sectional design that introduces time. Same subjects or units measured repeatedly over time.

E. Case study – researcher examines one or a few cases in detail, typically using data collection methods such as personal interviews, document analysis, and observation (distinctive from empirical inquiry; not quantitative typically but qualitative). Example: Do countries that adopt popular election methods have greater public trust of government. Case study vs Large-case # (n) study.

F. ______________ modeling – simple and abstract representation of reality that purports to show how variables or parts of a system fit together. Assumes people are rational or “utility-maximizers” meaning they have preferences and make choices that always increase their well-being.

For example, a simple formal model of vote choice is UAvB= UA – UB where U is expected utility and A/B are candidates or parties (0 means you abstain, + number means you vote for A, - means you vote for B). But, if the absolute value of UAvB is less than the value of another factor C (information and transaction costs of voting), then one abstains still. Also, we must consider E, the intangible benefits of voting. Called paradox of voting because of another factor P (probability of casting a decisive vote). So, we get P|UAvB|+E </> C