Survey Research Survey Research

And a few words about elite And a few words about elite interviewinginterviewing

AgendaAgenda

Elite interviewingElite interviewing

The logic of a surveyThe logic of a survey

Survey formatsSurvey formats

Types of Survey QuestionsTypes of Survey Questions

Writing Good QuestionsWriting Good Questions

Formatting A SurveyFormatting A Survey

Hand back quizHand back quiz

Elite InterviewingElite Interviewing

ElitesElites People in governmentPeople in government ActivistsActivists Labor LeadersLabor Leaders

Not ideal for standardized questionsNot ideal for standardized questions

Some standardization is desirableSome standardization is desirable

Elite InterviewingElite Interviewing

One on OneOne on One

Face to FaceFace to Face

PreparationPreparation

Questions up front?Questions up front?

ConfidentialityConfidentiality

Right of Approval?Right of Approval?

Get accurate recordGet accurate record

SurveysSurveys

Fixed questionnaire Fixed questionnaire

Sample of the population of interestSample of the population of interest

Choose a good random sampleChoose a good random sample

Generalize to population of interestGeneralize to population of interest

Administering SurveysAdministering Surveys

Face to faceFace to face

TelephoneTelephone

MailMail

InternetInternet

Non-response is a general concern Non-response is a general concern

Survey Questions- TypesSurvey Questions- Types

Open endedOpen endedAdvantagesAdvantages

UnstructuredUnstructured More nuancedMore nuanced In their own wordsIn their own words

DisadvantagesDisadvantages Time consuming to Time consuming to

administeradminister Must be Must be

interpreted/groupedinterpreted/grouped Tend to only support Tend to only support

nominal level datanominal level data

Closed endedClosed endedAdvantagesAdvantages

Yield immediate dataYield immediate data People answering in same People answering in same

termsterms People more willing to People more willing to

respondrespond

DisadvantagesDisadvantages May miss important May miss important

dimensionsdimensions May lose some detailMay lose some detail May aggregate answers in May aggregate answers in

odd waysodd ways May be confusing May be confusing

Open Ended QuestionsOpen Ended Questions

Is there anything in particular about MR. Is there anything in particular about MR. CLINTON that might make you want to CLINTON that might make you want to vote FOR him? vote FOR him?

Is there anything in particular about MR. Is there anything in particular about MR. CLINTON that might make you want to CLINTON that might make you want to vote Against him? vote Against him?

What do you think are the most important What do you think are the most important problems facing this country? problems facing this country?

Closed Ended QuestionsClosed Ended Questions

Think about BILL CLINTON. In your Think about BILL CLINTON. In your opinion does the phrase "he is honest” opinion does the phrase "he is honest” describe Bill Clinton extremely well, quite describe Bill Clinton extremely well, quite well, not too well, or not well at all?well, not too well, or not well at all?

Closed EndedClosed Ended

Some people think it is important to protect the Some people think it is important to protect the environment even if it costs some jobs or environment even if it costs some jobs or otherwise reduces our standard of living. otherwise reduces our standard of living. (Suppose these people are at one end of the (Suppose these people are at one end of the scale, at point number 1.) Other people think scale, at point number 1.) Other people think that protecting the environment is not as that protecting the environment is not as important as maintaining jobs and our standard important as maintaining jobs and our standard of living. (Suppose these people are at the other of living. (Suppose these people are at the other end of the scale, at point numbeer 7.) And, of end of the scale, at point numbeer 7.) And, of course, some other people have opinions course, some other people have opinions somewhere in between, at points 2,3,4,5 or 6.somewhere in between, at points 2,3,4,5 or 6.

Open vs. Closed EndedOpen vs. Closed Ended

What was your total household income before What was your total household income before taxes in 2003?taxes in 2003?I am going to mention a number of income I am going to mention a number of income categories. When I come to the category, which categories. When I come to the category, which best describes your total household income best describes your total household income before taxes in 2003, please stop me.before taxes in 2003, please stop me. Under $10,000Under $10,000 10 to less than $20,00010 to less than $20,000 20 to less than $30,00020 to less than $30,000 30 to less than $40,00030 to less than $40,000 40 to less than $50,00040 to less than $50,000 50 to less than $60,00050 to less than $60,000

Branching QuestionBranching Question

Start with a screenerStart with a screenerCollect more detailCollect more detailGenerally speaking, do you usually think of Generally speaking, do you usually think of yourself as a Republican, a Democrat, an yourself as a Republican, a Democrat, an Independent, or what?Independent, or what?Would you call yourself a strong Democrat or a Would you call yourself a strong Democrat or a not very strong Democrat?not very strong Democrat?Would you call yourself a strong Republican or a Would you call yourself a strong Republican or a not very strong Republican?not very strong Republican?Do you think of yourself as closer to the Do you think of yourself as closer to the Republican or to the Democratic Party?Republican or to the Democratic Party?

Midpoint/Don’t KnowMidpoint/Don’t Know

Midpoint- allows for neutral responseMidpoint- allows for neutral response Advantage- might be more accurateAdvantage- might be more accurate Disadvantage- too easy?Disadvantage- too easy?

Don’t knowDon’t know Advantage- cuts down on guessing, Advantage- cuts down on guessing,

eliminates false attitudeseliminates false attitudes Disadvantages- missing data, not used Disadvantages- missing data, not used

equally by all groupsequally by all groups

Randomized Response TechniqueRandomized Response Technique

Good for getting more truthful answers on Good for getting more truthful answers on controversial issues. controversial issues. Randomly assign people to answer Randomly assign people to answer sensitive or random question (interviewer sensitive or random question (interviewer does not know)does not know)Look at question overallLook at question overallCompare rate of yes answers to see if it Compare rate of yes answers to see if it differs from expected yes answers for differs from expected yes answers for random questionrandom question

Group 1Group 1

Imagine the US is preparing for an Imagine the US is preparing for an outbreak of an unusual Asian Disease outbreak of an unusual Asian Disease which is expected to kill 600. Two which is expected to kill 600. Two alternative plans to deal with it have been alternative plans to deal with it have been proposed. proposed. If A is chosen, 200 People will be savedIf A is chosen, 200 People will be savedIf B is chosen, there is a 1/3 Chance that If B is chosen, there is a 1/3 Chance that 600 people will be saved and a 2/3 600 people will be saved and a 2/3 chance that no people will be savedchance that no people will be saved

Group 2Group 2

Imagine the US is preparing for an outbreak of Imagine the US is preparing for an outbreak of an unusual Asian Disease which is expected to an unusual Asian Disease which is expected to kill 600. Two alternative plans to deal with it kill 600. Two alternative plans to deal with it have been proposed.have been proposed.

If Program A is adopted, 400 people will dieIf Program A is adopted, 400 people will die

If Program B is adopted, there is a 1/3 If Program B is adopted, there is a 1/3 probability that no people will die and a 2/3 probability that no people will die and a 2/3 probability that 600 people will dieprobability that 600 people will die

ResultsResults

If A is chosen, 200 People will be savedIf A is chosen, 200 People will be savedIf B is chosen, there is a 1/3 Chance that If B is chosen, there is a 1/3 Chance that 600 people will be saved and a 2/3 600 people will be saved and a 2/3 chance that no people will be savedchance that no people will be savedIf Program A is adopted, 400 people will If Program A is adopted, 400 people will diedieIf Program B is adopted, there is a 1/3 If Program B is adopted, there is a 1/3 probability that no people will die and a 2/3 probability that no people will die and a 2/3 probability that 600 people will dieprobability that 600 people will die

Equivalence framesEquivalence frames

A and B are numerically equivalent across A and B are numerically equivalent across conditionsconditions

When framed in terms of death, people When framed in terms of death, people take risk (group 2)take risk (group 2)

When Framed in terms of saving, people When Framed in terms of saving, people choose sure thingchoose sure thing

Take home point: Little differences in Take home point: Little differences in wording can make a big differencewording can make a big difference

Bad QuestionsBad Questions

AmbiguousAmbiguous

Double Barreled Double Barreled

Leading/Loaded Questions Leading/Loaded Questions

Ambiguous QuestionsAmbiguous Questions

What is your income?What is your income?

How often do you participate in community How often do you participate in community events?events? AlwaysAlways SometimesSometimes RarelyRarely NeverNever

Double Barreled QuestionsDouble Barreled Questions

How much to do agree with the following How much to do agree with the following statement: There is too much violent statement: There is too much violent crime. Citizens should be able to carry crime. Citizens should be able to carry handguns for personal protection. handguns for personal protection.

Irish, Italian, Jewish, and other minorities Irish, Italian, Jewish, and other minorities overcame prejudice and worked their way overcame prejudice and worked their way up. Blacks should do the same without any up. Blacks should do the same without any special favors. special favors.

Leading QuestionsLeading Questions

Do you believe that the government Do you believe that the government should take action to stop the flood of should take action to stop the flood of illegal immigrants coming into our country?illegal immigrants coming into our country?

Do you intend to vote for candidate A who Do you intend to vote for candidate A who will bring you peace and prosperity, or will bring you peace and prosperity, or candidate B who will bring you war and candidate B who will bring you war and crippling taxes? crippling taxes?

Challenge- What WordingChallenge- What Wording

Some cases clearly wrongSome cases clearly wrong

Other’s less clear cutOther’s less clear cut

75% agree with the statement: The federal 75% agree with the statement: The federal government should spend more money on government should spend more money on aid to the pooraid to the poor

75% oppose the statement: The federal 75% oppose the statement: The federal government should spend more money on government should spend more money on welfare.welfare.

Formatting a SurveyFormatting a Survey

Order MattersOrder Matters

Generally, put open ended questions firstGenerally, put open ended questions first

Put sensitive measures firstPut sensitive measures first

Randomize where possibleRandomize where possible

MIDTERM1

90.0080.0070.00

Co

un

t

16

14

12

10

8

6

For Next TimeFor Next Time

More on Surveys and SamplingMore on Surveys and Sampling

Annotation 2 dueAnnotation 2 due