PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with...

30
PS 366 4

description

Measurement Bias & error with the instrument – Random? – Systematic?

Transcript of PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with...

Page 1: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

PS 366

4

Page 2: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Related to reliability, validity:

• Bias and error– Is something wrong with the instrument?

– Is something up with the thing being measured?

Page 3: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Bias & error with the instrument

– Random?

– Systematic?

Page 4: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Bias & error with the thing being measured

– Random?• failure to understand a survey question

– Systematic?• does person have something to hide?

Page 5: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Example:– Reliability, validity, error & bias in measuring

unemployment

– Census survey [also hiring reports, claims filed w/ government, state data to feds...]

– What sources of bias?

Page 6: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Unemployment [employment status]:

– Fully employed– Part time– looking for work, + part time– looking for work, no job– lost job, not looking for work– retired

Page 7: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• Example: – Reliability, validity, error & bias in measuring

victims of violent crime

– Census surveys, police records, FBI UCR

– What sources of bias?

Page 8: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement

• How do we ask people questions about attitudes, behavior that isn’t socially accepted?– prejudice– Racism– Feelings toward gays & lesbians– shoplifting

Page 9: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement: Item Count Technique

• Here are 3 things that sometimes make people angry or upset. After reading these, record how many of them upset you. Not which ones, just how many?

• federal govt increasing the gas tax

• professional athletes getting million dollar salaries

• large corporations polluting the environment

Page 10: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement: Item Count Technique

• federal govt increasing the gas tax

• professional athletes getting million dollar salaries

• large corporations polluting the environment

• federal govt increasing the gas tax

• professional athletes getting million dollar salaries

• large corporations polluting the environment

• a black family moving next door

Page 11: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Measurement: Item Count Technique

• Randomly assign ½ of subjects to the 3 item list

• Randomly assign ½ subjects to the 4 item list

• Difference in mean # of responses between groups = % upset by sensitive item– (mean 1 – mean 2) *100 = %

Page 12: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Item Count• Control TREATMENT

% upset

• Non South 2.28 2.24 0

• South 1.952.37 42

2.37 – 1.95 = 0.42 *100 = 42%

Page 13: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Item Count – Using poll information

• 1) The candidate graduated from a prestigious college

• 2) The candidate ran a business

• 3) The candidate’s family background

• 1) The candidate graduated from a prestigious college

• 2) The candidate ran a business

• 3) The candidate’s family background

• 4) The candidate is ahead in polls

Page 14: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Use poll info• Control TREATMENT % use poll

• All 2.28 1.36 1.39 3.2

• Young 1.04 1.46 41

• Is it significant?– Depends....how much does mean reflects the group?

How much variation around the mean?

Page 15: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency: Chapter 4

• Statistics that describe the ‘average’ or ‘typical’ value of a variable

– Mean– Median– Mode

Page 16: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency

• Why median vs. mean?

– Household income

– Home prices

Page 17: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency

• Mean125 92 72126120 99130100sum=864

• mean = sum X/ N

• = 864 / 8

• mean = 108

• Is this representative?

Page 18: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency

• Mean125 92300126120 99130100sum=1092

• mean = sum X/ N

• = 1092 / 8

• mean = 136.5

• Is this representative?

Page 19: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency

• Mean130126125120100 99 92

• median = (N +1) /2– (8+1)/2– 9/2– 4.5 th – (120, 125)

• Is this representative?

Page 20: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Central Tendency

• Example$120,00$60,000$40,000$40,000$30,000$30,000$30,000

• Mean = $50,000 Mdn = $40,000 Mo = $30,000

• Which is most representative?

Page 21: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Mean vs Median

Page 22: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Mean vs. Median

• median = ½ point. 12.5 = 32,000

• mean = (Sum of X’s) / n = 816,000 / 25 = 32,640

Why is mean higher than median?

Page 23: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Normal vs. Skew Distributions

Page 24: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Which direction is skew here?

Page 25: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Frequency Distribution

Page 26: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Median vs Mean Price

• Seattle, 98121– Median $375,900– Mean $434,612

• Seattle, 98112 – Median $790,000– Mean $955,750

Page 27: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

The Distribution

• Where is mean, median, mode if

– Normal

– Left / negative skew

– Right / positive skew

Page 28: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Normal vs. Skew Distributions

Page 29: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

The Distribution

• Where is mean, median, mode if

– Normal ALL THE SAME

– Left / negative skew Mean less than median

– Right / positive skew Mean greater than median

Page 30: PS 366 4. Measurement Related to reliability, validity: Bias and error – Is something wrong with the instrument? – Is something up with the thing being.

Chapter 4 Practice• # 6

• # 7

• #8

• #15