Measurement Error and Related Definitions Error = measured value - true value
1 MEASUREMENT Measurement Error & Survey Construction.
-
Upload
jonathon-bye -
Category
Documents
-
view
234 -
download
3
Transcript of 1 MEASUREMENT Measurement Error & Survey Construction.
2
Measurement: Measurement Error in Survey Research
• What is measurement Error?
Measurement Error = Observed Score - True Score
• What are the sources of measurement error in survey research and how do we minimize it?
•One Requirement of Effective Research Design:Minimizing measurement error (Remember MAXMINCON Principle!)
3
Measurement Error in Survey Research
• Instrumentation Error/Bias• Error that results from lack of fidelity of the
measurement instrument (e.g., poor instructions, wording, scales, response options)
• Response Error/Bias• Error that stems from the mentality and/or
predisposition of study participants
Types of Measurement Error/Bias:
4
Measurement Error in Survey Research--Instrumentation Bias
Sources of Instrumentation Bias -- How to Minimize It?A. Poor Wording of Items/Instructions:
– Guideline #1: Are some words in the item ambiguous? Spell out acronyms, define difficult to understand terms, and reword the items using clearer and more commonly recognized vocabulary.
– Guideline #2: Does the question state the necessary/ appropriate frame of reference/criterion for answering? If not, the criterion must be clearly indicated.
– Guideline #3: Does the question overemphasizes/ exaggerates/dramatizes some condition. If so, it must be toned down and stated in less dramatic terms.
5
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #4: Is the question as specific as it can reasonably be? If the item is too general, state it more specifically.
– Guideline #5: When possible, avoid using negatives and double-negatives in statements--they are confusing.
– Guideline #6: Avoid putting a blank in the middle of an item.
6
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #7: Use your pronouns carefully when writing both, the items as well as the instructions.
– Guideline #8: Choose proper and clear wording and sentence structure for the items and instructions.
– Guideline #9: Is the question likely to lead respondents toward a particular answer (i.e., is it a leading question)? If so, the leading phrase must be removed.
7
Measurement Error in Survey Research--Instrumentation Bias
B. Poor Content of Items/Questions:– Guideline#10: Is the question applicable to all respondents? If not,
reword it and/or use a detour around it for those to whom it does not apply.
– Guideline#11: Avoid branching if at all possible. The two items above can easily be rewritten as one question that applies to everyone.
– Guideline #12: Does the item contain an example that is also a possible answer? If so, change or discard the example.
8
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #13: Does the question require respondents to remember too much detail or to recall distant events? If so, it must be modified or generalized to make recall easier.
– Guideline #14: Is the question as free from threat to respondents as possible? If not, change it to reduce the threat; consider depersonalizing it and/or asking about hypothetical situations.
– Guideline #15: Does the question include more than one issue (i.e., a double-barreled item)? If so, it must be split or modified to include only a single question .
– Guideline #16: Is the question “lladed” with a reason for responding in a particular way? If so, the reason must be deleted.
9
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #17: Underline, italicize, or use bold print to draw respondent’s attention to important terms in the instructions.
C. Poor Response Format/Options:C. Poor Response Format/Options:
– Guideline #18: Make sure the list of response options is exhaustive.
– Guideline#19: Are “Yea-sayers” or “nay-sayers” likely to always choose one answer? Reword the item to avoid using “Yes” and “No” as choices.
– Guideline #20: Use “Always” and “Never” with caution; “Almost Always” and “Almost Never” may be much better substitutes.
10
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #21: Note that not all rating scales are bipolar or have to be balanced/symmetrical with a neutral mid-point (e.g., importance, likelihood, etc.).
– Guideline #22: When the respondent is to select only one choice from a list of several options, make sure the choices are mutually exclusive.
– Guideline #23: Make sure that the formats of your items and your response options match (are consistent).
11
Measurement Error in Survey Research--Instrumentation Bias
D. Organization and Order:
– Guideline #24: Be cognizant of position/order bias.
– Guideline #25: When appropriate, use the funnel approach to avoid the potential assimilation effect. That is:
The more general first, followed by the more specific
– Guideline #26: Use duller demographic questions in a section at the end of the questionnaire (the opposite in interviews)
– Guideline #27: Use more interesting and attention-grabbing set of questions earlier in the survey.
12
Measurement Error in Survey Research--Instrumentation Bias
– Guideline #28: Use more sensitive, personal and/or threatening items later in the survey.
– Guideline #29: Organize questionnaire by arranging items into subsections (based on content or/and response format), each with a clear and short instruction.
– Guideline #30: At the end of the survey, indicate what your respondents are expected to do next—e.g., mail the survey to whom, at what address, by when.
14
Measurement Error in Survey Research--Response Bias
Sources of Response Bias -- How to minimize it?– SOCIAL DESIRABILITY:
Response based on what is perceived as being socially acceptable or respectable.
Response Error/Bias: Error that stems from the mentality and/or predisposition of study participants.
15
Measurement Error in Survey Research--Response Bias
– YEA- AND NAY-SAYING:Response influenced by the global tendency toward
positive or negative answers.
–ACQUIESCENCE:Response based on respondent’s perception of what would be desirable to the sponsor.
16
Measurement Error in Survey Research--Response Bias
– THREAT:
Response influenced by anxiety or fear instilled by the nature of the question.
–PRESTIGE:Response intended to enhance the image of the respondent in the eyes of others.
17
Measurement Error in Survey Research--Response Bias
– AUSPICES:Response dictated by the image or opinion of the
sponsor rather than the actual question.
–HOSTILITY:Response arising from feelings of anger or resentment engendered by the response task.
18
18
– ORDER:The sequence in which a series is listed affects the
responses to the items.
– EXTREMITY:Clarity of extremes and ambiguity of midrange
options may encourage extreme responses.
–MENTAL SET:Cognitions or perceptions based on previous items influence response to later ones (the “priming” problem).