研究设计之 测量 School of Public Administration & Policy Dr. Kaifeng Yang.
-
date post
19-Dec-2015 -
Category
Documents
-
view
328 -
download
5
Transcript of 研究设计之 测量 School of Public Administration & Policy Dr. Kaifeng Yang.
从名义从名义 // 概念定义到操作定义概念定义到操作定义
Nominal (Conceptual) Definition
A definition that is simply assigned to a term without any claim that the definition represents a “real” entity. It is A) defined by others; b) what it is vs. what it is not; c) shared; d) realistic and sensible.
Operational (Working) Definition
Specifies precisely how a concept will be measured
菜谱
概念化概念化
The process … we specify what we mean when we use particular terms. It involves describing the essential features of the terms. Sometimes, it involves indicators we’ll be using to measure our concept and the different aspects of the concept (dimensions).
Indicator(指标 ): a sign of the presence or absence of the
concept Dimension (纬度 /变量 ): a specifiable aspect of a
concept
爱?
指标化指标化 : Political Alienation: Political AlienationConceptualization
Defining the concept
Nominal Definition(Variables)
Operational Definition
(Indicators)
Instrumentation
Political distrust, powerlessness, and meaninglessness
Political Distrust: the general feeling that government isno longer of the people, by the people, and for the people
Indicators are responses to the following questions:1. Do you think that quite a few of the people running the government are a little crooked, not very many are, or do you think that hardly any are crook at all?2. How much of the time do you think you can trust
the government to do what is right?3. Would you say that the government is pretty much
run by a few big interests looking out for themselvesor that it is run for the benefit of all people?
General Social Survey, 2000, National Opinion Research
Theory
abstract
DataConcrete
指标化:指标化: Other ExamplesOther Examples Literacy:
– The completion of six years of formal education– Those nations in which at least 50 percent of the population has
had six years of formal education, as indicated in a publication of the United Nations
Democracy:– A system of government in which public officials are selected in
competitive elections– Those countries in which the second-place finishers in elections
for the chief executive office has received at least 25 percent of the vote at least once in the past eight years
To operationalize your concepts is arbitrary, but you have to make some justification if not based on common literature or policy.
指标化:例子指标化:例子
Social Status
Education
Income
Occupation
Numbers of school years completed
Cross family income last year
Blue collar, white collar, or professional
PoliticalParticipation
Voting
Campaign contribution
Attend political rally
“Did you vote in the last…”
“Did you contribute to …”
“Have you attended a …”
Concept Variable Operational Definition
In the sample article? Communication PerformanceCommunication Performance
Conceptualization
Nominal Definition(Variables)
Operational Definition
(Indicators)
Instrumentation
External Communication Performance
Reverse: Reverse: 从问题从问题 // 数据到概念?数据到概念?
“How many times a week do you drink an alcoholic beverage?”1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10
Percentage of a nation’s population that is literate “I often feel lonely”
1. agree 2. disagree
“number of hours of sleep each night” “number of trips taken abroad each year by presidents”
Level of MeasurementLevel of Measurement
1. “How many times a week do you drink an alcoholic beverage?”
1. 1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10
2. “How many times a week do you drink an alcoholic beverage?” _____
3. “Do you think you are addicted to alcohol? 1. Yes2. No
4. “I am alcoholic.” 1. Strongly agree 2. Agree 3. Neutral 4. Disagree 5. Strongly Disagree
Precision/Level of MeasurementPrecision/Level of Measurement
Nominal Measures(定类)exhaustive and mutual exclusive categories; none is more or less than another
Ordinal Measures(定序)can logically rank-order: with more or less of particular attribute
Interval Measures(定距)How much smaller or larger; the distance between each unit on the scale must be the same;
Ratios Measures(定比)Interval measures; the zero point has a true meaning (absence of the characteristic being measured)
PracticesPractices
You have data on the annual salary of 200 employees of an organization. How the data can be presented as ratio, interval, ordinal, and nominal data?
例子:怎么测量国家的“自由”程度?例子:怎么测量国家的“自由”程度?
(Country) 1 2 3 4 5
Privately owned newspapers 1 0 0 0 1
Legal right to form political parties 1 1 0 0 0
Contested elections for public offices
1 1 0 0 0
Voting rights for most of the adults 1 1 0 1 0
Limitations on government’s ability to incarcerate citizens
1 0 0 0 1
Index score 5 3 0 1 2
1= Yes; 0=No.
李克特量表李克特量表 Likert Format (for other formats, refer to the textbook)
SD
1
D
2
U
3
A
4
SA
5
Privately owned newspapers
Legal right to form political parties
Contested elections for public offices
Index score for Observation x 15 or 15/3=5
测量的好坏?测量的好坏?How do you measure individuals’ alcohol addiction?
“How many times a week do you drink an alcoholic beverage?”
1. 1. None 2. 1~2 3. 3~5 4. 5~7 5. 7~10 6. >10
“How many times a week do you drink an alcoholic beverage?” _____
“Do you think you are addicted to alcohol? 1. Yes2. No
“I am alcoholic.” 1. Strongly agree 2. Agree 3. Neutral 4. Disagree 5. Strongly Disagree
How many bottles of alcohol do you normally drink for a week? ____
How many bottles of alcohol do you normally drink for a day?
测量的准确度测量的准确度 To what extent are you measuring what you intended
to measure? Ideal: Measurement error is 0
Measurement = Intended characteristic + Systematic error + Random error
Systematic error is durable Random error is temporary
Pre-test = Math Ability + ____+_____
准确度准确度 : : 信度信度
Reliability: produce same results over repeated applications. Stability and Consistency
– Test-Retest method 再测信度* Two times * maturation * sensitivity
– Alternative format复本信度* two times *maturation
– Split-Halves method (multiple measures, one concept)折半信度– Sub-sample method 亚样本信度
Split-HalvesSplit-Halves
Measuring Performance measurement (PM) effectiveness
1. Our PM information can be trusted2. Our PM helps managerial decisions3. Our PM improves communication between city hall and us4. Our PM help elected officials formulate budget5. Our PM indictors reflect our management effectiveness6. Our PM indictors are reliable7. Our investment in PM is worthwhile 8. Our PM improves productivity9. Our PM motivates employees10. Our PM helps adjusting strategic planning
准确度:效度准确度:效度 Validity: the extent to which an empirical measure
adequately reflects the real meaning of the concept under consideration.– Face validity表面效度 : Does it appear…? Judgment!– Content validity内容效度 : Got everything in?– Consensual validity共识效度 : Everybody agrees!– Construct validity—Criterion-related validity
* Predictive validity * Concurrent validity– Interitem association– Discriminant validity
测量误差与信度、效度测量误差与信度、效度Measurement = Intended characteristic + Systematic error + Random error
• Systematic error is durable, affecting validity
• Random error is temporary, affecting more on reliability
Which of the following questions concerns reliability?
1. What does the Scholastic Aptitude Test (SAT) really measure?
2. Do attitude scales really tap personal feelings or just what people think investigators want to hear?
3. Is poverty best represented by measures of low absolute income (a minimum subsistence) or low relative income (e.g., 50% of national median income)?
4. How closely does self-reported church attendance correspond to actual church attendance?
5. Are scores on the even-numbered and odd-numbered items on this test highly correlated?
Research indicates that in recent years respondents’ answers to survey questions measuring racial prejudice underestimate actual racial prejudice. This underestimate is an example of __________ measurement error, which affects measurement __________.
random; reliability random; reliability and validity systematic; validity systematic; reliability and validity
Systematic error would be least likely to occur with which of the following measures?
1. a miscalibrated yardstick that is an inch short2. an IQ test developed for English-speaking U.S.
citizens that is administered to French Canadians3. a question measuring child abuse that asks, “how
often do you hit your child when she doesn’t obey you?”
4. scores on the multiple-choice portion of an exam 5. reports of gambling winnings on IRS tax return
Validity and ReliabilityValidity and Reliability• Reliability is a necessary, but insufficient, condition, for validity
• validity is more important, but reliability is more easily tested
• Think about possible errors
Low V
Low R
Low V
High R
Low V
Low R
High V
High R
提高效度与信度:减少误差提高效度与信度:减少误差
Participant (object who)Situational factor (where, how, when)Measurer (subject who)Instrument (what)
Facial attractiveness?
提高效度与信度:减少误差提高效度与信度:减少误差
Participant (object who)客体 /对象Situational factor (where, how, when)情境与程序
Measurer (subject who)主体Instrument (what)工具
Facial attractiveness?