Interocularly merged face percepts eliminate binocular rivalry
Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement,...
Transcript of Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement,...
![Page 1: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/1.jpg)
Measurement, validity and reliability
![Page 2: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/2.jpg)
Some thoughts….
'Concepts without percepts are empty...Percepts without concepts are blind'.
Immanuel Kant
Aristarchos of Samos evolved around 280-265 BC the heliocentric theory of the universe. The reason that he did not cause a scientific revolution 1700 years before Copernicus is that he could not support his theory with sufficiently precise observations and measurement.
A. R. Burns. The Penguin History of Greece
![Page 3: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/3.jpg)
Some more thoughts…
The social scientist finds himself in continuous effort to combine two ideals: vision and precision. Clearly, he needs precise instruments if he wants to develop testable propositions. But the social world is very complex and doesn’t provide us with the well delineated objects from which the natural sciences start. Thus, he also needs a great deal of creative imagination - of vision - to decide on the objects about which such propositions should be developed
Paul Lazarsfeld (1965) foreword to Constructive Typology and Social Theory
![Page 4: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/4.jpg)
Concepts/constructs
• Concepts are abstractions, they can, though needn’t be, expressions of ideas that exist in everyday discourse, where usage may be relatively imprecise
• By their nature concepts are unobservable
• They are of use because
– they help to organise experience
– they help to communicate it to others
– they help to explain experience
![Page 5: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/5.jpg)
Examples of concepts
• Bureaucracy
• Organization
• Leadership
• Payment system
• Hierarchy
• Market
• Management
![Page 6: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/6.jpg)
Operationalization
• Conceptualisation
– process of identifying and clarifying concepts.
• Operationalization
– specifying observable phenomena (indicators) that
indicate
• presence or absence
• strength or weakness
– specifying instruments to measure the indicators
![Page 7: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/7.jpg)
Measurement
'...assignment of numbers to objects or events according to rules' S. S. Stevens
Domain
Range
![Page 8: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/8.jpg)
Classification
![Page 9: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/9.jpg)
Corporate
Communication Concept
![Page 10: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/10.jpg)
Corporate
Communication
Shareholders
Regulators Employees
Customers
Public
relations
Concept
Dimensions
![Page 11: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/11.jpg)
Corporate
Communication
Shareholders
Regulators Employees
Customers
Public
relations
Concept
Dimensions
Written
Face to face
Sub-
dimensions
Electronic
![Page 12: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/12.jpg)
Corporate
Communication
Shareholders
Regulators Employees
Customers
Public
relations
Concept
Dimensions
Written
Face to face
Sub-
dimensions
Electronic
With
colleagues Further
sub-
dimensions
With senior
management
With
subordinates With
supervisor
With line
management
![Page 13: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/13.jpg)
With
supervisor
Amount of
contact Who
initiates? Length of
contact
Type of
communication
Further
sub-
dimension
Indi-
cators Presence
of others
![Page 14: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/14.jpg)
With
supervisor
Amount of
contact Who
initiates? Length of
contact
Type of
communication
Further
sub-
dimension
Indi-
cators Presence
of others
Survey
questions
In the past 7 days how many times have you had a
face-to face conversation with your supervisor?
Never
A few times a week
About once a day
Several times a day
Don’t have a supervisor
![Page 15: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/15.jpg)
Validity and reliability
• Validity
– Does the measure accurately reflect the
concept it is meant to measure?
• Reliability
– Does a particular measure, applied to the
same object, yield the same result each time?
• Validity presumes reliability
![Page 16: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/16.jpg)
How to evaluate reliability I
• Test-retest method
– consistency of repeated measurements on
same subjects (possibly by different
observers)
– problems
• reality may change
• subjects react to the testing process
• memory
![Page 17: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/17.jpg)
How to evaluate reliability II • Inter-item reliability
– Use in parallel more than one indicator of the same construct (Union Militancy)
• I am opposed to strike action under any circumstances.
• I could not vote for strike action regardless of the facts and circumstances of the case.
– If certain assumptions are met… • reliability can be measured as the correlation between the
responses to two items
• if there are more items as a function of the average inter-item correlation
![Page 18: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/18.jpg)
Threats to measurement validity
• Measure covers
– only part of the concept,
– more than the concept,
• Interviewees
– misunderstand questions,
– systematically exaggerate or understate,
– selectively refuse to answer,
– systematically make mistakes when answering questions,
– want to finish interview and systematically agree
![Page 19: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/19.jpg)
Types/tests of validity I
• Subjective Validation
– Face validity, Content validity.
• Criterion-related Validation
– The higher the correlation the more valid is
the measure.
• Construct validity
![Page 20: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/20.jpg)
Types/Tests of Validity II
Measure Measure
Concept A Concept B
Criterion-related V. Construct V.
1b
4b
2b 3b Measure
Criterion
measure
3a
1a
2a
Concept A
![Page 21: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/21.jpg)
Measurement error
![Page 22: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/22.jpg)
Measurement error continued
Measured
value = True value + Bias + Random
error
Measured value what you see or record True value true underlying state of affairs Bias things measured unintentionally - response sets
- yea saying - memory
Random error response errors that are as likely to be big/small + as big/small -
![Page 23: Measurement, validity and reliabilityusers.ox.ac.uk/~sfos0015/rd_5_2017.pdf · Measurement, validity and reliability . Some thoughts…. 'Concepts without percepts are empty...Percepts](https://reader035.fdocuments.net/reader035/viewer/2022071004/5fc17c4f841810372245c976/html5/thumbnails/23.jpg)
To be continued…