Lecture 07

COMPILES AND DEVELOPED BY SIR IMRAN ZAIDI

1

LECTURE SEVEN

MEASUREMENT OF VARIABLES: OPERATIONAL

DEFINITION AND SCALES


2

THE RESEARCH DESIGNDETAILS OF STUDY MEASURMENT

Purpose of the study

ExplorationDescriptionHypothesis testing

Types of Investigation

Establishing:-Casual relationships-Correlations-Group differences,

Extent of researcher Interference

Minimum: Studying eventsas they normally occurModerate: Minimum amount of interferenceMaximum: High degree of control and artificial settings

Study setting

Contrived

Noncontrived

Measurement and measures

Operational definitionitems (measure)ScalingCategorizingCoding

Unit of analysis (Population to be

studied)

IndividualsDyadsGroupsOrganizationsMachinesetc.

Sampling design

Probability/nonprobability

Sample Size (n)

Time horizon

One-Shot (cross-sectional)

Multishot (longitudinal)

Data-Collection method

ObservationInterview

Questionnaire

Physical measurementUnobtrusive

1. Feel for data

2. Goodness of data

3. Hypotheses testing

PR

OB

LE

M S

TA

TE

ME

NT

DATA ANALYSIS


3

OPERATIONAL DEFINITION

Reduction of abstract concepts to render them measurable in tangible way is called operationalizing the concepts. It is done by looking at the behavioral dimensions, facets, or properties denoted by the concept.

Reduction of abstract concepts to render them measurable in tangible way is called operationalizing the concepts. It is done by looking at the behavioral dimensions, facets, or properties denoted by the concept.


4

EXPLANATION OF OPERATIONAL DEFINITIONEXPLANATION OF OPERATIONAL DEFINITION

Certain things lend themselves to easy measurement through the use of appropriate measurement instrument, as for example, blood pressure, pulse rate, body temperature, height, weight, etc. The same is true for measuring office floor area. As for example:

• How long have you been working in this organization?• How long have you been working on this particular

assignment?• What is your job title?• What is your marital status?

when we get into realm (particular area) of people subjective feelings, attitudes, and perceptions, the measurement of these factors or variable becomes difficult. So the abstract notation are broken down into observable characteristic behavior i.e., dimensions, and elements

Certain things lend themselves to easy measurement through the use of appropriate measurement instrument, as for example, blood pressure, pulse rate, body temperature, height, weight, etc. The same is true for measuring office floor area. As for example:

• How long have you been working in this organization?• How long have you been working on this particular

assignment?• What is your job title?• What is your marital status?

when we get into realm (particular area) of people subjective feelings, attitudes, and perceptions, the measurement of these factors or variable becomes difficult. So the abstract notation are broken down into observable characteristic behavior i.e., dimensions, and elements


5

EXAMPLE OF DIMENTIIONS AND ELEMENTSEXAMPLE OF DIMENTIIONS AND ELEMENTS

The concept of thirst is abstract: we cannot see thirst. However, we would expect a thirsty person to drink plenty of fluid. If several people say they are thirsty, then we may determine the thirst level of each of these individuals by the measure of the quantity of fluid that they drink to quench their thirst. We will thus be able to measure their level of thirst, even though the concept of thirst itself is abstract and nebulous (unclear).In the above example the thirst is the concept, the drinking of plenty of fluid is the dimension, and the measuring of the quantity of fluid that they drink to quench their thirst is the element.

The concept of thirst is abstract: we cannot see thirst. However, we would expect a thirsty person to drink plenty of fluid. If several people say they are thirsty, then we may determine the thirst level of each of these individuals by the measure of the quantity of fluid that they drink to quench their thirst. We will thus be able to measure their level of thirst, even though the concept of thirst itself is abstract and nebulous (unclear).In the above example the thirst is the concept, the drinking of plenty of fluid is the dimension, and the measuring of the quantity of fluid that they drink to quench their thirst is the element.


6

D

C

Learning

Understanding Retention Application

Answer questionscorrectly

Give appropriateexamples

Recall materialafter some lapse

of time

Solve problemsapplying concepts

understood andrecalled

Integrate withother relevant

material

D D

E E E E E

EXAMPLE OF OPERATIONAL DEFINITION: DIMENTION (D) (INDICATORS) AND ELEMENT (E)

(VARIABLES) OF THE CONCEPT (C) LEARNING


7

METHODS OF SCALES

A scale is a tool or mechanism by which individuals are distinguished as to how they differ from one another on the variable of interest to our study.

There are four basic methods of scales: nominal, ordinal, interval, and ratio. The degree of sophistication to which the scales are fine-tuned increases progressively as we move from nominal to the ratio scale.

A scale is a tool or mechanism by which individuals are distinguished as to how they differ from one another on the variable of interest to our study.

There are four basic methods of scales: nominal, ordinal, interval, and ratio. The degree of sophistication to which the scales are fine-tuned increases progressively as we move from nominal to the ratio scale.


8

NOMINAL SCALE

A nominal scale is one that allows the researcher to assign subjects to certain categories or groups. The categories are also collectively exhaustive (complete). In other words, there is no third category into which respondents would normally fall. Thus the nominal scale gives us some basic, categorical, gross information.

A nominal scale is one that allows the researcher to assign subjects to certain categories or groups. The categories are also collectively exhaustive (complete). In other words, there is no third category into which respondents would normally fall. Thus the nominal scale gives us some basic, categorical, gross information.


9

EXAMPLE OF NOMINAL SCALEEXAMPLE OF NOMINAL SCALE

1. variable of gender, respondents can be grouped into two categories- male female. These two groups can be assigned code numbers 1 and 2.

2. If we had interviewed 200 people, and assigned code number 1 to all male respondents and number 2 to all female respondents, then computer analysis of the data at the end of the survey may show that 98 of the respondents are men and 102 are women. This frequency distribution tells us that 49%of the survey’s respondents are men and 51% women. Other than this marginal information, such scaling tells us nothing more about the two groups.

1. variable of gender, respondents can be grouped into two categories- male female. These two groups can be assigned code numbers 1 and 2.

2. If we had interviewed 200 people, and assigned code number 1 to all male respondents and number 2 to all female respondents, then computer analysis of the data at the end of the survey may show that 98 of the respondents are men and 102 are women. This frequency distribution tells us that 49%of the survey’s respondents are men and 51% women. Other than this marginal information, such scaling tells us nothing more about the two groups.


10

An ordinal scale not only categorized the variables in such a way as to denote differences among the various categories, it also rank-orders the categories in some meaningful way. With any variable for which the categories are to be ordered according to some preference, the ordinal scale would be used. The preference would be ranked (for example: best to worse, first to last etc.) and numbered 1, 2, and so on.

An ordinal scale not only categorized the variables in such a way as to denote differences among the various categories, it also rank-orders the categories in some meaningful way. With any variable for which the categories are to be ordered according to some preference, the ordinal scale would be used. The preference would be ranked (for example: best to worse, first to last etc.) and numbered 1, 2, and so on.

ORDINAL SCALE


11

EXAMPLE OF ORDINAL SCALEEXAMPLE OF ORDINAL SCALERank the following five characteristics in a job in terms of how important they are for you. You should rank the most important item as 1, the next in importance as 2, and so on, until you have ranked each of them 1, 2, 3, 4, or 5.

JOB CHARECTERISTICS RANKING OF IMPORTANCE

The opportunity provided by the job to:

1. Interact with others. 4

2. Use a number of different skills. 2

3. Complete a whole task from

beginning to end.

1

4. Serve others. 5

5. Work independently. 3


12

An interval scale measure the distance between any two points on the scale. This help us to compute the means and the standard deviations of the responses on the variables.

In other words, the interval scale not only groups, it also measures the magnitude of the differences in the preferences among the individuals.

It is more powerful scale than the nominal and ordinal scale, and has for its measure of central tendency the arithmetic mean. Its measure of dispersion are the range, the standard deviation, and the variance.

INTERVAL SCALE


13

EXAMPLE OF INTERVAL SCALEEXAMPLE OF INTERVAL SCALEIndicate the extent to which you agree with the following statements as they related to your job, by circling the appropriate number against each, using the scale given below.

Indicate the extent to which you agree with the following statements as they related to your job, by circling the appropriate number against each, using the scale given below.

Strongly

Disagree

1

Disagree

2

Neither Agree

NorDisagre

3

Agree

4

Strongly Agree

5 The following opportunities offered by the job are very important to me:

a. Interacting with others. 1 2 3 4 5

b. Using a number of different skills. 1 2 3 4 5

c. Completing a task from beginning to end. 1 2 3 4 5

d. Serving others. 1 2 3 4 5

e. Working independently. 1 2 3 4 5


14

The ratio scale overcomes the disadvantage of the arbitrary origin point of the interval scale, in that it has an absolute zero point, which is a meaningful measurement point. Thus the ratio scale not only measures the magnitude of the differences between points on the scale but also tapes the propositions in the differences. It is most powerful of the four scales because it has a unique zero origin (not an arbitrary origin) and subsumes all the properties of the other three scales.The measurement of central tendency of the ratio scale could be either the arithmetic or the geometric mean and the measure of dispersion could be either the standard deviation, or variance, or the coefficient of variation.

The ratio scale overcomes the disadvantage of the arbitrary origin point of the interval scale, in that it has an absolute zero point, which is a meaningful measurement point. Thus the ratio scale not only measures the magnitude of the differences between points on the scale but also tapes the propositions in the differences. It is most powerful of the four scales because it has a unique zero origin (not an arbitrary origin) and subsumes all the properties of the other three scales.The measurement of central tendency of the ratio scale could be either the arithmetic or the geometric mean and the measure of dispersion could be either the standard deviation, or variance, or the coefficient of variation.

RATIO SCALE


15

1. How many other organization did you work for before joining this system? ____

2. Indicate the number of children you have in each of the following categories:____below three years of age____between three to six years____over six years but under twelve years____twelve years and over.

3. How many retail outlets do you operate? ____.

The responses to the questions could range from 0 to any reasonable figure.

1. How many other organization did you work for before joining this system? ____

2. Indicate the number of children you have in each of the following categories:____below three years of age____between three to six years____over six years but under twelve years____twelve years and over.

3. How many retail outlets do you operate? ____.

The responses to the questions could range from 0 to any reasonable figure.

EXAMPLE OF RATIO SCALEEXAMPLE OF RATIO SCALE


16

PROPERTIES OF FOUR SCALES

Highlights Statistical ToolsScales Difference/

CategoryOrder/

Rank

Distance/

Magnitude

Unique Origin

Measures of Central

Tendency

Measure of Dispersion

Some Tests of

Significance

Nominal Yes No No No Mode ________ Chi-square Test

( X2 )

Ordinal Yes Yes No No Median Semi-inter-quartile range

Rank-order

correlation

Interval Yes Yes Yes No Arithmetic

Mean

Standard deviation, variance, coefficient of variation

t, F

Ratio Yes Yes Yes Yes Arithmetic or Geometric

Mean

Standard deviation or variance, or coefficient of variation

t, F


17

EXERCISE-1

Schematically depict the operational definition of the concept of stress and develop 10 question that would measure stress

Schematically depict the operational definition of the concept of stress and develop 10 question that would measure stress

EXERCISE-2Schematically depict the operational definition of the concept of motivation and develop 10 question that would measure motivation

Schematically depict the operational definition of the concept of motivation and develop 10 question that would measure motivation


18

EXERCISE-3

Develop an ordinal scale for consumer preference for different newspaper.Develop an ordinal scale for consumer preference for different newspaper.

EXERCISE-4

Develop a nominal scale of gender for students in Dadabhoy.Develop a nominal scale of gender for students in Dadabhoy.


19

EXERCISE-5

Develop a ratio scale for employees of dadabhoy for absenteeism.Develop a ratio scale for employees of dadabhoy for absenteeism.

EXERCISE-6

Develop a interval scale of effectiveness for students in Dadabhoy.Develop a interval scale of effectiveness for students in Dadabhoy.


20

EXERCISE-7

delineate the dimensions and elements of the concept of waging war in the context of the percent political environment.

delineate the dimensions and elements of the concept of waging war in the context of the percent political environment.

EXERCISE-8

Delineate the dimensions and elements of the concept “intangible assets” of an organization.

Delineate the dimensions and elements of the concept “intangible assets” of an organization.


21

SCALE

Assigning numbers or symbols to elicit the attitudinal responses of subjects toward object, event, or persons is called scale.

Assigning numbers or symbols to elicit the attitudinal responses of subjects toward object, event, or persons is called scale.

TYPES OF SCALES

There are two main categories of attitudinal scales.

1. Rating scale.

2. Ranking scale.

There are two main categories of attitudinal scales.

1. Rating scale.

2. Ranking scale.


22

Rating ScaleRating ScaleRating scales have several response categories and are used to elicit responses or behavioral concept with regard to the object, events, or person studied.

Rating scales have several response categories and are used to elicit responses or behavioral concept with regard to the object, events, or person studied.

Ranking ScaleRanking ScaleRanking scales make comparison between or among objects, events, or persons and elicit the preferred choices and ranking among them.

Ranking scales make comparison between or among objects, events, or persons and elicit the preferred choices and ranking among them.


23

RATING SCALES USED IN ORGANIZATIONRATING SCALES USED IN ORGANIZATION

1. Dichotomous Scale.2. Category Scale.3. Likert Scale.4. Semantic (artificial) Differential Scale.5. Numerical Scale.6. Itemized Rating Scale.7. Fixed or Constant Sum Rating Scale.8. Stapel (basic) Scale.9. Graphic rating Scale.10. Consensus Scale.

1. Dichotomous Scale.2. Category Scale.3. Likert Scale.4. Semantic (artificial) Differential Scale.5. Numerical Scale.6. Itemized Rating Scale.7. Fixed or Constant Sum Rating Scale.8. Stapel (basic) Scale.9. Graphic rating Scale.10. Consensus Scale.


24

1. DICHOTOMOUS SCALE1. DICHOTOMOUS SCALE

The dichotomous scale is used to either a Yes or NO answer. It is a nominal scale. The dichotomous scale is used to either a Yes or NO answer. It is a nominal scale.

EXAMPLEEXAMPLE

Question: Do you have a job?

Answer: Yes / No

Question: Do you like the subject BRM?

Answer: Yes / no

Question: Do you have a job?

Answer: Yes / No

Question: Do you like the subject BRM?

Answer: Yes / no


25

2. CATEGORY SCALE2. CATEGORY SCALE

The category scale uses multiple items to elicit a single response. This is also uses a nominal scale.

The category scale uses multiple items to elicit a single response. This is also uses a nominal scale.

EXAMPLEEXAMPLEWhere in Karachi do you reside?

------------- district east

------------- district west

------------- district south

------------- district malir

------------- district central

Where in Karachi do you reside?

------------- district east

------------- district west

------------- district south

------------- district malir

------------- district central


26

3. LIKERT SCALE3. LIKERT SCALEThe likert scale is designed to examine how strongly subjects agree or disagree with statement on a 5 point scale with anchors. This is an interval scale.

The likert scale is designed to examine how strongly subjects agree or disagree with statement on a 5 point scale with anchors. This is an interval scale.

Strongly Disagree

1Disagree

2

Neither Agree

Nor Disagree

3

Agree

4

Strongly agree

5

EXAMPLEEXAMPLE

Using the Likert scale, state the extent to which you agree with each of the following. Circle your answer.

My work is very interesting. 1 2 3 4 5

I am not engaged in my work all day. 1 2 3 4 5

Life without my work will be dull. 1 2 3 4 5


27

Several bipolar attributes are identified at the extremes of the scale, and respondents are asked to indicate their attitudes, on what may be called a semantic (artificial) space, towards a particular individual, object, or event of each if the attributes. The bipolar adjectives used as Good-Bad, Strong-Weak, Hot-Cold. etc.

Several bipolar attributes are identified at the extremes of the scale, and respondents are asked to indicate their attitudes, on what may be called a semantic (artificial) space, towards a particular individual, object, or event of each if the attributes. The bipolar adjectives used as Good-Bad, Strong-Weak, Hot-Cold. etc.

4. SEMANTIC DIFFERENTIAL SCALE4. SEMANTIC DIFFERENTIAL SCALE

EXAMPLEEXAMPLEResponsive ------------------------------------------- UnresponsiveBeautiful ----------------------------------------------------------- UglyBad ---------------------------------------------------------------- GoodHot ------------------------------------------------------------------ Cold

Responsive ------------------------------------------- UnresponsiveBeautiful ----------------------------------------------------------- UglyBad ---------------------------------------------------------------- GoodHot ------------------------------------------------------------------ Cold

imran


28

5. NUMERICAL SCALE5. NUMERICAL SCALE

The numerical scale is similar to the semantic scale, with the difference that number on a 5 points or 7 points scale are provided, with polar adjectives at both ends. This is also an interval scale.

The numerical scale is similar to the semantic scale, with the difference that number on a 5 points or 7 points scale are provided, with polar adjectives at both ends. This is also an interval scale.

EXAMPLEEXAMPLEResponsive 1 2 3 4 5 6 7 Unresponsive

Beautiful 1 2 3 4 5 6 7 Ugly

Bad 1 2 3 4 5 6 7 Good

Hot 1 2 3 4 5 6 7 Cold


29

6. ITEMIZED RATING SCALE6. ITEMIZED RATING SCALEA 5-point or 7-point (or 9, or whatever) scale with different anchors (e.g., Very Unimportant to Very Important, Extremely low to Extremely high) as needed, is provided for each item and the respondent states the appropriate number on the side of each item, or circle the relevant number against each item. The responses to the items are then summated. This is an interval scale.

A 5-point or 7-point (or 9, or whatever) scale with different anchors (e.g., Very Unimportant to Very Important, Extremely low to Extremely high) as needed, is provided for each item and the respondent states the appropriate number on the side of each item, or circle the relevant number against each item. The responses to the items are then summated. This is an interval scale.

1

Very Unlikely

2

Unlikely

3

Neither Unlikely

Nor Likely

4

likely

5

Very Likely

EXAMPLEEXAMPLERespond to each item using the scale below, and indicate your response number on the line by each item.Respond to each item using the scale below, and indicate your response number on the line by each item.

1. I will be changing my job within the next 12 months. --

2. I will take on new assignments in the near future. --

3. It is possible that I will be out of this organization with in the next 12 months.

--


30

7. FIXED OR CONSTANT SUM SCALE7. FIXED OR CONSTANT SUM SCALE

The respondents are here asked to distribute a given number of points across various items. This is an ordinal scale

The respondents are here asked to distribute a given number of points across various items. This is an ordinal scale

EXAMPLEEXAMPLEIn choosing a toilet soap, indicate the importance you attach to each of the following five aspects by allotting points for each to total 100 in all. Fragrance ----------

Color ----------

Shape ----------

Size ----------

Texture of lather ----------

Total Points 100


31

8. STAPAL (BASIC) SCALE8. STAPAL (BASIC) SCALEThis scale simultaneously measure both the direction and intensity of the attitude toward the item under study. The characteristic of interest to the study is placed at the center and a numerical scale ranging, say from +3 to -3, on either side of the item. This gives the idea of how closer or distant the individual response to the stimulus. Since this does not an absolute zero point, this is an interval scale.

This scale simultaneously measure both the direction and intensity of the attitude toward the item under study. The characteristic of interest to the study is placed at the center and a numerical scale ranging, say from +3 to -3, on either side of the item. This gives the idea of how closer or distant the individual response to the stimulus. Since this does not an absolute zero point, this is an interval scale.

EXAMPLEEXAMPLEState how you would rate your supervisor’s abilities with respect to each of the characteristics mentioned below, by circling the appropriate number.

+3 +3 +3

+2 +2 +2

+1 +1 +1

Adopting modern Technology Product innovation Interpersonal Skills

-1 -1 -1

-2 -2 -2

-3 -3 -3


32

A graphical scale representation helps the respondents to indicate on this scale their answers to a particular question by placing a mark at the appropriate point in the line. This is an ordinal scale. The faces scale, which shows faces ranging from smiling to sad is also a graphic scale, used to obtain responses regarding people’s feelings.

A graphical scale representation helps the respondents to indicate on this scale their answers to a particular question by placing a mark at the appropriate point in the line. This is an ordinal scale. The faces scale, which shows faces ranging from smiling to sad is also a graphic scale, used to obtain responses regarding people’s feelings.

9. GRAPHIC RATING SCALE9. GRAPHIC RATING SCALE

EXAMPLEEXAMPLE

On a scale of 1 to 10 how would you rate

your supervisor?

1

5

10

Good

Better

Best


33

Scales are also developed by consensus, where panel of judges selects certain items, which in its view measure the relevant concept. The items are chosen particularly based on their pertinence or relevant to the concept.

Scales are also developed by consensus, where panel of judges selects certain items, which in its view measure the relevant concept. The items are chosen particularly based on their pertinence or relevant to the concept.

10. CONSENSUS SCALE10. CONSENSUS SCALE


34

1. PAIRED COMPARISON.

2. FORCED CHOICE.

3. COMPARATIVE SCALE.

1. PAIRED COMPARISON.

2. FORCED CHOICE.

3. COMPARATIVE SCALE.

RANKING SCALES USED IN ORGANIZATIONRANKING SCALES USED IN ORGANIZATION

1. PAIRED COMPARISON1. PAIRED COMPARISONIt is used when, among a small number of objects, respondents are asked to choose between two objects at a time.

As the number of objects to be compared increases, so does the number of paired comparisons. The paired choices for n objects will be n (n-1) / 2. The greater the number of objects or stimuli, the greater the number of paired comparisons presented to the respondents, and the greater the respondent fatigue. Hence paired comparison is a good method if the number of stimuli presented is small.

It is used when, among a small number of objects, respondents are asked to choose between two objects at a time.

As the number of objects to be compared increases, so does the number of paired comparisons. The paired choices for n objects will be n (n-1) / 2. The greater the number of objects or stimuli, the greater the number of paired comparisons presented to the respondents, and the greater the respondent fatigue. Hence paired comparison is a good method if the number of stimuli presented is small.


35

It enables respondents to rank objects relative to one another, among the alternatives provided. This is easier for the respondents, practically if the number of choices to be ranked is limited in number.

It enables respondents to rank objects relative to one another, among the alternatives provided. This is easier for the respondents, practically if the number of choices to be ranked is limited in number.

2. FORCED CHOICE2. FORCED CHOICE

EXAMPLEEXAMPLERank the following Companies of Computer that you would like to subscribe to in the order of preference, assigning 1 for the most preferred choice and 5 for the least preferred.

Dell 1

Hewlett-Packard (HP) 3

IBM 2

Toshiba 5

Compaq 4


36

It provides a benchmark or a point of reference to assess attitudes toward the current object, event, or situation under study.

It provides a benchmark or a point of reference to assess attitudes toward the current object, event, or situation under study.

3. COMPARATIVE SCALE3. COMPARATIVE SCALE

EXAMPLEEXAMPLEIn a volatile (evaporation) financial environment, compared to stocks, how wise or useful is it to invest in Treasury bonds? Circle the appropriate response.

More Useful About the Same

Less Useful

1 2 3 4 5


37

GOODNESS OF DATA

It is the surety that the instrument that we develop to measure a particular concept is indeed accurately measuring the variable, and that infect, we are actually measuring the concept perceptual and attitudinal measure. This ensures that in operationally defining perceptual and attitudinal variables, we have not overlooked some important dimensions and elements or included some irrelevant ones.

It is the surety that the instrument that we develop to measure a particular concept is indeed accurately measuring the variable, and that infect, we are actually measuring the concept perceptual and attitudinal measure. This ensures that in operationally defining perceptual and attitudinal variables, we have not overlooked some important dimensions and elements or included some irrelevant ones.


38

RELIBILITYThe reliability of a measure indicate the extent to which it is without bias (error free) and hence ensures consistent measurement across time and across the various items in the instrument. In other words, the reliability of a measures is an indication of the stability and consistency with which the instrument measures the concept and helps to assess the “goodness” of a measure.

The reliability of a measure indicate the extent to which it is without bias (error free) and hence ensures consistent measurement across time and across the various items in the instrument. In other words, the reliability of a measures is an indication of the stability and consistency with which the instrument measures the concept and helps to assess the “goodness” of a measure.

STABILITYThe ability of a measure to remain the same over time-despite uncontrollable testing condition.There are two tests of stability are of most importance.

1. Test-Retest Reliability.2. Parallel Form Reliability.

The ability of a measure to remain the same over time-despite uncontrollable testing condition.There are two tests of stability are of most importance.

1. Test-Retest Reliability.2. Parallel Form Reliability.


39

The reliability coefficient obtained with a repetition of the same measure on a second occasion is called test-retest reliability. That is, when a questionnaire containing some items that are supposed to measure a concept is administered to a set of respondents now, and again to the same respondents, say several weeks to 6 months later, the correlation between the scores obtained at the two different times from one and the same set of respondents is called the test-retest reliability.

The reliability coefficient obtained with a repetition of the same measure on a second occasion is called test-retest reliability. That is, when a questionnaire containing some items that are supposed to measure a concept is administered to a set of respondents now, and again to the same respondents, say several weeks to 6 months later, the correlation between the scores obtained at the two different times from one and the same set of respondents is called the test-retest reliability.

TEST-RETEST RELIABILITYTEST-RETEST RELIABILITY


40

When responses on two comparable sets of measure tapping the same construct are highly correlated, we have parallel-form reliability. Both forms have similar items and the same response format, the only changes being the wordings and the order or sequence of the questions.

When responses on two comparable sets of measure tapping the same construct are highly correlated, we have parallel-form reliability. Both forms have similar items and the same response format, the only changes being the wordings and the order or sequence of the questions.

PARALLEL-FORM RELIABILITYPARALLEL-FORM RELIABILITY


41

INTERNAL CONSISTENCY OF MEASURE

The internal consistency of measures is indicative of the homogeneity of the items in the measure that tap the construct. In other words, the items should “hang together as a set” and be capable of independently measuring the same concept so that the respondents attach the same overall meaning to each of the items.Consistency can be examined through :

1. Inter-item Consistency Reliability.2. Split- Half Reliability.

The internal consistency of measures is indicative of the homogeneity of the items in the measure that tap the construct. In other words, the items should “hang together as a set” and be capable of independently measuring the same concept so that the respondents attach the same overall meaning to each of the items.Consistency can be examined through :

1. Inter-item Consistency Reliability.2. Split- Half Reliability.


42

This is a test of the consistency of respondents answer to all the items in a measure. To the degree that items are independent measures of the same concept, they will be correlated with one another.

This is a test of the consistency of respondents answer to all the items in a measure. To the degree that items are independent measures of the same concept, they will be correlated with one another.

INTER-ITEM CONSISTENCY RELIABILITYINTER-ITEM CONSISTENCY RELIABILITY

SPLIT-HALF RELIABILITYSPLIT-HALF RELIABILITY

Split-half reliability reflects the correlations between two halves of an instrument. The estimates would vary depending on how the items in the measure are split into two halves.

Split-half reliability reflects the correlations between two halves of an instrument. The estimates would vary depending on how the items in the measure are split into two halves.


43

As we know the terms internal validity and external validity. That is we were concerned about the issue of the authenticity of the cause-and-effect relationship (internal validity), and their generalizability to the external environment (external validity). We are now going to examine the validity of the measuring instrument itself. That is when we ask a set of questions (i.e., develop a measuring instrument) with the hope that we are tapping the concept, how can we be reasonably certain that we are indeed measuring the concept we set out to do and not some thing else?We may group validity test under three headings:

1. Content validity.2. Criterion-related validity.3. Construct validity.

As we know the terms internal validity and external validity. That is we were concerned about the issue of the authenticity of the cause-and-effect relationship (internal validity), and their generalizability to the external environment (external validity). We are now going to examine the validity of the measuring instrument itself. That is when we ask a set of questions (i.e., develop a measuring instrument) with the hope that we are tapping the concept, how can we be reasonably certain that we are indeed measuring the concept we set out to do and not some thing else?We may group validity test under three headings:

1. Content validity.2. Criterion-related validity.3. Construct validity.

VALIDITY


44

CONTENT VALIDITYCONTENT VALIDITYIt ensures that the measure includes an adequate and representative set of items that tap the concept. The more the scale items represent the domain (circle of affection) or universe of the concept being measured, the greater the content validity.

It ensures that the measure includes an adequate and representative set of items that tap the concept. The more the scale items represent the domain (circle of affection) or universe of the concept being measured, the greater the content validity.

FACE VALIDITYFACE VALIDITYIt is considered by some as a basic and a very minimum index of content validity. Face validity indicates that the items that are intended to measure a concept, so on the face of it look like they measure the concept.

It is considered by some as a basic and a very minimum index of content validity. Face validity indicates that the items that are intended to measure a concept, so on the face of it look like they measure the concept.


45

It is established when the measure differentiates individuals on a criterion it is expected to predict. This can be done by establishing concurrent ( with consensus) validity or predictive validity

It is established when the measure differentiates individuals on a criterion it is expected to predict. This can be done by establishing concurrent ( with consensus) validity or predictive validity

CRITERION-RELATED VALIDITY

CONCURRENT VALIDITYCONCURRENT VALIDITY

It is established when the scale discriminates individuals who are known to the different; that is they should score differently on the instrument .

It is established when the scale discriminates individuals who are known to the different; that is they should score differently on the instrument .

PREDICTIVE VALIDITYPREDICTIVE VALIDITY

It indicates the ability of the measuring instrument to differentiate among individuals with reference to a future criterion.It indicates the ability of the measuring instrument to differentiate among individuals with reference to a future criterion.


46

It testified to how well the results obtained from the use of the measure fit the theories around which the test is designed. This is assessed through convergent and discriminant validity.

It testified to how well the results obtained from the use of the measure fit the theories around which the test is designed. This is assessed through convergent and discriminant validity.

CONSTRUCT VALIDITY

CONVERGENT VALIDITYCONVERGENT VALIDITYIt is established when the scores obtained with two different instrument measuring the same concept are highly correlated.It is established when the scores obtained with two different instrument measuring the same concept are highly correlated.

DISCRIMINANT VALIDITYDISCRIMINANT VALIDITYIt is established when, based on the theory, two variables are predicted to be correlated, and the scores obtained by measuring them are indeed empirically found to be so.

It is established when, based on the theory, two variables are predicted to be correlated, and the scores obtained by measuring them are indeed empirically found to be so.


47

TESTING GOODNESS OF MEASURES

Goodnessof data

Reliability(accuracy

InMeasure-

ment)

Validity(we are

MeasuringThe right

Thing)

Stability

Consistency

Test-retest reliability

Interitem consistency reliability

Parallel-form reliability

Split-half reliability

Logical validity(content)

Criterion-relatedvalidity

Congruent validity(construct)

Face validity ConvergentPredictive Concurrent Discriminant


48

TYPES OF VALIDITYValidity Description

Content Validity Does the measure adequately measure the concept?

Face Validity Do “experts” validate that the instrument measures what its name suggests it measures?

Criterion-related Validity Does the measure differentiate in a manner that helps to predict a criterion variable.

Concurrent Validity Does the measure differentiate in a manner that helps to predict a criterion variable currently?

Predictive Validity Does the measure differentiate individuals in as manner as to help predict a future criterion?

Construct Validity Does the instrument tap the concept as theorized?

Convergent Validity Do two instruments measuring the concept correlate highly?

Discriminant Validity Does the measure have a low correlation with a variable that is supposed to be unrelated to this variable?


49

THANK YOU FOR YOUR

CONCENTRATION

Lecture 07

Education

Transcript of Lecture 07