Lecture 06

38
COMPILES AND PREPARED BY SIR IMRAN ZAIDI 1 LECTURE SIX ELEMENTS OF RESEARCH DESIGN

description

Lecture 06

Transcript of Lecture 06

Page 1: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

1

LECTURE SIX

ELEMENTS OF RESEARCH DESIGN

Page 2: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

2

THE RESEARCH PROCESSOBSRVATIONBoard area of research

interestidentified

1

PRELIMINARYDATA GATHERING

InterviewingLiterature survey

2

PROBLEMDEFINITION

Researchproblem

delineated

3THEORETICALFRAMEWORK

Variables clearlyidentified and

labeled

4

GENERATIONOF

HYPOTHESES

5

SCIENTIFICRESEARCH

DESIGN

6

DATA COLLECTIONANALYSIS AND

INTERPRETATION

7

DEDUCTIONHypotheses

substantiated?Research question

answered?

8

ReportWriting

ReportPresentation

ManagerialDecisionMaking

No Yes

9 10 11

Page 3: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

3

THE RESEARCH DESIGN

A research design is a plan, structure and strategy of investigation so conceived as to obtain answers to research questions or problems. The plan is the complete scheme or program of the research. It includes an outline of what the investigator will do from writing the hypothesis and their operational implication to the final analysis of data.

Page 4: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

4

THE RESEARCH DESIGN

Purpose of the study

ExplorationDescriptionHypothesis testing

Types of Investigation

Establishing:-Casual relationships-Correlations-Group differences,

Extent of researcher Interference

Minimum: Studying eventsas they normally occurModerate: Minimum amount of interferenceMaximum: High degree of control and artificial settings

Study setting

Contrived

Noncontrived

Measurement and measures

Operational definitionitems (measure)ScalingCategorizingCoding

Unit of analysis (Population to

be studied)

IndividualsDyadsGroupsOrganizationsMachinesetc.

Sampling design

Probability/nonprobability

Sample Size (n)

Time horizon

One-Shot (cross-sectional)

Multishot (longitudinal)

Data-Collection method

ObservationInterview

Questionnaire

Physical measurementUnobtrusive

1. Feel for data

2. Goodness or data

3. Hypotheses testing

PR

OB

LE

M S

TA

TE

ME

NT

DATA ANALYSIS

DETAILS OF STUDY MEASURMENT

Page 5: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

5

An exploratory study is undertaken when not much is known about the situation in hand, or no information is available on how similar problems or research issues have been solved in the past.Exploratory studies are also necessary when some facts are known, but more information is needed for developing a viable theoretical framework.

An exploratory study is undertaken when not much is known about the situation in hand, or no information is available on how similar problems or research issues have been solved in the past.Exploratory studies are also necessary when some facts are known, but more information is needed for developing a viable theoretical framework.

EXPLORATORY STUDY

PURPOSE OF THE STUDY

Page 6: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

6

A descriptive study is undertaken in order to ascertain and be able to describe the characteristics of the variables of interests in a solution. For instance, a study of a class in terms of the percentage of members who are in their senior and junior years, sex composition, age groupings, number of semesters left until graduation, and number of business courses taken, can be considered as descriptive in nature.

A descriptive study is undertaken in order to ascertain and be able to describe the characteristics of the variables of interests in a solution. For instance, a study of a class in terms of the percentage of members who are in their senior and junior years, sex composition, age groupings, number of semesters left until graduation, and number of business courses taken, can be considered as descriptive in nature.

DESCRIPTIVE STUDY

ExampleA bank manager wants to have a profile of the individuals who have loan payments outstanding for 6 months and more. It would include details of their average age, earnings, nature of occupation, full-time/part-time employment status, and the like. This might help him to elicit further information or decide right away on the types of individuals who should be made ineligible for loans in the future.

ExampleA bank manager wants to have a profile of the individuals who have loan payments outstanding for 6 months and more. It would include details of their average age, earnings, nature of occupation, full-time/part-time employment status, and the like. This might help him to elicit further information or decide right away on the types of individuals who should be made ineligible for loans in the future.

Page 7: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

7

Studies that engage in hypotheses testing usually explain the nature of certain relationships, or establish the differences among groups or the independence of two or more factors in a solution.

Studies that engage in hypotheses testing usually explain the nature of certain relationships, or establish the differences among groups or the independence of two or more factors in a solution.

HYPOTHESES STUDY

ExampleA marketing manager wants to know, the sales of the company will increase, if he doubles the advertising dollars. Here, the manager would like to know the nature of the relationship that can be established between advertising and sales by testing the hypothesis: If advertising is increased, then sales will also go up.

ExampleA marketing manager wants to know, the sales of the company will increase, if he doubles the advertising dollars. Here, the manager would like to know the nature of the relationship that can be established between advertising and sales by testing the hypothesis: If advertising is increased, then sales will also go up.

Page 8: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

8

TYPES OF INVESTIGATIONCAUSAL VERSUS CORRELATIONAL

Causal study: The study in which the researcher wants to delineate the cause of one or more problems is called a causal study.Correlational study: When the researcher is interested in delineating the important variables associated with the problem, the study is called a correlational study.

ExampleA causal study question:

Does smoking cause cancer?A correlational study question:

Are smoking and cancer related?OR

Are smoking, drinking, and chewing tobacco associated with cancer? If so, which of these contributes most to the variance in the dependent variable?

ExampleA causal study question:

Does smoking cause cancer?A correlational study question:

Are smoking and cancer related?OR

Are smoking, drinking, and chewing tobacco associated with cancer? If so, which of these contributes most to the variance in the dependent variable?

Page 9: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

9

The extent of interference by the researcher with the normal flow of work at the workplace has a direct bearing on whether the study undertaken is causal or correlational. A correlational study is conducted in the natural environment of the organization with minimum interference by the researcher with the normal flow of work.

EXTENT OF RESEARCHER INTERFERENCE

Page 10: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

10

STUDY SETTING: CONTRIVED AND NONCONTRIVED

Noncontrived settings: If organizational research be done in the natural environment where work proceeds normally, the research is in noncontrived settings.

contrived settings: If organizational research be done in artificial environment the research is in contrived settings.

Correlational studies are invariably conducted in noncontrived settings, whereas most rigorous causal are done in contrived lab settings.

Noncontrived settings: If organizational research be done in the natural environment where work proceeds normally, the research is in noncontrived settings.

contrived settings: If organizational research be done in artificial environment the research is in contrived settings.

Correlational studies are invariably conducted in noncontrived settings, whereas most rigorous causal are done in contrived lab settings.

Page 11: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

11

CONTRIVED AND NONCONTRIVED SETTINGS

1. FIELD STUDY: If various factors are examined in the natural settings in which daily activities going on as normal with minimal researcher interference, the study is field study (noncontrived).

2. FIELD EXPERIMENT: If cause and effect relationships are studied with some amount of researcher interference, but still in the natural settings where work continues in the normal environment, the study is field experiment (contrived).

3. LAB EXPERIMENT: If the researcher explores cause and effect relationship not only exercising a high degree of control but in an artificial and deliberately created settings (contrived).

1. FIELD STUDY: If various factors are examined in the natural settings in which daily activities going on as normal with minimal researcher interference, the study is field study (noncontrived).

2. FIELD EXPERIMENT: If cause and effect relationships are studied with some amount of researcher interference, but still in the natural settings where work continues in the normal environment, the study is field experiment (contrived).

3. LAB EXPERIMENT: If the researcher explores cause and effect relationship not only exercising a high degree of control but in an artificial and deliberately created settings (contrived).

Page 12: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

12

A bank manager wants to analyze the relationship between interest rates and bank deposit patterns of clients. She tries to correlate the two by looking at deposits into different kinds of accounts (such as savings, certificates of deposit, and interest-bearing checking accounts) as interest rates changed.This is a field study where the bank manager has merely taken the balances in various types of accounts and correlated them to the changes in interest rates. Research here is done in a noncontrived setting with no interference with the normal work routine.

A bank manager wants to analyze the relationship between interest rates and bank deposit patterns of clients. She tries to correlate the two by looking at deposits into different kinds of accounts (such as savings, certificates of deposit, and interest-bearing checking accounts) as interest rates changed.This is a field study where the bank manager has merely taken the balances in various types of accounts and correlated them to the changes in interest rates. Research here is done in a noncontrived setting with no interference with the normal work routine.

EXAMPLE OF FIELD STUDYEXAMPLE OF FIELD STUDY

Page 13: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

13

The bank manager now wants to determine the cause-and-effect relationship between interest rate and the inducements it offers to clients to save and deposit money in the bank. She select branches within a 60-mile radius for the experiment. For 1 week only, she advertise the annual rate for new certificates of deposit received during that week in the following manner: the interest rate would be 9% in one branch, 8% in another, and 10% in the third. In the fourth branch, the interest rate remains unchanged at 5%. Within the week, she would be able to determine the effects, if any, of interest rates on deposit mobilization.The above would be a field experiment since nothing but the interest rate in manipulated, with all activities occurring in the normal and natural work environment. Hopefully, all four branches chosen would be more or less compatible in size, number of depositors, deposit patterns, and the like, so that the interest savings relationships are not influenced by some third factors. But it is possible that some other factors might affect the findings. For example, one of the areas may have more retirees who many not have additional disposable income that they could deposit, despite the attraction of a good interest rate. The banker may not have been aware of this fact while setting up the experiment.

The bank manager now wants to determine the cause-and-effect relationship between interest rate and the inducements it offers to clients to save and deposit money in the bank. She select branches within a 60-mile radius for the experiment. For 1 week only, she advertise the annual rate for new certificates of deposit received during that week in the following manner: the interest rate would be 9% in one branch, 8% in another, and 10% in the third. In the fourth branch, the interest rate remains unchanged at 5%. Within the week, she would be able to determine the effects, if any, of interest rates on deposit mobilization.The above would be a field experiment since nothing but the interest rate in manipulated, with all activities occurring in the normal and natural work environment. Hopefully, all four branches chosen would be more or less compatible in size, number of depositors, deposit patterns, and the like, so that the interest savings relationships are not influenced by some third factors. But it is possible that some other factors might affect the findings. For example, one of the areas may have more retirees who many not have additional disposable income that they could deposit, despite the attraction of a good interest rate. The banker may not have been aware of this fact while setting up the experiment.

EXAMPLE OF FIELD EXPERIMENTEXAMPLE OF FIELD EXPERIMENT

Page 14: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

14

EXAMPLE OF LAB EXPERIMENTEXAMPLE OF LAB EXPERIMENTThe bank manager now wants to establish the causal connection between interest rates and saving, beyond a doubt. Because of this she wants to create an artificial environment and trace the true cause and effect relationship. She recruit 40 students who are all business majors in their final year of study and are more or less of the same age. She splits them into four groups and gives each one of them amount of $1,000, which they are told they might utilize to buy their needs or save for the future, or both. She offers them an incentive, interest on what they save but manipulates the interest rates by offering a 6% interest rate on savings for group 1, 8% for group 2, 9% for group 3, and keeps the interest at the lowest rate of 1% for group 4.Here the manager has created an artificial laboratory environment and has manipulated the interest rates for savings. She has also chosen subjects with similar backgrounds and exposure to financial matters (business students). If the banker finds that the savings by the four groups increase progressively, keeping in step with the increasing rates of interest, she would be able to established a cause and effect relationship between interest and the disposition to save. In this lab experiment with the contrived settings, the researcher interference has been maximal, inasmuch as the setting is difficult, the independent variable has been manipulated, and most external contaminating factors such as age and experience have been controlled.

The bank manager now wants to establish the causal connection between interest rates and saving, beyond a doubt. Because of this she wants to create an artificial environment and trace the true cause and effect relationship. She recruit 40 students who are all business majors in their final year of study and are more or less of the same age. She splits them into four groups and gives each one of them amount of $1,000, which they are told they might utilize to buy their needs or save for the future, or both. She offers them an incentive, interest on what they save but manipulates the interest rates by offering a 6% interest rate on savings for group 1, 8% for group 2, 9% for group 3, and keeps the interest at the lowest rate of 1% for group 4.Here the manager has created an artificial laboratory environment and has manipulated the interest rates for savings. She has also chosen subjects with similar backgrounds and exposure to financial matters (business students). If the banker finds that the savings by the four groups increase progressively, keeping in step with the increasing rates of interest, she would be able to established a cause and effect relationship between interest and the disposition to save. In this lab experiment with the contrived settings, the researcher interference has been maximal, inasmuch as the setting is difficult, the independent variable has been manipulated, and most external contaminating factors such as age and experience have been controlled.

Page 15: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

15

Decision points for embarking on an experimental designDecision points for embarking on an experimental designIs tracing causal

effects necessary?

Yesand if

No

Internal validity is more important than

external validity

Generalizability is more important than

internal validity.

Both internal validity and external validity are

important.

Engage in a lab experiment.

Engage in a field experiment.

First do a Lab experiment, then, a FIELD experiment.

Are there cost constraints?

NoYes

Engage in a simpler experimental design.

Engage in a more sophisticated design.

Do not undertake an experimental design study

Page 16: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

16

UNITS OF ANALYSISUNITS OF ANALYSISThe unit of analysis refers to the level of aggregation (bunch) of the data collected during the subsequent data analysis stage.

If the problem statement focuses on how to rates levels of employees in general, then we are interested in individuals employees in the organization and would have to find out what we can do to raise their motivation. Here the unit of analysis is the individual.

If the researcher is interested in studying two-person interactions, then several two-person groups, also known as dyads.

If the problem statement is related to group effectiveness, then the unit of analysis would be at the group level. If we compare different departments in the organization, then the data analysis will be done at the departmental level.

If we compare different organizations, then the data analysis will be done at the organizational level.

If we compare the different cities of any country, then the data analysis will be at the city level.

If we compare the different countries, then the data analysis will be at the country level. etc.etc.

Page 17: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

17

TIME HORIZONTIME HORIZON

One Shot or Cross-Sectional Studies

If data are gathered just once, perhaps over a period of days or weeks or months, in order to answer a research question. are called one-shot or cross-sectional studies.

EXAMPLES1. Data were collected from stock brokers between April and June of last year to study their concerns in a turbulent (beyond control) stock market. Data with respect to this particular research had not been collected before, nor will they be collected again from them for this research.

2. A drug company desirous of investing in research for a new obesity (reduction) pill conducted a survey among obese people to see how many of them would be interested in trying the new pill. This is a one-shot or cross-sectional study to assess the likely demand for the new product.

Page 18: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

18

Multishot or Longitudinal StudiesIf the researcher might want to study people or phenomena at more than one point in time in order to answer the research question or when data on the dependent variable are gathered at two or more points in time to answer the research question, the studies are called longitudinal studies.

For instance, the researcher might want to study employees’ behavior before and after a change in the top management, so as to know what effects the change accomplished. Here, because data are gathered at two different points in time, the study is not cross-sectional or of the one-shot kind, but is carried longitudinally across a period of time.

EXAMPLEOne could study the sales volume of a product before and after an advertisement, and provided other environmental changes have not impacted on the results, one could attribute the increase in the sales volume, if any, to the advertisement. If there is no increase in sales, one could conclude that either the advertisement is ineffective or it will take a longer time to take effect.

Page 19: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

19

EXERCISEIn the following scenarios indicate how the researcher should proceed in each case, that is, determine the following, give reason also:

1. The purpose of study,2. The type of investigation,3. The extent of researcher interference,4. The study settings,5. The time horizon for the study,6. The unit of analysis.

• Scenario AMs. Joyce Lynn, the owner of small business (a women’s dress boutique), has invited a consultant to tell her how business is different from similar small businesses within a 60-mile radius with respect to use of the most modern computer technology, sales volume, profit margin, and staff training.

In the following scenarios indicate how the researcher should proceed in each case, that is, determine the following, give reason also:

1. The purpose of study,2. The type of investigation,3. The extent of researcher interference,4. The study settings,5. The time horizon for the study,6. The unit of analysis.

• Scenario AMs. Joyce Lynn, the owner of small business (a women’s dress boutique), has invited a consultant to tell her how business is different from similar small businesses within a 60-mile radius with respect to use of the most modern computer technology, sales volume, profit margin, and staff training.

Page 20: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

20

• Scenario BMr. pall Hodge, the owner of several restaurants on the East Coast, is concerned about the wide differences in their profit margins. He would like to try some incentive plans for increasing the efficiency levels of those restaurants that lag behind. But before he actually does this, he would like to be assured that the idea would work. He asks a researcher to help him on this issue.

• Scenario BMr. pall Hodge, the owner of several restaurants on the East Coast, is concerned about the wide differences in their profit margins. He would like to try some incentive plans for increasing the efficiency levels of those restaurants that lag behind. But before he actually does this, he would like to be assured that the idea would work. He asks a researcher to help him on this issue.

EXERCISE

Page 21: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

21

EXPERIMENTAL DESIGN

Page 22: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

22

CONTROLLING OF CONTAMINATING FACTORS

When we postulate cause-and-effect relationships between two variables X and Y, it is possible that some other factor, says A, might also influence the dependent variable Y. In such a case, it will not be possible to determine the extent to which Y occurred only because of X, since we do not know how much of the total variation of Y was caused by the presence of the other factor A.

When we postulate cause-and-effect relationships between two variables X and Y, it is possible that some other factor, says A, might also influence the dependent variable Y. In such a case, it will not be possible to determine the extent to which Y occurred only because of X, since we do not know how much of the total variation of Y was caused by the presence of the other factor A.

Page 23: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

23

EXAMPLE OF CONTROLEXAMPLE OF CONTROL

For instance, a Human Resource Development manager might arrange for special training to a set of newly recruited secretaries in creating web pages, However, some of the new secretaries might function more effectively than others, mainly or partly because they have had previous intermittent experience with the web. In this case, the manager cannot prove that the special training alone caused greater effectiveness, since the previous intermittent experience of some secretaries with the web is a contaminating factor. If the true effect of the training on learning is to be assessed, then the learners’ previous experience has to be controlled. This might be done by not including in the experiment those who already have had some experience with the web. This is what we mean when we say we have to control the contaminating factors.

For instance, a Human Resource Development manager might arrange for special training to a set of newly recruited secretaries in creating web pages, However, some of the new secretaries might function more effectively than others, mainly or partly because they have had previous intermittent experience with the web. In this case, the manager cannot prove that the special training alone caused greater effectiveness, since the previous intermittent experience of some secretaries with the web is a contaminating factor. If the true effect of the training on learning is to be assessed, then the learners’ previous experience has to be controlled. This might be done by not including in the experiment those who already have had some experience with the web. This is what we mean when we say we have to control the contaminating factors.

Page 24: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

24

CONTROLLING THE CONTAMINATING EXOGENOUS OR “NUISANCE” VARIABLES

CONTROLLING THE CONTAMINATING EXOGENOUS OR “NUISANCE” VARIABLES

Matching GroupsOne way of controlling the contaminating or “nuisance” variables is to match the various groups by picking the confounding characteristics and deliberately spreading them across groups.

RandomizationIn randomization, the process by which individuals are drawn (i.e., everybody has a known and equal chance of being drawn) and their assignment to any particular group (each individual could be assigned to any one of the groups set up ) are both random.

Page 25: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

25

To what extent would the result found in the lab setting be transferable or generalizable to the actual organizational or field settings? In other words, if we do find a cause-and-effect relationship after conducting a lab experiment, can we then confidently say that the same cause-and-effect relationship will also hold true in the organizational setting?

To what extent would the result found in the lab setting be transferable or generalizable to the actual organizational or field settings? In other words, if we do find a cause-and-effect relationship after conducting a lab experiment, can we then confidently say that the same cause-and-effect relationship will also hold true in the organizational setting?

EXTERNAL VALIDITY

Internal validity refers to the confidence we place in the cause-and-effect relationship with in the lab settings.Internal validity refers to the confidence we place in the cause-and-effect relationship with in the lab settings.

INTERNAL VALIDITY

Page 26: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

26

Sales promotion Sales

Dairyfarmers’ advertisement

Independent variable Dependent variable

Uncontrolled variable

Time: t1 t2 t3

History EffectsCertain events or factors that would have an impact on the independent variable-dependent variable relationship might unexpectedly occur while the experiment is in progress, and this history of events would confound the cause-and-effect relationship between the two variables, thus affecting the internal validity.

FACTORS AFFECTING INTERNAL VALIDITY

Page 27: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

27

Maturation EffectsCause-and-effect inferences can also be contaminated by the effects of the passage of time—another uncontrollable variable. Such contamination is called Maturation effects.

Enhanced technology

Efficiency Increase

Gaining experience and doing the job faster

Independent variable Dependent variable

Uncontrolled variable

Time: t1 t2 t3

Page 28: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

28

Testing EffectsFrequently, to test the effect of a treatment, subjects are given what is called a pretest (say, a short questionnaire eliciting their feelings and attitudes). That is, first a measure of the dependent variable is taken (the pretest), then the treatment given, and after that a second test, called the posttest, administered. The difference between the posttest and the pretest scores is then attributed to the treatment. However, the very fact that respondents were exposed to the pretest might influence their responses on the posttest, which would adversely impact on internal validity.

Instrumentations EffectsInstrumentation effects are yet another source of threat to internal validity. These might arise because of a change in the measuring instrument between pretest, and posttest, and not because of the treatment’s differential impact at the end.

Page 29: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

29

Selection Bias EffectsThe threat to internal validity could also come from improper or unmatched selection of subjects for the experimental and control groups.

MortalityAnother confounding factor on the cause-and-effect relationship is the mortality or attrition of the members in the experimental or control group or both, as the experiment progresses.

Page 30: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

30

Statistical RegressionThe effect of statistical regression are brought about when the members chosen for the experimental group have extreme scores on the dependent variable to begin with. We know from the law of probability that those with very low scores on a variable have a greater probability of showing improvement and scoring closer to the mean on the posttest after being exposed to the treatment. This phenomenon of low scores tending to closer to the mean is known as “regression towards the mean” (statistical regression). Likewise, those with very high abilities would also have a greater tendency to regress towards the mean-they will score lower on the posttest than on the pretest.

Page 31: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

31

Pretest and Posttest Experimental Group DesignAn experimental group (without a control group) may be given a pretest exposed to a treatment, and then given a posttest to measure the effects of the treatment. Where O refers to some process of observation or measurement, X represents the exposure of a group to an experimental treatment, and the X and Os in the row are applied to the same specific group. Here, the effects of the treatment can be obtained by measuring the difference between the posttest and the pretest (O2-O1). Note, however, that testing and instrumentation effects might contaminate the internal validity. If the experiment is extended over a period of time, history and maturation effects may also confound the results.

TYPES OF EXPERIMENTAL DESIGNS

Group Pretest score Treatment Posttest Score

Experimental group O1X O2

Treatment effect = (O2-O1)

Page 32: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

32

Posttests Only with Experimental and Control GroupsSome experimental designs are set up with an experimental and a control group, the former alone being exposed to a treatment and not the latter. The effects of the treatment are studied by assessing the difference in the outcomes-that is, the posttest scores of the experimental and control groups. Here is a case where the testing effects have been avoided because there is no pretest, only a posttest. however, to make sure that the two groups are matched for all the possible contaminating “nuisance” (unwanted) variables. Otherwise, the true effects of the treatment cannot be determined by merely looking at the difference in the posttest scores of the two groups. Randomization would take care of this problem.There are at least two possible threats to validity in this design. If the two groups are not matched or randomly assigned, selection biases could contaminate the results. Mortality (the drop out individuals from groups) can also confound the results,

Group Treatment Outcome

Experimental group

Control group

X O1

O2

Treatment effect = (O2-O1)

Page 33: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

33

Pretest and Posttest Experimental and Control Group DesignsTwo groups-one experimental and the other control-are both exposed to the pretest and the posttest. The only difference between the two groups is that the former is exposed to a treatment whereas the latter is not. Measuring the difference between the differences in the post-and pretest scores of the two groups would give the net effects of the treatment. Both groups have been exposed to both the pre-and posttests, and both groups have been randomized; thus we could expect that the history maturation, testing, and instrumentation effects have been controlled. This is so due to the fact that whatever happened with the experimental group (e.g., maturation, history, testing, and instrumentation) also happened with the control group, and in measuring the net effects (the difference in the differences between the pre-and posttest scores) we have controlled these contaminating factors. Through the process of randomization, we have also controlled the effects of selection biases and statistical regression. Mortality could, however, pose a problem in this design. In experiments that take several weeks, as in the case of assessing the impact of training on skills development, or measuring the impact of technology advancement on effectiveness, some of the subjects in the experimental group may drop out before the end of the experiment. It is possible that those who drop out are in some way different from those who stay on until the end and take the posttest. If so, mortality could offer a plausible (apparently valid) rival explanation for the difference between O2 and O1.

Page 34: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

34

Group Pretest Treatment Posttest

Experimental group Control group

O1

O3

X O2

O4

Treatment effect = [(O2-O1) - (O4-O3)]

Pretest and posttest experimental and control groupPretest and posttest experimental and control group

Page 35: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

35

To gain more confidence in internal validity in experimental design, it is advisable to set up two experimental groups and two control groups for the experiment. One experimental group and one control group can be given both the pretest and the posttest. The other two groups will be given only the posttest. Here the effects of the treatment can be calculated in several different ways. To the extent that we come up with almost the same results in each of the different calculations, we can attribute the effects to the treatment. This increases the internal validity of the results of the experimental design. This design, known as the Solomon four-group design, is perhaps the most comprehensive and the one with the least number of problems with internal validity.

SOLOMON FOUR GROUP DESIGN

Page 36: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

36

.

Group Pretest Treatment Posttest

1. Experimental

2. Control

3. Experimental

4. Control

O1

O3

X

X

O2

O4

O5

O6

Treatment effect (E) could be judged by:

E= (O2-O1)

E= (O2-O4)

E= (O5-O6)

E= (O5-O3)

E= [(O2-O1) - (O4-O3)]

If all Es are similar, the cause-and-effect relationship is highly valid.

SOLOMON FOUR GROUP DESIGN MODELSOLOMON FOUR GROUP DESIGN MODEL

Page 37: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

37

Solomon Four-Group Design and Threats to Internal Validity

Let us examine how the threats to internal validity are taken care of in the Solomon four-group design. It is important to note that subjects have been randomly selected and randomly assigned to groups. This removes the statistical regression and selection biases. Group 2, the control group that was exposed to both the pre-and posttest, helps us to see whether or not history, maturation, testing, instrumentation, regression, or mortality threaten internal validity. If scores O3 and O4 (pre-and posttest scores of group 2) remain the same, then it is established that neither history, nor maturation, nor testing, nor instrumentation, nor statistical regression, nor mortality has had an impact. In other words, these have had no impact at all.

Page 38: Lecture 06

COMPILES AND PREPARED BY SIR IMRAN ZAIDI

38

THANK YOU FOR YOUR

CONCENTRATION