Practice- Based Evaluation Research Week 4 Day 2 DIE 4564 Research Methods.
51
Practice- Based Evaluation Research Week 4 Day 2 DIE 4564 Research Methods
-
Upload
cuthbert-blair -
Category
Documents
-
view
226 -
download
0
description
Evaluation Research Evaluation Research – research undertaken for the purpose of determining the impact of some intervention, such as a program aimed at solving a problem.
Transcript of Practice- Based Evaluation Research Week 4 Day 2 DIE 4564 Research Methods.
Practice- Based Evaluation ResearchWeek 4 Day 2
DIE 4564 Research Methods
Chapter Outline• Topics Appropriate for Evaluation Research• Formulating the Problem: Issues of
Measurement• Types of Evaluation Research Designs• The Social Context• Social Indicators Research• Ethics and Evaluation Research• Quick Quiz
Evaluation Research• Evaluation Research – research undertaken
for the purpose of determining the impact of some intervention, such as a program aimed at solving a problem.
Topics Appropriate for Evaluation Research
• Needs assessment studies – studies that aim to determine the existence and extent of problems, typically among a segment of the population.
Topics Appropriate for Evaluation Research
• Cost-Benefit Studies – studies that determine whether the results of a program can be justified by its expense (both financial and other).
http://www.youtube.com/watch?v=Qt8HNNWIV-c
Topics Appropriate for Evaluation Research
• Monitoring Studies – studies that provide a steady flow of information about something of interest, such as medical error rates or the outbreak of an epidemic. Six Sigma – management philosophy that uses
data to eliminating mistakes, waste, and re-work. Sigma = standard deviationSix = goal level of perfection Based on measuring, analyzing, improving,
and controlling
Topics Appropriate for Evaluation Research
• Program Evaluation Studies – the determination of whether a social intervention is producing the intended result. Also called Outcome Assessment Studies
Topics Appropriate for Evaluation Research
• Review Question A researcher wants to determine if changing
what is stocked in school vending machines will improve nutrition intake. This is an example of ______ research.A. needs-assessmentB. cost-benefit C. outcomes assessmentD. survey
Topics Appropriate for Evaluation Research
• Review Question A researcher wants to determine if changing is
what stocked in school vending machines will improve nutrition intake. This is an example of ______ research.The correct answer is C. outcomes
assessment
Formulating the Problem: Issues of Measurement
• Specifying Outcomes• Measuring Experimental Contexts• Specifying Interventions• Specifying the Population• New versus Existing Measures• Operationalization Success/Failure
Formulating the Problem: Issues of Measurement
• Review Question Which one of the below is not one of the
measurement issues in evaluation research?A. Defining the units of analysisB. Measuring the outcomeC. Measuring the stimulus or interventionD. Defining successE. Defining failure
Formulating the Problem: Issues of Measurement
• Review Question Which one of the below is not one of the
measurement issues in evaluation research?The correct answer is A. Defining the units of analysis
Types of Evaluation Research Design
• Experimental Designs
Types of Evaluation Research Design
• Quasi-Experimental Designs – nonrigorous inquiries somewhat resembling controlled experiments but lacking key elements such as pre- and post-testing and/or control groups.
Types of Evaluation Research Design
• Quasi-Experimental Designs Time-Series Design – a research design that
involves measurement made over some period. Nonequivalent Control Groups – a control group
that is similar to the experimental group but is not created by the random assignment of subjects.
Multiple Time-Series Designs – the use of more than one set of data that were collected over time, so that comparisons can be made.
Types of Evaluation Research Design
Time-Series Design
Types of Evaluation Research Design
Time-Series Design
Types of Evaluation Research Design
Time-Series Design
Types of Evaluation Research Design
• Review Question True or False: Evaluation research
requires quantitative analysis of collected information.
Types of Evaluation Research Design
• Review Question False: Evaluation research can use both
qualitative and quantitative analyses of collected information.
The Social Context• Logistical Problems – getting subjects to do
what they are supposed to do. Evaluation research occurs within the
context of real life.
• Discuss example: Motivating people with diabetes
• Discuss example: Administrative control of break-room snacks
The Social Context• Use of Research Results
Why evaluation research results are not always put into practice:Implications may not be presented in a way
that is understandable to the non-researcher.Results may contradict deeply held beliefs.Researchers may have a vested interest in
the results.
The Social Context• Review Question
________ refers to getting research subjects to do what they’re supposed to do.
The Social Context• Review Question
Logistics refers to getting research subjects to do what they’re supposed to do.
Social Indicators Research• Social indicators – measurements that reflect
the quality or nature of social life. Social indicators are often monitored to determine the nature of social change in a society.
Social Indicators Research• Review Question
Which of the below would not be a type of social indicators research?
A. Comparing child obesity rates across states with and without schools physical fitness programs
B. Comparing infant mortality rates across countries with and without nationalized health care
C. Comparing injury rates before and after passage of a employee safety education law
D. Comparing a student’s test grades across different classes in a semester
Social Indicators Research• Review Question
Which of the below would not be a type of social indicators research?The correct answer is D.
Ethics and Evaluation Research• Sometimes the social interventions being assessed
in evaluation research themselves raise ethical issues.
• Evaluation research may entail added pressure to produce specific results, as desired by interested parties.
• Fraudulent research results in an evaluation study can have severer consequences than consequences produced by other types of research.
Quick Quiz
Chapter 12 Quiz
Why has evaluation research grown in popularity?
A. federal requirements demanding program evaluations
B. the availability of research funds to fulfill federal requirements
C. both of the aboveD. none of the above
Chapter 12 Quiz
Answer: C.Evaluation research has grown in popularity because of federal requirements demanding program evaluations, and the availability of research funds to fulfill federal requirements.
Chapter 12 Quiz
_____ aim to determine the existence and extent of problems.
A. Longitudinal studiesB. Needs assessment studiesC. Cost-benefit studiesD. Monitoring studies
Chapter 12 QuizAnswer: B.Needs assessment studies aim to determine the existence and extent of problems.
Chapter 12 Quiz
_____ provide a steady flow of information about something of interest.
A. Longitudinal studiesB. Needs assessment studiesC. Cost-benefit studiesD. Monitoring studies
Chapter 12 Quiz
Answer: D.Monitoring studies provide a steady flow of information about something of interest.
Chapter 12 Quiz
Besides making measurements relevant to the outcomes of a program, research must measure the _____.
A. primary causeB. program interventionC. tertiary effectD. all of the aboveE. none of the above
Chapter 12 Quiz
Answer: B.Besides making measurements relevant to the outcomes of a program, research must measure the program intervention.
Chapter 12 Quiz
A research design that involves measurements made over some period may be defined as
A. time series designs.B. quasi experiments.C. multiple time series designs.
Chapter 12 Quiz
Answer: A.A research design that involves measurements made over some period may be defined as time series designs.
Chapter 12 Quiz
_____ is a form of applied research.
A. EvaluationB. ExperimentationC. InterviewingD. Field research
Chapter 12 Quiz
Answer: A.Evaluation is a form of applied research.
Chapter 12 Quiz
_____ determine whether the results of a program can be justified by its expense.
A. Longitudinal studiesB. Needs assessment studiesC. Cost-benefit studiesD. Monitoring studies
Chapter 12 Quiz
Answer: C.Cost-benefit studies determine whether the results of a program can be justified by its expense.
Chapter 12 Quiz
A key variable for evaluation researchers is to measure the _____.
A. independent variableB. dependent variableC. response variableD. none of the above
Chapter 12 Quiz
Answer: C.A key variable for evaluation researchers is to measure the response variable.
Chapter 12 Quiz
_____ are distinguished from “true” experiments primarily by the lack of random assignment of subjects to an experimental and a control group.
A. Time series designsB. Quasi experimentsC. Multiple time series designs
Chapter 12 Quiz
Answer: B.Quasi experiments are distinguished from “true” experiments primarily by the lack of a random assignment of subjects to an experimental and a control group.
Chapter 12 QuizProfessor Yee wants to do an evaluation study of the effects of a patient education program on patient anxiety. He uses one wing in a hospital for the experiment and compares the results with a similar group of patients in a similar wing in another hospital. Which design would be best?
A. classicalB. nonequivalent control groupC. time-seriesD. posttest-only control group design
Chapter 12 Quiz
Answer: B.Professor Yee wants to do an evaluation study of the effects of a patient education program on patient anxiety. He uses one wing in a hospital for the experiment and compares the results with a similar group of patients in a similar wing in another hospital. The nonequivalent control group design would be best.
Chapter 12 Quiz
Evaluation researchers encounter more logistical problems than other researchers because evaluation research
A. occurs in the context of real life.B. takes longer.C. is more costly.D. has more measurement problems.E. examines more variables.
Chapter 12 Quiz
Answer: A.Evaluation researchers encounter more logistical problems than other researchers because evaluation research occurs in the context of real life.
