Program Evaluation Week 3. Tonight Name Tags Your Q’s Quant. vs. Qual Paradigms activity...
-
Upload
aldous-cooper -
Category
Documents
-
view
219 -
download
0
Transcript of Program Evaluation Week 3. Tonight Name Tags Your Q’s Quant. vs. Qual Paradigms activity...
Program Evaluation
Week 3
Tonight
• Name Tags• Your Q’s• Quant. vs. Qual• Paradigms activity• Evaluation developments• Context-Adaptive Model • HW
Name Tags
• Write first name is big, bold letters• Bring your name tag to every class
Your Q’s
• Find a partner, and share your questions with her.
• We will share these as a class
Quant. vs. Qual
• The field of research and evaluation often categorizes projects/research into two types
• These two categories, often labeled qualitative or quantitative share some similarities but have stark differences.
Activity 1• Find a partner and then make 1 t-chart• Then, categorize the following descriptors under
qualitative or quantitative
• Mainstream research; subjective; product; positivistic; truths; interpretive; researcher involved; causal; objective; alternative research; top-down; bottom-up; grand theory; process; naturalistic; Truth; exploratory; researcher distanced; comparative; social construction; significant difference; confirmatory; particularization; generalization
Evaluation Purposes• Summative– achievement; goals met• Formative– processes; instructional
recommendations
• Often more than one purpose
Paradigms Activity – Which paradigm used?
• Look at these articles, which paradigm(s) is the researcher working from?
• How do the data collected reflect this paradigm?
Evaluation developments • What changes in program evaluation have
occurred over the past 40 years?• Why have these changes taken place?• What does it mean for us?
What changes in program evaluation have occurred over the past 40 years?
• Performance outcomes to study of social processes
• P. 41 Four Generations of Evaluation • Tyler’s Model p. 20 • Experimental design p. 23-24• Why experimental design?
Why this change?
• Parlett & Hamilton (1972). P. 32
• Illuminative evaluation p. 32-33, 34
• Constructivism p. 42
Present
• P. 71• “The attention to issues of language here
parallels our understanding set out in this chapter: from initial positions of evaluation as the measurement of predicated program outcomes, and language as a straightforward representation of meanings and positions, each has matured into a complex account of social life and human experience.”
Context-Adaptive Model
• One way to help prepare evaluations• Consideration of how context guides design • 4 steps:– Audience and goals– Context and themes– Selecting an approach– Selecting a design
1. Audience and goals
• Who is requesting the evaluation?• Who will be effected by its results?• Why is the evaluation being conducted?
Context and Themes
• p. 19
Selecting an approach
• What type of evidence is required to make a convincing evaluation argument?
• P. 22– the basic concepts that link the evidence needed and paradigm include…
Selecting a design
• Is there a need for a large-scale evaluation beyond 1 classroom?
• Experimental or quasi-experimental?• Is the primary focus on individual or group
assessment?
Quantitative- Positivist Designs
• P. 23, 3 features of these designs
• Experimental design— random assignments and control group
• Quasi-experimental—non-random assignment and control group
• Pre-experimental– no control group, but pre vs. post with one group
Interpretivist Designs
• Emergent themes and categories• Responsive model, illumination model,
connoisseurship model
Mixed Designs
• Mixed strategies– using qualitative data to answer quantitative questions, or vice versa.
• Mixed designs or Mixed Model– using both qual and quant in the same evaluation.
Link between context and Design
• Look at vignette 1• Determine the design selected and then
decide what contextual factors influenced that design choice.
p. 39
• Act 1, 2
HW
• Read--4. a. Kiely & Rea-Dickens, Ch 5, Evaluating Teachers’ Competence
• b. Kiely & Rea-Dickens, Ch 6, Evaluating Language through Science
• Journal