Developing your Evaluation Toolkit
Embed Size (px)
Transcript of Developing your Evaluation Toolkit
Assessment and Evaluation
Developing your Evaluation ToolkitFocus on Learning 2013
Paula Buskard and Sandy Odrowski
Paula : Welcome and introductions1Agenda Session OutcomesOverview of what/why we evaluateAligning evaluation methods to outcomesEvaluation PlanningAfter the Evaluation: DebriefingQ & A
Session OutcomesOverview of what/why we evaluateAligning evaluation methods to outcomesThe Test: includes test planning, learner prep, and how to write a test More resources 2Session OutcomesBy the end of this session the participant will have the ability to:
Select suitable evaluation tools that are learnercentred and align to learning outcomes Identify best practice assessment tools that prepare the instructor for writing great evaluations. Identify ways to prepare the learner for evaluation.Discuss ways to conduct effective post test debriefing for both learner and instructor.
Sandy3Place mat ActivityEach member will takes a square/triangle and individually writes 3 questions/concerns they have about evaluation.
At your table you will find a flip chart divided into a placematEach member takes a section and individually writes 2 to 3 questions/concerns they have about evaluation (1 to 2 min)Group discusses and write top 3 questions in the centre Have them put away until end (hopefully we will answer most questions during sessionwe will refer to this at end)4Record the groups top 3 concerns in the centre
Take up and then put aside until end 5Evaluation What is it??Paula6Formativediagnostic tool (Where are learners at? What have they got so far? Do I need to go over something again?)
help learners take stock of and assess their own progress towards learning goals
assess a learners achievement of meeting the learning outcomes
determine gradesPaula8An Analogy of Learning...
9Formative SummativeSandy9Something to think aboutWhat is Needed to know (Learning Outcome)
What is Nice to know(Filler)Paula10What to evaluate? The Course Learning Outcomes describe clearly what learners will know and be able to do at the end of the course.
They are performance-based and results oriented.Sandy11The Golden Chain
Sandy12Snowball ExerciseEach group move to a posted Learning Outcome Take 2 minutes (I will call time) to identify evaluation methods that could be used to measure learning.When time is called, move to the next Learning Outcome and repeat.Once each group has had the opportunity to identify evaluation methods for each Learning Outcome let us know.Share in large group.Paula13What did you learn todayWorking with a partner, fill in the Handout labeled Evaluation Strategy Matching Exercise.
Sandy14Evaluation MethodsAdvantages and Concerns
In your group: Take a moment to discuss the Advantages and Concerns regarding the evaluation method listed on the flip chart paper at your table.
Record your findings and be prepared to discuss with the larger groupPaula to make up flip charts with one evaluation per chart Divide the paper in half (Advantages/Concerns)
After debrief from groupgive out handout15
16The goods Will this be on the test?What do I really have to know?How do I answer this question?Why did I get this question wrong?
Sandy-what students want to know
What do you want your learners to know?Paula18Four Points to Consider:1 Measure what is most important rather than what is easiest to measure
2 - Be aware of the learning embedded in evaluation
3 Understand the evaluators power
4 Take care not to reduce the curriculum to only what is evaluatedPaula19Guidelines for planningStart early (holidays, sequence, scaffolding)
Determine the number of items and time limitations
Construct the question- make sure that it measures learning and is linked to content objectives/outcomes
Edit the questions for errors in spelling, sentence structure, etc.Paula20Guidelines for planning ContdGroup the questions according to type and topic (make easier ones first) Write directions (include marking scheme) Have the answers ready for post test debrief ReviewPaula21How to write a good Evaluation Learner-Centered and Authentic Assessment Checklist
Developing Good Test Questions
Preparing your Learners
How do we prepare our learners
Teaching and Learning Methods align to desired outcome23The Golden chain?Develop a business plan for a small non profit organization.
Lecture and homework: practice multiple choice questions from the text
Evaluation: Final Exam: multiple choice. Course Learning Outcome
Learning and practice methods
Evaluation Method Sandy: activity: entire group-reads this slide and answers whether it follows the golden chainanswer is no24The Golden chain?Develop a business plan for a small non profit organization.
Lecture: types of business and parts of a business plan/. Case studies
Student develop a business plan for a mock or real non profit organizationCourse Learning Outcome
Learning and practice methods
Evaluation Method Sandy: answer is yes25debriefing and feedback
Exercise: In your small groups answer these 2 questions:
What could teachers consider after a test?
What could teachers do to help their learners after a test?
Paula27Teacher Debrief: clean upDid my learners understand what was asked on the test?
Were there questions that the learners missed that were the fault of the teaching? How will I remedy this?
Do the results indicate that the learners were prepared for the evaluation?
Paula28Teacher Debrief: clean up ContdIf I use this procedure again, what changes will I make?
What other methods of evaluation might be more appropriate?
Is this an evaluation that must be returned?
Is there evidence of academic dishonesty? If so, what can I do about it (this time and in the future?)
Paula29Learners DebriefQuickly go over the test and provide the correct answers to all of the questions. Discuss the outstanding questions those that most students answered well, those that some may have got stuck on, etc. Open the floor for focused discussions (learning opportunity). After a reasonable time is spent on a class discussion, invite students to make appointments with you to discuss individual concerns.24 hour rule
Sandy30Evaluation works best whenProvide clear, objective assignment guidelinesProvide samples, if possible, of desired productProvide a marking rubric showing where the marks will come fromTie evaluation strategy directly to the learning outcome being measuredDomain/levelSandy31Any Questions/Concerns left?
Take up and then put aside until end 32Any other Questions?Do you Want to Learn More?Durham College Assessment Using Technology
Eastern Region ABC program Excellent Evaluation Resources
Developing Good Test Questions
Sandy34Do you want to Learn More?Andrus, C and Bloor, D (2005) Teaching tools
Gross-Davis, B. (2001). Tools for teaching. San Francisco: Jossey-Bass
Fenwick T. and Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Toronto: Thompson Educational Publishing