Learner Assessment in an Online Environment
description
Transcript of Learner Assessment in an Online Environment
Learner Assessment in an Online EnvironmentPatricia McGee, Alisa Cooper, Jennifer Freeman
1. Developed my own assessments
2. Used publisher-created assessments
3. Worked with others to create assessments
4. Used assessments developed by someone other than publisher
Which best describes your experience with online assessments?
Overview of Workshop
Classroom vs. Online Assessment Best Fit Assessments for Online Courses Designing with the End in Mind Giving Assessment Directions &Feedback Rubrics and Checklists Involving the Learner
Asse
ssm
ent T
oolk
it
P. McGee
Classroom vs. Online Assessment
Patricia McGee
Instructional Functions of Assessment
One to many Just in time
assessment Immediacy Focus on methods
One to one Just in need
assessment Ambiguity Focus on strategies
Differences? Time Security Integrity Resources Learner autonomy Lack of just in need
responses Getting, keeping, and
directing attention Clarity of instructions
and expectations Other?
Challenges?
Cheating Randomized objective
quizzes/tests Timed and authenticated Multiple ways to demonstrate
understanding Choices Duplication Repetition Revise and resubmit
Small group assessments for accountability
Communication Detailed expectations, e.g.,
rubrics
Respond within a pre-designated timeframe
Peer critique review before final submission
Benchmarks, points of reflection, accomplishment & completion
FERPA?
Activity
Brainstorm for assessment challenges.
Discussion possible solutions
Be prepared to share a particularly difficult challenge.
Asse
ssm
ent T
oolk
it
P. McGee
Progressive Assessment & BenchmarksPatricia McGee
Learners should know….
That an assessment is coming A word about helping students plan
What happens after the assessment How to prepare for an assessment What participating in the assessment will look
like Practice
Classroom (online) Assessment Techniques
Informal Ungraded Low threat and no risk Inform instructor about student learning Inform learner about learning progress
From Classroom Assessment Techniques
CATS Examples: Prior KnowledgeCATBackground Knowledge Probe: short, simple questionnaires prepared by instructors for use at the beginning of a course or at the start of new units or topics; can serve as a pretest
Online ApplicationSurvey or Poll Survey Monkey™ Poll Everywhere™ PollDaddy™
CATS Examples: Prior KnowledgeCATEmpty Outlines: in a limited amount of time students complete an empty or partially completed outline of an in-class presentation or homework assignment
Online ApplicationShared Notetaking/Concept MappingBubblus™ Mind42™ Mindomo™ Wordle™
5 Suggestions for Online CATs
1. Customize to specific needs and learning environment
2. Be consistent with the instructional philosophy
3. Test out a CAT and assess effectiveness
4. Allow extra time to carry out and respond to the assessment
5. Let students know what you learn from their feedback and how you and they can use that information to improve learning
ACTIVITY Review the CATs and pick
one Identify the goal for your
CAT Explain why this CAT is
helpful/necessary in this particular area of the course
How and when will students receive feedback on the CAT?
Progressive Assessment
Online Progressive Assessment
Interaction Assignments Activities Participation
Performance vs Objective Assessments
Objective Classroom
Assessment Techniques
Multiple choice tests Embedded Test
Questions Pre and Post Tests Discussion Threads
Performance Activity-Based
Cases Projects Debates/
negotiations Production
Field experience Products Presentations Research paper
Example: Periodic Assessments
24
Example: benchmarks
25
Example: progressive benchmarks
26
Example: Assignment
BENCHMAR
K?
Activity: Progressive Assessments
What are most comprehensive benchmarks for a course?
What are challenges? Issues?
Design with the end in mindAlisa Cooper
Provide multiple and alternative modes of assessments
Learning and AssessmentCycle
Bloom’s Taxonomy
Backwards Design
Focus on learner
Focus on measure of learning
Assessment & Bloom’sProcesses Tools Attributes
Remember Recognizing, recalling Visual/Text/Audio stimuli, selecting, feedback
Understand Interpreting, classifying, comparing, summarizing, explaining
Sorting, tagging, labeling, entering, selecting
Apply Executing, implementing Manipulating, entering, feedback
Analyze Differentiating, organizing, attributing
Selecting, grouping, altering, tagging, labeling
Evaluate Checking, critiquing Commenting, entering, responding
Create Generating, planning, producing
Adding, generating, combining, publishing
ACTIVITY Using handout,
determine at what level of Bloom’s you are assessing
Explore tools that might allow you to assess at higher levels of Bloom’s see http://elearningtools.wetpaint.com
Giving Assessment Directions & FeedbackJenni Freeman
Assessment Instructions
This seems like a fundamental concept, but many assessments lack adequate instructions,
leading students to answer questions incorrectly or without the desired depth.
Two levels: How to use the tools, logistics Student expectations
Assessment Instructions: Questions to Answer
Objective Tests Using the tool? How many times available? When available? Results available? Other?
Assessment Instructions: Questions to Answer
Subjective Assessments How to submit? When due? How will it be graded? What does a successful paper look
like? Results available? Other?
Assessment Instructions: Best Practices• Separate general instructions (How to use the tools, logistics) from assessment-specific instructions (due dates, rubrics)
Centrally located pop-up Syllabus Designated discussion forum Provide grading rubrics Provide models if appropriate (an “A”
paper and an “F” paper)
Assessment Feedback
The sooner the better Instant results (objective tests) Auto-reply upon receipt (subjective assessments)
Details Feedback for all distracters, not just the wrong answers
(objective - students who guess correctly will also receive benefit)
Compose boilerplate language and phrases for common comments to students (subjective)
Assessment Feedback
The sooner the better Instant results (objective tests) Auto-reply upon receipt (subjective assessments)
Details Feedback for all distracters, not just the wrong answers
(objective - students who guess correctly will also receive benefit)
Compose boilerplate language and phrases for common comments to students (subjective)
ACTIVITY
Write instructions for your sample assessment keeping all of the above in mind. Share with your table.
What sort of feedback will your students need after completing the assessment?
Frequency of Feedback
Pedagogical Timing After practice
At completion/achievement of objective
Calendar Schedule Indirect – weekly
Direct - bi-monthly
For exampleChat – content learning Answer questions
correctly
Ask questions that Helped others learn Extended discussion
Make contributions to discussion that extended materials
Chat - performance Participate throughout
discussion
Do not interrupt
Respond when called upon
When appropriate, come to chat prepared.
Outcome + Criteria = Assessment1. How are you communicating that to learners?
2. How will you document the learning ?
3. Do you expect learner to learner a new tool for assessment ?
4. Have you provided for practice before assessment?
Using Rubrics Allows assessment to be more objective and consistent
Focuses instruction to clarify criteria in specific terms
Clearly shows the student how their work will be evaluated and what is expected
Rubric Tips
Ensure that the assessment criteria can be directly correlated to the course objectives.
Break the criteria down as specifically as possible and clearly define exemplary, satisfactory, and unsatisfactory efforts for each.
Be sure that students have access to the rubric before they begin the assignment.
Activity Review samples of rubrics. Identify the characteristics that need to be evaluated in
your assessment. Describe the best work you could expect using these
characteristics: top category Describe the worst acceptable product using these
characteristics: lowest category Describe intermediate work using these characteristics Build a rubric online using Rubistar
http://rubistar.4teachers.org Share your rubric with your table.
Involving the LearningAlisa Cooper
Peer Assessment
Contributes to…. Ownership and deeper
learning
Motivation
Independent and creative thinking
Self awareness of learning as compared to that of others
Strategies Participation in peer
assessment scales, processes
Rating
Peer Critique/Review
Mentoring/tutoring/coaching
Rubrics: for drafts, in conjunction with instructor assessment
Self-assessment
Incorporate meta-cognitive assessments
Provide a strategy for self-assessment and progress
Track progress Compare work to that
of others Grade own
performance
Self-Assessment
Techno CATs
Before/after unit
Reflection
Feedback on design
Feedback on technology
Increases accountability
Engages students
Starts discussions
Early alert
Practice
Progressive Benchmarks for the Learner
Examples
1. What was the one most useful thing you learned in this assignment, unit or module?
2. What suggestions would you give other students on ways to get the most out of this assignment, unit or module?
3. In what area did you learn or understand the most? Least? 4. List three ways you think you have developed or grown as a result of
this assignment, unit or module?5. What did you learn about writing, research, (or any other skill) from
this assignment, unit or module?6. What problems did you encounter in this assignment, project, unit, or
tool that was used?7. What unit/module of this course was your best work and why?
Team Assessment
Roles Teams
Final outcome against pre-determined criteria
Completion Performance against
other teams
Individual
Contribution
Objective assessments
Accountability Require pre-group work Require group members to
express individual opinions and monitor via another member
Include peer evaluation in grading Readiness Assurance Process
Test over readings Group: Test, discuss, reach
consensus and retest Provide information for peer
feedback process
Team Teaching Tips
Outline learning goals Teach team skills Clear and detailed
instructions Rubric Stages of team
development Forming - polite but
untrusting Storming - testing others Norming - valuing other
types Performing - flexibility
from trust 59
60
Team Contracts
Purpose, goals, and missions
Expectations
Roles
Conflict resolution strategies
Meetings
Communication
Decision-making policy
Agendas
Record-keeping
Poll/Survey Options
Functions Engage – collect data –
generate conversation
Informal assessment of understanding/comprehension
Formal assessment of critical thinking
Strategy Ideas Random questions for
points
Drill & practice activities with participation points (online quizzes)
Student constructed – collaborative/cooperative
Matrix survey
http://elearningtools.wetpaint.com/page/Polling%2C++Surveying
Take Aways
What NOT to do
Require a different or separate assessment for each use of tool
Assume the learner knows what to do Assume the fun factor over rides the learning
requirements
Evaluate your assessment practices regularly
Are their persistent problems? Is there an area of the course that gets “stuck” and
remediation or review is needed? Is a lot of time spent on answering questions or
clarifying?