Assessment Planning: Why,When,What?Maristela Petrovic-Dzerdz, Instructional Designer

Including Q&A Session with Dr. Anthony Marini, Senior Teaching Development Associate

Educational Development CentreMay 2015

Assessment Planning: Why,When,What?Educational Development CentreMay 2015

Maristela Petrovic-Dzerdz, MEng, BEd, PBCIDInstructional Designer

This presentation is based on:Smith P.L. & Ragan T. J. (2005) Instructional Design (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.

Post-workshop Bring in 2-3 assessment items you use in your

course (example of an essay question, multiple choice question, problem item, etc.) including the description of the assignment, for a more detailed analysis and group discussion.

Agenda

Why to assess? When to design assessment? What to consider when designing

assessment? When to assess? What to assess?

Case study: Training on creating effective visuals

Case study: Learning Outcomes

Given a description of the audience characteristics, purposes of the presentation, and a source materials, the learner will be able to develop a presentation that incorporates the principles of visual design and good lettering.

Given a description of the audience characteristics, purposes of the presentation, and a handout developed for a presentation, the learner will be able to evaluate the handout based upon the principles of visual design, effective lettering, and appropriate use of the medium, and to suggest revisions.

Case study: One summative assessment50-item test, open-ended and multiple-choice questions

List and discuss the eight principles of visual design

Which of the following is not a characteristic of the design element “surface”? A. texture B. color C. value D. form

Case study: Test results Instructor was happy with grade distribution

WHY (to assess)? Stakeholders

Teacher/course designer

Students

Others (institution, employers, parents, etc.)

Student perspective

Student behaviour is driven largely by assessment“How can I meet the assessment requirements most effectively (in the least amount of time and with the least effort)”?

Course design questions

Where are we going? How will we get there? How do we know when we’re there?

WHEN (to design assessment)?

Assessment

Learning outcome

s

Instructional

strategies

Course design sequence

1. Write learning objectives

2. Design assessment items (part of task analysis)

What will constitute evidence that the learners have learned?

How should students demonstrate their knowledge? Are there skills to assess together with content?

Learning

objectives

Assessmen

t

Why writing assessment items immediately after writing objectives?

The intentions of the objectives will be fresh in your mind

If you cannot write an item for the objective, maybe you need to reexamine the objective

Learning

objectives

Assessmen

t

Robert Gagne: “Conditions of Learning” (1965).

McTighe and Wiggins: “Backward design” (1988).

John Biggs: “Constructive alignment” (1999).

Dee Fink : “Significant learning” (2003)

Takeaway 1

Try to think of assessment as soon as you know the course goals and learning outcomes

Adequate or passing performance

Adequate or passing performance % of items learners must get correct “Learners must get at least one correct out of

each of the easy, medial and difficult categories”

Designing assessment blueprint:

Takeaway 2

As you start designing assessment, think about the desired level of performance. Is the required passing level ensuring the minimal acceptable performance?

WHAT & WHEN (to assess)

A1

A2A3

Types of assessment - WHEN Entry Skill Assessments

Skills and knowledge one must have to begin instruction

Preassessment What learners already know about the topic that

they are to learn about though instruction Can be used to gain students’ attention and to

inform of the objective(s) of the lesson Focus on the portion of the instruction not

previous learned

Postassessment

Types of assessment – FOR WHAT PURPOSE

Formative Assessment How is the student progressing? Combine with

feedback – corrective measures. The purpose is LEARNING

Summative Assessment What did the student learn in this course? Can

student combine the knowledge and skills and demonstrate performance at the expected level? The purpose is to determine the LEVEL of COMPETENCE

WHAT to assess?

Learning to drive

Learning Domainsbased on the characteristics of the “content” that a learner must learn

Verbal information

(declarative knowledge)

Intellectual skills• individual

competence and ability to respond to stimuli

Cognitive strategies• capability to

learn, think, and remember

Attitudes• ingrained bias

towards different ideas, people, situation

Psychomotor skills• capability to

learn to drive, ride a bike, draw a straight line, etc.

Robert Gagné1916-2002“Conditions of

Learning” 1965.

“Most fundamental, most widely used and most useful in designing instructional materials” (Smith & Ragan, 2005.)

WHAT (to assess)?

Verbal information (declarative knowledge) Intellectual skills (procedural knowledge) Cognitive strategies Attitudes Psychomotor skills

+ Enterprise schema and Integrated goals (With D. Merrill)

WHAT (to teach and assess)?

Verbal information (declarative knowledge) Intellectual skills (procedural knowledge) Cognitive strategies Attitudes Psychomotor skills

Declarative Knowledge: “Knowing that…”

Recall in verbatim, paraphrased, or summarized form facts, lists, names, or organized information

Recall, recognize, state in own words

Knowing declarative knowledge helps students learn higher-order, more complex objectives, such as intellectual skills

WHAT (to teach and assess)?

Verbal information (declarative knowledge) Intellectual skills Cognitive strategies Attitudes Psychomotor skills

Intellectual skills: “ Knowing how…”

The application of rules to previously unencountered examples

Intellectual Skills: Subcategories

Discrimination

Concepts

Principles (if-then)

Procedures

Problem Solving

Discrimination: learning to differentiate, to perceive that something either matches or differs from other things

The acquisition of concepts helps the learner to simplify the world (responds to things as members of “groups”)

Which of a number of figures is a rhombus?

Principles (if-then) help us predict, explain, control

Procedures: In what order certain steps should be taken (and why)

Problem Solving: Learned capability involving selection and application of multiple rules

Proving a geometric theorem (learning how to respond to a class of problems)

Takeaway 3 Are you assessing “Knowing that” or “Knowing

how” (or both)?

If you are assessing “knowing how”, are you PROGRESSIVELY assessing skills and knowledge in Discrimination Concepts Principles Procedures Problem solving

WHAT (to assess)?

Verbal information (declarative knowledge) Intellectual skills Cognitive strategies Attitudes Psychomotor skills

Cognitive strategies (knowing how to learn)

Rehearsal strategies (select, recall) Elaboration strategies (tie new information

to prior knowledge) Organizational strategies (define the

relationship among information) Metacognition (knowledge about own

cognitive processes and ability to control them)

Affective strategies (for focusing attention, maintaining concentration, establish and maintain motivation, manage time)

WHAT (to assess)?

Verbal information (declarative knowledge) Intellectual skills Cognitive strategies Attitudes Psychomotor skills

Attitudes (“How to behave”)

Attitudes influence the choices that learners make

Instruction in attitudes often subtle and indirect

Simulation games

WHAT (to assess)?

Verbal information (declarative knowledge) Intellectual skills Cognitive strategies Attitudes Psychomotor skills

Psychomotor Skills Coordinated muscular movements that are

typified by smoothness and precise timing

Learning to drive

Takeaways

Plan for assessment as soon as you know the course goals and intended learning outcomes

Consider the purpose of assessment and when in the course it should happen

Think about the expected “performance” – what is a desired level? What is a passing level?

Takeaway (cont’d)

Design “assessment blueprint” aligned with intended learning outcomes

There are different domains of learning and they should be assessed accordingly

References

Smith P.L. & Ragan T. J. (2005) Instructional Design (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.

Gagné, R.M. (1985). The Conditions of Learning and Theory of Instruction (4th ed.). New York: CBS College Publishing.

Bloom B.S. , Hastings, J.T., Madaus, G.F. , (1971) Handbook on formative and summative evaluation of student learning. McGraw-Hill, New York.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education 32:347- 364.

Wiggins G. and McTighe J., (2005). Understanding by Design. Alexandria, VA : Association for Supervision and Curriculum Development.

Assessment Planning: Why,When,What?May 2015, EDC, Carleton University

Maristela Petrovic-Dzerdz, MEng, BEd, PBCIDInstructional Designer

Thank You

Q&A Session with Dr. Anthony Marini3M Teaching Fellow

EDC Senior Teaching Development Associate

May 2015

Question 1 “I have a question about the expected

performance and desired level. Today we hear students say that if you have paid your tuition, you should expect to pass the course. We had in the past cases when (and it was also in newspapers) too many students fail the course, so the question was raised if the ‘clients’ pay, yes they should expect a good course design, but should they also expect to pass the course because they paid for it?”

Question 2 “What to do if you have a graduate student who,

you realize, does not have skills necessary at the graduate level? We know the rule that you should not get less than B- in a graduate course. A couple of times in my career I had to give a graduate student less than B- , and in one particular case I had to give student, who was about to graduate, a D- ; I was very concerned about this person going into the world with masters degree and Carleton University credentials and I wanted to send a strong message. The student complained, appealed…It is very difficult. What should be the passing level? Maybe I have not been explicit about the passing level up front? “

Question 3 “The ‘safe and effective’ thing resonates

with me, since I teach engineering. If someone gets D- in my engineering design course, I wouldn’t trust him to design a house for my dog. From course design, from best educational practices perspective, it makes a lot of sense; but in university environment, do I have a freedom to choose what the passing level should be and not allow for a 50% to be a passing level? “

Question 4 “In our department we had a discussion

because it seems that the grades are too high. Some instructors say that the students put so much effort, came to all the classes, all the office hours…How much can we justify grading the effort as to grading the actual level of performance in the course? “

Question 5 “In Europe, we have that maybe 20% of

your mark is effort and participation and the 80% is actual performance. Maybe this is a solution?“