Assessment for learning not marks - Deakin University Blogs · Deakin University CRICOS Provider...

Deakin University CRICOS Provider Code: 00113B

Susie Macfarlane, Dr Dominique Condo & Dr Claire Margerison

Assessment for learning not marks: Designing standards-based rubrics for students’ learning and feedback literacy


Background context


Issues: Unit / course context

1. Students weren’t looking at rubric

2. Students complaining about marks, but were not

focussed on learning or feedback, as focussed on marks

3. There is no clear statement of the standard we expect

students to achieve

4. Students didn’t use feedback to improve their work


Literature:Impact of

rubrics, grades and feedback on

learning


Q1. How does the assessment support students’ understanding of the standard?

Do grades tell us what a student can or can’t do?

Does a pass mark represent the expected standard?

Boud, D. (2017). Standards-based assessment for an era of increasing transparency. In Carless, D., Bridges, S., Chan, C. and Glofcheski, R. (Eds). Scaling up assessment for learning in higher education. Dordrecht: Springer, 19-31.


Marks (out of 100) provide false precision about standard of work

“If only three or four categories of difference can be ascertained, then this is all that can be

legitimately reported. Any finer grain of reporting is spurious and cannot be justified.”

(Boud, 2017)

Q1. How does the assessment support students’ understanding of the standard?


Q2. How does the assessment support learning?

Marks provide very little information the student can

use to improve

Students need rich information



Who sets the standard?

Do students engage with and trying to understand the standard?

Who evaluates the work?

Who provides feedback?

Is feedback a one-way flow of information, or a dialogue?

Do students use the feedback?

Do we know if students’ work improved?

Q3. How does the assessment support students’ active participation?


Definition:

A rubric must have:

• evaluative criteria• quality definitions for those criteria at particular levels, and • a scoring strategy.

Dawson, 2015

Popham, 1997


Rubrics:

• Make expectations and criteria explicit

• Facilitate feedback and self-assessment

• Support students’ self-regulated learning

• Students like rubrics and find them useful

(Jonsson and Svingby, 2007)

(Jonsson and Svingby, 2007)

Jonsson and Pandero, 2017

Reddy and Adrade, 2017


Aims

Aims

Students engage with feedback

Focus students and markers on assessment criteria (standards and

capabilities)(rather than marks)

Students evaluate work against the expected

standard

Feedback is information

about the gap

between the actual level and the reference level

which is used to alter the gap in some way.

(Ramaprasad 1983 p.4, cited in Boud and Molloy, 2013)

Feedback definition:

Aims

Students use feedback to improve their work

Students and markers evaluate work against the expected standard

Focus students and markers on assessment criteria (standards and

capabilities)(rather than marks)


Rubric design

Aim

Focus on standards and capabilities

(rather than marks)

Rewrote assessment criteria: authentic cognitive skills or task that can be evidencedRewrote the rubric with 3 levels, one at the expected standard, clear descriptorsFramed language around development and learning

Strategies

Expected standard

Positive statement of what the student can do


ExcellentProfessionalSophisticatedStrong

GoodAdequate

PoorWeak

Unpacking notions of quality


ClarityAccuracyPrecision

Quality descriptors - cognitive

RelevanceBreadthDepth

LogicSignificanceFairness

FeasibilitySustainabilityEffectiveness Necessity

Discipline or context specific understandings of quality

Elder & Paul (2010) Foundation for Critical thinking

SufficiencyCompleteness ValidityRationality

ConsistencyAuthenticityEfficiency

Fluent, accurate task performanceRespectful client interaction

Correct clinical reasoningAmount of assistance required

Application to new contexts

Specific, contextualised and authentic quality

indicators

Aim


(rather than marks)

Students and markers evaluate work against the expected standard

Students submit a draft and evaluate it using the rubric. Markers also provide feedback.Removed marks from the rubric students used to self evaluate

Strategies


Engaging students in the rubric.


Students create the rubric

Students critique or contribute to the rubric

Students use the rubric to evaluate their own or others’ work

Students use the rubric to evaluate an example

Teacher uses rubric to evaluate an example

Teacher shows high and low quality work

Teacher explains rubric

Nicol (2010) Assess. & Eval. in HE

Student engagement with rubricFr

om lo

wes

t to

high

est l

evel

of p

artic

ipat

ion


Aim



(rather than marks)

Students self review and receive feedback

on draft

Strategies

Students submit a draft and evaluate it using the rubric. Markers also provide feedback on draft.Removed marks from the rubric students used to self evaluate

Feedback Mark 2

Students submit a draft and evaluate it using the rubric. Markers also provide feedback on draft.


Aim


Students resubmit assignment explaining their use of feedbackPD of Unit ChairsTrained markers to provide feedback on the standard student has achieved on Assessment Criteria


(rather than marks)

Students self review and receive feedback

on draft

Strategies

Removed marks from the rubric students used to self evaluate


Impact and Outcomes

Unit Chair perspective

No remark requests since new rubric

Feedback students can use: 50% on draft

Students’ self assessment

Alignment - Markers provide feedback on students’ self evaluation

Confidence as assessor


Response from markers

“As a marker, I felt that self assessed rubrics were very helpful for providing meaningful and individualised feedback.”

“Knowing the level the student felt they were at definitely helped direct feedback and helped to justify any discrepancies between marks given and the students perceived level.”

“Being able to provide feedback based not only on the work the student produced, but also on their planned direction (self identified areas to improve) I believe meant constructive feedback was easier to provide.”


Outcomes: self evaluation

Asking students to self-mark helps fine tune their abilities to review and improve their work and helps me provide more individualised feedback. Self-marking uniquely enables me to provide feedback on how accurately students perceive their own work, for example a student may think that they should focus on citing more references when really they need to provide more details about the references they identify.

By helping students review their own work more accurately they are better able to improve their work independently.

I can also better provide feedback on the skills individual students find the most difficult and want help with e.g. if a student identified that they found it difficult to source and discuss peer reviewed literature I can provide specific feedback about how they did or didn't achieve this and further recommend how to improve. At the end of this process if a student improved on a weakness it is easier for an assessor like me to identify this and commend them on their hard work and show that the effort paid off.


Response from markers

“I think the rubric is easy to use and provide feedback with.It ensures more consistency in the assessment/feedback process and I think makes the expectations clear to the marker and student alike.

However it can sometimes be a little too simple and there is room for interpretation within each of the criteria within the rubric.”


1. Replaced exams with other assessment

2. Made a common structure for rubric

3. Implemented students’ self assessment using the rubric

Course wide changes in M.Diet


Discussion


References

Ajjawi & Boud, D (2018) Examining the nature and effects of feedback dialogue, Assessment & Evaluation in Higher Education, 43:7, 1106-1119, DOI: 10.1080/02602938.2018.1434128

Barton, K, Schofield, S, McAleer, S, Ajjawi, R. (2016) Translating evidence-based guidelines to improve feedback practices: the interact case study, BMC Medical Education, 16: 53

Bearman M and Ajjawi R (2018) From “Seeing Through” to “Seeing With”: Assessment Criteria and the Myths of Transparency. Frontiers in Education, 3: 96, doi: 10.3389/feduc.2018.00096


Boud, D & Molloy, E, (2013) Feedback in Higher and Professional Education: Understanding it and doing it well, Routledge, UK.

Carless, D (2006) Differing perceptions in the feedback process, Studies in Higher Education, 31(2) 219 – 233

Dawson, P. (2015). Assessment rubrics: towards clearer and more replicable design, research and practice. Assess. Eval. High. Educ. 42, 347–360. doi: 10.1080/02602938.2015.1111294

Jönsson A., Panadero E. (2017) The Use and Design of Rubrics to Support Assessment for Learning. In: Carless D., Bridges S., Chan C., Glofcheski R. (eds) Scaling up Assessment for Learning in Higher Education. The Enabling Power of Assessment, vol 5. Springer, Singapore

Hattie, J and Timperley, H (2007) The Power of Feedback, Review of Educational Research, 77: 81-112

https://doi.org/10.1080/02602938.2018.1434128


References

Hounsell, D (2015) Flipping feedback, WISE Assessment Briefing #12, CETL, Available at https://t.co/cq1cek57ui

Nicol, D. (2010). From monologue to dialogue: Improving written feedback processes in mass higher education. Assessment and Evaluation in Higher Education, 35(5), 501-517. doi:10.1080/02602931003786559

Reddy, Y. M., and Andrade, H. (2010). A review of rubric use in higher education. Assessment and Evaluation in Higher Education, 35, 435–448. doi: 10.1080/02602930902862859

Sadler, D. Royce(2009) 'Grade integrity and the representation of academic achievement', Studies in Higher Education, 34: 7, 807 — 826

https://t.co/cq1cek57ui

Key elements of feedbacks

Hattie & Timperley (2007). The power of feedback

Where am I going?

How am I going?

Where to next?

ULOsTask

Rubrics

Rubrics Self evaluation

DraftsPeer feedback

Feedback responseFeedback is used


Programmatic assessment

“Marks and grades may for convenience be aggregated across a programme, but such a process says little about how a student meets the outcomes for a programme as a whole.”



False rigour

“…an explicit model produces outputs (grades) that appear to have

been substantially ‘validated’ through careful attention to all the

steps.

However, the model itself is characterized by indeterminacy, that is

inherently weak. … its implementation creates a veil of rigour that

makes it difficult for learners to question either the process or the

outcome”

Sadler (2009, p.5)

2015 SENS Rubric enhancement project

RUBRIC PRINCIPLES

1. Assessment Criteria that clearly express and allow judgement of the key learning outcomes of the unit, and are have a clear and obvious to connection to ULOs, CLOs and GLOs

2. Clearly describe the observable, measurable learning outcomes the students demonstrate, not content topics

3. Focus the assessment criteria on authentic graduate capabilities and knowledge, not the specifics of a particular assessment task

4. Specifically describe the different levels of performance for each criteria, to give students a clearer understanding of the characteristics of a good performance and what they need to do to improve next time

5. Write the Quality Descriptors to clearly communicate the authentic characteristics of a good performance in the world (e.g. using the Paul and Elder (2010) Universal intellectual standards of critical thinking: clarity, accuracy, precision, relevance, depth, breadth, significance, logic, fairness)

6. Performance level labels refer to the level of achievement, and clearly indicate the level students are expected to achieve.


Issues: M.Diet

Students: High achieving, competitive & marks focussed

Staff: Small course team, willingness to make changes and interested in continuous improvement cycle

Everyone in team, willingness to come to a shared understanding


Course wide standards based assessment

• Introduced competency based assessment across whole course

• No marks are given

• Students collect portfolio of evidence against the DAA competencies

• use portfolio in their mock interviews

Added in transferable skills / GLOs

Assessment for learning not marks - Deakin University Blogs · Deakin University CRICOS Provider...

Documents

Transcript of Assessment for learning not marks - Deakin University Blogs · Deakin University CRICOS Provider...