What is current research telling What is current research telling us about assessment and us about assessment and

feedback in higher education?feedback in higher education?

TEANTEAN

‘ ‘there is more leverage to improve there is more leverage to improve teaching through changing teaching through changing assessment than there is in changing assessment than there is in changing anything else’ anything else’ (Gibbs & Simpson, 2004:22)(Gibbs & Simpson, 2004:22)

markingmarking

formative assessmentformative

assessment

Academic engagementAcademic

engagement

Changing views of feedback

Changing views of feedback

Criteria(e.g. LOs, prof.

standards, grade descriptors

Criteria(e.g. LOs, prof.

standards, grade descriptors

transparencytransparency

Peer & self assessmentPeer & self assessment

Learning –oriented assessme

nt

Learning –oriented assessme

nt

Skills of judgementSkills of

judgement

moderationmoderation

Need for dialogueNeed for dialogue

Tension with

standards

Tension with

standards

Innovative assessment

methods

Innovative assessment

methods

Computer assisted

assessment

Computer assisted

assessment

& assessment discourse

Assessment of professional competency

Student-tutor power

relationship

Holistic v

analytical

Professional judgement comes from:

‘repeated engagement in the appropriate activities in the company of those with expertise’ (Arnal & Burwood (date): 388)

This is a ‘community of practice’ approach to learning academic standards.

We learn informally, through participation; through being partners in the assessment processes, not through being instructed.

(O’Donovan & Rust, 2008).

For

mal

act

iviti

es a

nd in

puts

Passive Student engagement

Active student engagementinform

al activities and inputs

1. The traditional model

Tacit standards absorbed informally and serendipitously

3. The Social Constructivist Approach

Active methods used to communicate tacit knowledge, e.g. marking exemplars using criteria

2. Explicit model

Standards explicitly articulated and passively presented to students

4. The ‘cultivated’ community of practice approach

Tacit standards communicated through participation in informal knowledge exchange ‘seeded’ by activities

The past

The future

Adapted from O’Donovan et al (2008)

Approaches to developing student understanding of assessment criteria

markingmarking

formative assessmentformative

assessment

Academic engagementAcademic

engagement

Changing views of feedback

Changing views of feedback

Criteria(e.g. LOs, prof.

standards, grade descriptors

Criteria(e.g. LOs, prof.

standards, grade descriptors

transparencytransparency

Peer & self assessmentPeer & self assessment

Learning –oriented assessme

nt

Learning –oriented assessme

nt

Skills of judgementSkills of

judgement

moderationmoderation

Need for dialogueNeed for dialogue

Tension with

standards

Tension with

standards

Innovative assessment

methods

Innovative assessment

methods

Computer assisted

assessment

Computer assisted

assessment

& assessment discourse

Assessment of professional competency

Student-tutor power

relationship

Holistic v

analytical

Transparent standards?

• The drive for transparency is based on the theoretical idea that if students know what standards they are being judged against, they are more able to fill the gap between their present performance and those standards (Sadler 1989), and marking will be fairer and more reliable.

Programme assessment environment

Factors• How many exams• Variety of ass. methods• How much summative

assessment• How much formative

assessment• How much oral feedback• How much written feedback• Timeliness of feedback• Explicit criteria & standards• Alignment between outcomes

& assessment

Can influence• Student effort• How much of the syllabus they

cover• Usefulness of feedback• Use of feedback by students• Whether students know what

is expected of them• Whether they focus on deep or

surface approaches to learning• Whether exams encourage

learning

From Gibbs & Dunbat-Goddet (2007)

Feature of assessment environment

Scales from AEQ (v3.3) with significantly lower scores

High variety of assessment methods

Quantity and quality of feedback;

Use of feedback;

Appropriate assessment;

Clear goals and standards;

Deep approach to learning;

Learning from the examination

Satisfaction

High degree of explicitness of goals and standards

Coverage of the syllabus;

Quantity and quality of feedback;

Use of feedback;

Appropriate assessment;

Deep approach to learning;

Learning from the examination

High degree of alignment of goals and standards

Coverage of the syllabus;

Use of feedback;

Appropriate assessment;

Clear goals and standards;

Deep approach to learning;

Learning from the examination

Feature of assessment environment

Scales from AEQ (v3.3) with significantly higher scores

High volume of formative only assessment

Coverage of syllabus; quantity and quality of feedback; use of feedback; learning from examination; appropriate assessment; clear goals and standards; deep approach to learning; satisfaction.

Learning what is required

This cannot be short circuited by trying to write down the criteria; it is tacit knowledge.

• Repeated cycles of formative assessment allow students to gradually become part of a subject community of practice.

“Students…narrowed their attention and their effort to those things they were told would be assessed, put in less effort, covered less of the syllabus, adopted less of a deep approach and gained less satisfaction from their studies.’

From Gibbs & Dunbar-Goddet (2007) p 24

Characteristics that appear to support success and retention

• Academic engagement

• Social engagement

• Active learning

Crosling, Thomas & Heagney (2007)

Improving Student Retention in Higher Education: The Role of Teaching and Learning

Spoon feeding

• Too much guidance prevents students engaging properly with the task, with ‘doing’ the subject

• Instead they seek to give us exactly what we have set out

• Miller (2007) talks about ‘painting by numbers’ but not ever really getting the whole picture of what assignments are about.

Dialogue

• Research is increasingly stressing the importance of student/ tutor dialogue in order to help them understand both feedback and guidance.

Challenges in marking research

• Disjunction between espoused and actual marking (Orrell) - criteria featured in neither much (teacher educators may be an exception);

• Tutors use internalised, holistic judgement• Staff work backwards from holistic judgement to

the published criteria (Grainger et al, Tan & Prosser)

• Many studies show variations between tutors in marking and interpretation of criteria in practice

• Impact of power relations and negotiation in marking (Orr)

Issues in Assessing Professional Competence (Tang 2008)

• consistency between different assessors, • the dual (conflicting?) role of facilitating student learning

and judging whether the standards have been achieved • judgements made less by use of the published standards

but more holistically drawing on the tacit knowledge and expertise of the assessor.

• concern about how much assessment takes into account different contexts.

• Tang argues that a mechanistic approach to standards may lead to a focus on discrete teaching behaviours rather than a broader assessment of good teaching.

Discussion

• To what extent do the findings explored so far match your experience of assessment in teacher education and higher education in general?

markingmarking

formative assessmentformative

assessment

Academic engagementAcademic

engagement

Changing views of feedback

Changing views of feedback

Criteria(e.g. LOs, prof.

standards, grade descriptors

Criteria(e.g. LOs, prof.

standards, grade descriptors

transparencytransparency

Peer & self assessmentPeer & self assessment

Learning –oriented assessme

nt

Learning –oriented assessme

nt

Skills of judgementSkills of

judgement

moderationmoderation

Need for dialogueNeed for dialogue

Tension with

standards

Tension with

standards

Innovative assessment

methods

Innovative assessment

methods

Computer assisted

assessment

Computer assisted

assessment

& assessment discourse

Assessment of professional competency

Student-tutor power

relationship

Holistic v

analytical

Don’t take a deficit approach

• It is easy to conflate the idea of ‘ability’….with [a]..lack of awareness and experience of the values, assumptions and practices of higher education.’

• ‘problems in decoding and responding to expectations appear to be particularly acute in relation to assessment criteria

(Haggis 526 & 528)

markingmarking

formative assessmentformative

assessment

Academic engagementAcademic

engagement

Changing views of feedback

Changing views of feedback

Criteria(e.g. LOs, prof.

standards, grade descriptors

Criteria(e.g. LOs, prof.

standards, grade descriptors

transparencytransparency

Peer & self assessmentPeer & self assessment

Learning –oriented assessme

nt

Learning –oriented assessme

nt

Skills of judgementSkills of

judgement

moderationmoderation

Need for dialogueNeed for dialogue

Tension with

standards

Tension with

standards

Innovative assessment

methods

Innovative assessment

methods

Computer assisted

assessment

Computer assisted

assessment

& assessment discourse

Assessment of professional competency

Student-tutor power

relationship

Holistic v

analytical

• Assessment of learning• Assessment for learning• Assessment as learning

Characteristics of assessment which Characteristics of assessment which promote learning and employabilitypromote learning and employability

• Formative, involving dialogue• Demands higher order learning • Learning and assessment are integrated• Students are involved in assessment• It promotes thinking about the learning process; • Assessment expectations should be made clear;• Involves active engagement of students, developing

independent learning;• Tasks should be authentic and involve choice; • Tasks align with important learning outcomes• Assessment should be used to evaluate teaching.

Formative assessment

Consistent evidence shows formative assessment has particular benefits for low attainers and disadvantaged learners

Black & Wiliam 1998

2727

Feedback Agenda for ChangeFeedback Agenda for Change

• It needs to be acknowledged that high level and complex learning is best developed when feedback is seen as a relational process that takes place over time, is dialogic, and is integral to learning and teaching. Feedback must be seen as a process not a product.

•  There needs to be recognition that valuable and effective feedback can come from varied sources, but if students do not learn to evaluate their own work they will remain completely dependent upon others. The abilities to self and peer-review are essential graduate attributes.

•  

• comes at the draft stage;

• Is part of the teaching & learning;

• is forward looking;• Involves dialogue;• focuses on skills

rather than content. • Is relational

• Arrives after the work is finished;

• Is not integrated into the teaching;

• Doesn’t involve any dialogue;

• Focuses on the work completed;

• Is anonymous

Studies suggest that useful feedback to students

But written feedback often

What does timely feedback really mean?

The problem with written feedback is that ‘monologue’ is trying to do the work of ‘dialogue’

Writing more and better feedback on assignments won’t improve matters because it cannot provide the interaction necessary to help students clarify their understanding.

(Nicol 2008)

3030

Feedback from different sourcesFeedback from different sources

collaborative tasks

team assessment

peer assessment

work-based mentors

studio feedback, etc

These allow students to check out their understanding in relative safety and see how others go about things.

3131

Tutor leads discussion of exemplars previously marked and annotated with feedback

Students write and submit individual assignm-ents

Tutor assesses assignm-ents and prepares feedback

Tutor hands back assignments and leads discussion on feedback

Out

of

clas

s ac

tivity

In-c

lass

act

ivity

Submission pointModule timeline

Example of module-level approaches:the use of exemplars annotated with feedback toencourage dialogue about assessment criteria

From Oxford Brookes FDTL project on feedback

3232

3. In-class discussion of generic feedback

4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback

5. Tutor assesses assignm-ents

6. Tutor hands back assignments with minimal formative feedback

Out

of

clas

s ac

tivity

In-c

lass

act

ivity

Submission pointModule timeline

Example of module-level approaches:Generic (non-personalised) feedback on draftsplus reflective commentary

1. Students draft & submit individual assignm-ents

2. Tutor marks sample of assignments and prepares generic feedback

From Oxford Brookes FDTL project on feedback

Conclusions

• How we help students to become part of our academic communities, really understanding what is expected of them, and thus able to be in control of their achievement;

• How we integrate feedback more thoroughly into our teaching and make it part of a dialogue with students;

• How we broaden our diet of assessment methods in order to validly assess the broad range of learning outcomes that characterise modern higher education without confusing students about expectations;

• How we tackle the reliability issues of assessing students holistically using tacit knowledge

• How we help students understand assessment and become partners in the process

• Any questions?