4th APS Global Education Conference

Singapore, July 2012

Leading in Assessment for Learning (AfL):

Mediating the Change

Chris Davison,

School of Education,

The University of New South Wales

“No matter how good you are, you can always do better”

By the end of this session you will have a better understanding of how assessment for learning can actually help improve learning and teaching, and how educational leaders can better lead this process

We know that many traditional assessment activities and processes fail learners and teachers by:

1. not integrating assessment into learning and teaching, hence assessment information is not used to shape and guide learning while learning

2. not providing sufficient qualitative information about assessment tasks, nor effective feedback about their learning, hence many students don’t know what to do, nor how to improve

3. not supporting and scaffolding lower level students (and not challenging and extending those at a higher level), hence undermining student motivation and ensuring students do not reach their potential

4. not emphasizing what students can so, only what they cannot

This is especially the case with assessments which have traditionally relied heavily on externally set and assessed tests and impressionistic and/or norm-based processes to sort and rank students

• “I was really upset because it’s completely inappropriate, it gives me information that I am already able to gather in my own classroom, ... there are many issues that I find are very problematic ... the experience is quite traumatic for my students, you know, because its overwhelming for them because...I’m asking them to do tasks that are way beyond them and so in most instances, I’ve just stopped because the child just sits there, and there is a sense of feeling of failure ... and that’s not what an assessment should do, you know, it should be gathering information in a non- threatening supportive way.”

(Australian primary teacher, 2011)

• Sometimes I‘ll do something, I think, “oh, its’ just going to be too hard, but I just want to see” and I’m really surprised, you know, that there are some who get to that, and I think it’s good for us to realise and for others, you know, teachers to realise, that they (students) can be stretched, you know, they‘re not just at that point.

(Australian secondary teacher, 2011)

• “The best kind of testing is what you generate yourself and that relates to what you're doing in the classroom”

(Australian primary teacher, 2011)

 

For these reasons [School-based] Assessment for Learning (AFL) is being strongly promulgated by a large number of education systems internationally, e.g. Department of Education and the Arts, 2005; Hong Kong Curriculum Development Council, 2001; Learning and Teaching Scotland, 2006; Singapore Ministry of Education, 2008; New Zealand Ministry of Education, 2002, as well as by various educational systems in Australia, including

– NSW DET, 2008, http://arc.boardofstudies.nsw.edu.au/go/sc/afl/

– Curriculum Corporation/ Education Services Australia http://www.assessmentforlearning.edu.au;

Assessment for learning (AfL) is any assessment (including summative assessment) for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning. An assessment activity can help learning if it provides information to be used as feedback, by teachers, and by their pupils in assessing themselves and each other, to modify the teaching and learning activities in which they are engaged (adapted from Black, Harrison, Lee, Marshall & Wiliam, 2003, p. 2-3)

In Australia… “The new Years 7-10 syllabuses advocate

assessment for learning, as this is a type of quality assessment that has had world-wide success in enhancing teaching and improving student learning. Assessment for learning gives students opportunities to produce work that leads to development of their knowledge, understanding and skills. Teachers decide how and when to assess student achievement, as they plan the work students will do, using a range of appropriate assessment strategies including self-assessment and peer assessment .. (so) assessment helps students learn better, rather than just achieve a better mark.”

http://arc.boardofstudies.nsw.edu.au/go/7-8/assessment-for-learning-in-years-7-10/

In Hong Kong …“Based on the beliefs that every student is unique and possesses the ability to learn, and that we should develop their multiple intelligences and potentials … there should be a change in assessment practices and schools should put more emphasis on 'Assessment for Learning' as an integral part of the learning, teaching and assessment cycle … teachers should use assessments (e.g. as simple as effective verbal questioning, observation of student behaviour) and provide immediate feedback to enhance student learning in everyday classroom lessons. The focus is on why they do not learn well and how to help them to improve rather than just to use assessments to find out what knowledge students have learned”http://cd.emb.gov.hk/basic_guide/BEGuideeng0821/chapter05.html

Assessment for Learning Resource Bank – References on Assessment

    暗箱內探 透過課堂評估 提高學習水平 ( 譯本 ) ( 課程發展處 , 2002)Original title: Inside the Black Box: Raising standards through classroom assessment© 1998 Paul Black and Dylan Wiliam

  促進學習的評估 飛越暗箱 ( 譯本 ) ( 課程發展處 , 2002)Original Title:Assessment for Learning: Beyond the Black Box© 1999 The Assessment Reform Group

In Brunei:

Two key concepts :

School-based

assessment (SBA)

+

Assessment for

learning (AfL)

= SBAfL

Brunei SBAfL Implementation

1. Eight one day PD modules with between-module activities at school, supported by local staff developers

2. Resources: video clips, course materials, key readings and weblinks provided by the PD facilitators; online sharing and support

3. Key components of the program:1. Teacher reflection/discussion on current assessment

practices/challenges/issues 2. Input by facilitators on key principles/practices3. Group activities: Practical hands-on activities to develop and apply

understanding of key SBA concepts4. Handouts: Exemplars of good practice/material to use for activities5. DVD clips: local and international examples of AfL practices. Input

from external experts 6. Follow-up classroom – based tasks and readings: To be completed

by teachers after each module4. Follow up school-based support and ongoing professional

development; development of an assessment for learning competencies framework (Alonzo, under development); ongoing monitoring and evaluation

16

In Singapore …

“For teaching and learning to be effective, teachers will identify and monitor pupils’ changing needs, abilities and interests so that they can modify or adapt their teaching methods to help pupils improve their learning. Teachers will also give timely and useful feedback to pupils and provide them with opportunities to act on the feedback to improve their learning. Together, the processes of diagnosing pupils’ needs, abilities and interests, monitoring pupils’ learning progress, and feeding-forward to improve learning constitute Assessment for Learning (AfL).”

(Ministry of Education, 2008)

The key characteristics of “assessment” in AfL:

1. assessment is embedded in teaching and learning;

2. learning goals are explicitly shared with students and students are taught how to know and to recognize the standards they are aiming for;

3. students are engaged in continuous peer and self-assessment;

4. constructive qualitative feedback helps students to recognize the next steps needed for learning and how to take them;

5. teachers, parents and students regularly review and reflect on assessment data;

6. it is assumed every student can improve(adapted from the Assessment Reform Group, 1999, p.7)

The key characteristics of “assessment” in AfL:

Exemplar 1: P5/6 whole class lesson Exemplar 2: S3 teacher – student feedback Exemplar 3: Student-led parent teacher

conferenceExemplar 4: Student formative evaluation

Why so much advocacy of AfL?

1.Black & Wiliam (1998) have convincingly demonstrated the learning gains that can be achieved through well focused teacher-based formative assessment enhance students’ learning more than any other strategy across age levels and in different contexts

2.Hattie (2009) in a more recent study of major influences on educational achievement (using 800+ meta-analyses) found that formative practice, in particular self-assessment and feedback, had the highest effect sizes (i.e., impact on student outcomes) out of more than 100 different instructional and contextual factors

Influences on student achievement

.40.30

.15

0

.50

.60

.70

.80

.90

1.0REVERSE

Developmental

Effects

Typical

Teach

er

Effects

ZONE OFDESIREDEFFECTS

Role of assessment? Effect Size

Self-evaluation 1.44

Feedback .72Providing formative evaluation to teachers .70Frequent/ Effects of testing .46Teaching test taking skills .22

Effective Feedback (adapted from Kluger & DeNisi, 1996, from Hattie, 2007)

1. Provide correct(ive) feedback .43

2. About previous attempts .55

3. Related to more difficult goals .51

4. That does not discourage .33

5. or threaten self-esteem .47

International research highlights five key characteristics of successful assessment reform

1. First, teachers (and school leaders) need to emphasize

– the change in the purposes of assessment (enhancing learning and teaching) and

– the change in teacher and student roles (involving learners more actively in evaluating and improving their own learning)

… i.e., ensure assessment reform goes beyond superficial adjustments in assessment format, frequency and feedback (Davison, 2007)

2. Second, teachers (and school leaders) need to learn how to develop assessment tasks and activities and assessment criteria/rubrics which are appropriate for range of individual needs

““It’s a good piece of assessment, but it’s not a good piece of….ah…..it doesn’t actually help them. It helps us to assess them, but it doesn’t help them to progress.” (Hong Kong English teacher, 2006)

Although such instruments can be difficult to construct and more time-consuming than traditional testing and/or impressionistic marking (Fox, 2008), investment in system-wide change can make huge differences to student outcomes.

3. Third, teachers (and school leaders) need to develop more effective and dialogue-based feedback practices which scaffold and support less confident learners and engage and challenge higher level learners, so all are able to take responsibility for their learning and achieve their best

BUT … this means not just changing the assessment practice, but changing the assessment culture (Davison & Leung, 2009), and this take time (7-10 years) .

4. Fourth, teachers (and school leaders) need to ensure the reliability, i.e. trustworthiness, of teacher-based assessments, thus it is critical to incorporate mechanisms to ensure teachers (and students) are making consistent and trustworthy assessment decisions, e.g. through widespread use of exemplars of performance expectations/levels, recording and reviewing of data, benchmarking with other classes (and educational systems) and developing online communities of practicee.g.. See Queensland Assessment Authority, http://www.qsa.qld.edu.au/3162.html

Example 1: Key changes Hong Kong 2005-2010

1. From a focus on task types to a focus on learners and their needs, although not surprisingly strong bias towards more formal assessment, and still little or no reference to whole-class contingent formative assessment practices

2. From a focus on the timing of assessment to a focus on “making the time”, particularly through the integration of formative assessments into the regular curriculum

– “We know we have to give feedback though we are busy – it’s really teaching, not assessment”

3. From a focus on student improvement to a focus on teachers helping students improve, with a particular emphasis on sharing and evaluating success criteria

– “We now have an awareness of levels in assessing students so we can help them improve”

– “We support each student individually, to learn more about how to help students improve”

4. From a slight focus on teachers to a strong focus on learners and their roles, although still less confidence and skill with self and peer assessment than other aspects of formative assessment

–“The assessment gives me more confidence in assessing my student and guiding them in peer assessment”

5. From an focus on better grades to a developing focus on better learning and teaching, and more trustworthy assessment decisions, although still very teacher-directed, rather than student-directed

– “Jointly assessing the students' performances … is useful as we can discuss the criteria, the task

and the score”– “Now I'm able to compare my students’ (work) with their counterparts and can help/improve my

teaching”

Perhaps not surprisingly, improvements in the teacher-student relationship were also reported by teachers

– “(It has) Enhanced the relationship with my students!”– “I have more chances to know my students, get more

chances to motivate and give orientation to students”– “It has increased the magnitude of my oral

communication with my students” – “There has been a major change in my role as a

teacher. I've been actively involved in helping my students to improve their learning, helping them to understand the assessment criteria and providing feedback on how they can do better”

BUT still a need for further teacher development and support:

– “We need more on setting scaffolding questions/ guiding questions for students.”

– “I still find it difficult to design good assessment tasks which fit for students' needs (especially a class of diversified students)”

– “We still to learn more about how to come up with better tasks that can be integrate into the curriculum, tasks that are really interesting relevant & fun”

– “I need to scaffold the tasks to the actual assessment, to set a tasks which can elicit quality language use!”

Teachers’ Overall Attitude towards SBA

Teachers' Overall Attitude Towards SBA

3

3.2

3.4

3.6

3.8

4

4.2

4.4

4.6

Teachers' views ontheir preparation

for SBA

Teachers' views ontheir school's

preparation forSBA

Teachers' views onimplementation of

SBA

Teachers’ views onpotential problems

of SBA

Teachers' views onthe effects of SBA

on students

Teachers' views onthe effects of SBA

on teachers

Questionnaire Category

Val

ue

Longitudinal Study Round1MeanLongitudinal Study Round 2MeanLongitudinal Study Round 3MeanContinuation Study Round 1 Mean

Key findings from Brunei: The contribution of SBAfL to learning and teaching

Years 7-82010 Pre and Post-Program Questionnaires

Other interesting findings•SBAfL is seen as a way to help teachers assess students with lower ability levels and make their teaching more effective

Other interesting findings•SBAfL is seen as a way to help teachers assess students with lower ability levels and make their teaching more effective

The implementation of SBAfL in the classroomFindings from the Years 7-8

2010 Pre and Post-Program Questionnaires

Overall SIGNICANTLY POSITIVE change in teacher assessment readiness pre and post PD

Overall SIGNICANTLY POSITIVE change in teacher assessment readiness pre and post PD

Note: Lower confidence levels were expressed by teachers in Maths and Malay groups for ’writing effective assessment rubrics’

Most importantly, there is an urgent need to align all the key elements of assessment and professional support to form a coherent whole so the educational system and processes enable teachers, students and schools to focus on using assessment to improve learning …

“How can we go on with our efforts on focusing on students' needs when accountability is still an issue every teacher has to deal with? Accountability such as in submitting marks every single week ….and then if no marks are submitted, we are thought to be doing nothing in the class?”

http://sbaflforteachers.groups.live.com/discussions/discussion.aspx?id=cns!9E85C75729049!200

Fullan (2001) highlights the role of the school leaders, including principals and district administrators, in mediating and legitimating change and in providing teachers with both psychological and material support (see also Wideen, 1994). Principals as the people most likely to be in a position to shape the organizational conditions necessary for success, such as the development of shared goals, collaborative work structures and climates, and procedures for monitoring results. However, when illustrating the problems that a principal might confront, Fullan (2001) argues that:

The subjective world of principals is such that many of them suffer from the same problem in implementing a new role as facilitator of change”as do teachers in implementing new teaching roles: What the principal should do specifically to manage change at the school level is a complex affair for which the principal has little preparation. The psychological and sociological problems of change that confront the principal are at least as great as those that confront teachers (p.83).

•  

Fullan (2001) concludes whether or not implementation succeeds depends on the congruence between reformers and local needs, and how the changes are introduced and followed through. He argues:

– Not only is meaning difficult to grasp when different worlds have limited interactions, but misinterpretation, attribution of motives, feelings of being misunderstood, and disillusionment on both sides are almost guaranteed.

– The quality of the relationships between reformers and those impacted by the reform is crucial in supporting change efforts when there is agreement, and to reconciling problems when there is conflict among different groups.

In AfL such conflict is inevitable when the system itself is not seen as part of the reform process.

Key questions to guide assessment reform So… what kind of system and school-wide processes for

change can be adopted for the implementation/enhancement of AfL that

• are theoretically and philosophically consistent with AfL?

• model desired outcomes and lead - by example - to sustainable improvements in assessment, learning and teaching?

• put the learners and teachers at the centre of the change process?

• involve students, parents and the wider school community in understanding and supporting reforms?

• maintain confidence in the assessment system?

We need a parallel set of characteristics of “assessment” reform for the introduction and development of AFL at the system level, adapted from the original tenets of the Assessment Reform Group (1999, p.7), that is, ensure: – AFL is embedded in curriculum and assessment institutionally and

pedagogically;– assessment reform goals are explicitly shared with all stakeholders

and stakeholders together work to identify how to know and to recognize the standards they are aiming for;

– all stakeholders are engaged in continuous peer and self-assessment;

– constructive qualitative feedback helps stakeholders to recognize the next steps needed for reform and how to take them;

– all stakeholders regularly review and reflect on assessment data;– it is assumed every school, teacher and student can improve;

In such a system AfL is not just a target of assessment reform but in fact the driver of how to achieve such reform.

Question:

How can we (better) develop such and assessment for learning culture in our schools?