CURRICULUM FOR EXCELLENCE AND BTC 5 : POSSIBLE IMPLICATIONS FOR THE SCOTTISH QUALIFICATIONS AUTHORITY

A conversation with Ruth Sutton

Feb. 17th 2010

CFE, BTC 5, TEACHERS AND SCHOOLS

National Assessment arrangements have now begun to roll out, with very rapid change expected over the next few months

For some ‘CfE deniers’ this is coming as a shock

Record-keeping and reporting are still unknown quantities: BTC 5 refers constantly to ‘regular’ reporting to parents, but provides no indication of frequency, without which the implications are impossible to determine

Secondary teachers seem more anxious and confused than their primary counterparts. Is this the case, and if so, why?

Some questions I’ve raised recently with Scottish teachers about assessment

How can we achieve an acceptable balance of validity (accuracy) and reliability (consistency and fairness) in national assessment and avoid the ‘collateral damage’ to teaching and learning?

How can we substitute ‘shared professional judgement’ for the external test ?

The assessment balancing act: do the elements pull in different directions?

Validity Reliability

ManageabilityCost and ‘credibility’

Best fit

Assessment of learning

Checks learning to dateAudience beyond the

classroomPeriodicUses numbers, scores

and gradesCriterion/standards

referencedNo need to involve the

learner

Assessment for learning

Suggests next learningAudience is teachers and

learnersContinual – conversation

and markingSpecific feedback, using

wordsSelf-referenced, ‘ipsative’Must involve the learner –

the person most able to improve learning

‘Shared professional judgment’What is happening with Curriculum for

Excellence?1.Standards remain relatively ‘loose’2.Those involved in making judgments need to

agree on the precise meaning and implications of the ‘standards and expectations’

3.Judgments will be shared and cross-checked, to reduce ‘rater variables’ (ie. the individual teacher’s judgment) to an acceptable level, and improve reliability, given the inevitable margin or error

The benefits of ‘moderation’ in the current Scottish circumstancesCfE emphasises skills rather than contentThe evidence of skills is necessarily contextual, and

more easily found across a range of classroom activities than in separate assessment ‘events’

Teachers will share their understanding of the CfE framework in order to design their teaching activities

Sharing their understandings and planning is a more efficient way to work than each teacher or subject team working alone

Teachers will be more confident and clear, gathering only as little evidence as they need, not as much as they can manage

Parents will receive more accurate reports about their children’s progress, based on shared considered evidence and judgment

Costs and benefits of moderation,for teachers?Costs: Time to discuss expectationsPossible change in planning habitsInitial confusion resulting from a poor understanding

of the purposes and principles of assessment

Benefits:Greater clarity about CfE ‘experiences and outcomes’Getting teaching and assessment ideas from other

teachersImproved manageability through gathering less

evidenceGreater confidence when reporting to parentsBetter understanding of the purposes and principles

of assessment

Professional sharing of ‘standards’The teachers’ discussion will probably have four

stages:1.What do these ‘standards’/ ‘experiences and

outcomes’ really mean, and what do they look like in practice?

2.What evidence would properly support a teacher’s judgment of a student’s achievement of the ‘standard’?

3.What ‘success criteria’ will we need to identify progress within the level?

4.What teaching activities will be necessary to enable the student to learn, practice and demonstrate achievement of the ‘standards’?

Backwards Design Traditionally teachers have planned ‘forwards’ from a definition

of learning objectives, into teaching activities and finally into assessment activities and evidence of performance

Backwards Design – the process ask big questions about what the pupils will learn, and

prioritising the most important elementsConsider what evidence would/will there be that this desired

learning has happened? ensure that this evidence will be a valid reflection of the

expected learning, and sufficient (‘proportionate’) for reporting purposes

design teaching and learning activities which will generate this evidence of learning, starting with the necessary check for prior learning and misconceptions

The benefit of Backwards Design is that it enables both teachers and pupils to identify the learning goals and focus on them in both teaching and assessment.

Experience of ‘moderation’This is a personal view, based on experience of helping

teachers implement standards-based curriculum and assessment over the past twenty years or so

Teachers find ‘moderation’ -Challenging, as it exposes their professional learning

and judgment to the scrutiny of their peersVery helpful, as it involves detailed practical

discussion of learning expectations and the teaching tasks necessary to encourage learning and achievement

Necessary, to avoid individual teachers gathering too much, poor quality evidence to compensate for their lack of confidence

Some suggestions..Don’t spend too long initially on semantic

deconstruction of the words in the ‘standard: when someone opens the Roget’s Thesaurus it’s probably time to move on

To help lower the temperature, use neutral exemplars to clarify and illuminate the discussion, before reaching for work done by your own students

Encourage teachers to accept that professional differences of view are probable and acceptable. They are also opportunities for professional learning if we keep an open mind.

Learning from others

Wales has begun a process of structured moderation at KS2 and KS3, to improve the quality of teacher judgment and to encourage sharing of best teaching practices.

Queensland, Australia has a long history of assessment using teacher moderation. Check out www.education.qld.gov.au/qcar/social-mod.html

New Zealand’s new curriculum will also be supported by a moderation process, leading to ‘Overall Teacher Judgment’.

Queensland’s Conference Model of Moderation

“Using the conference model for moderation, teachers discuss and deliberate in making their judgments about the quality of all of the evidence presented as student work.

Teachers make judgments on several criteria to reach an

'on-balance' holistic judgment. This is not a procedural approach but one that is based on the teachers' professional knowledge in shared and collaborative decision making.

Teachers mark (some or all) student responses individually, and then select assessment samples representative of their application for A to E standards. They meet with other teachers to discuss their judgments by sharing their samples. Teachers reach a consensus on the interpretation and application of the standards.”

The Role of the Facilitator

“In the Conference Model of social moderation the role of the facilitator may include:

Establishing the moderation environmentIdentifying the curriculum intentLeading professional dialogueFacilitating conversations that support evidence-

based teacher judgmentClarifying moderation protocolsIt is not expected that the facilitator act as an expert,

but rather assist teachers reach consensus through a shared understanding of the curriculum intent and the grade awarded”

Queensland Social Moderation Protocols

“Commit to the purpose of the moderation processAdopt a sense of responsibility in and for the

groupRespect and listen to others openlyAccept where others are atCooperate in good faithAim for consensus in decision makingAddress problems respectfully by seeking

clarification and understanding, focusing on the student work and not the teacher who presents it

Treat others as you would like to be treatedCritique not criticise”

Finding the assessment balance: the ‘Muddle in the Middle’

In the primary years, • the norms of assessment remain largely ‘formative’• AiFL has been accepted and adopted, on the whole• The structures and cultures of most primary schools encourage collaboration

In the secondary years, • the dominant assessment culture has been external and summative• AiFL is ‘patchy’, at best: ‘tools’ are used without necessary understanding of

‘principles’• Structures are fragmented vertically, discouraging collaboration in skills

assessment (Num., Lit., and HWB) across ‘subjects’

S1 (and possibly P7) to S3 sit uncomfortably between these two

SQA’s position

• My impression is that SQA, like any other qualifications authority, has been over many years focused on ‘subjects’, reliability and a particular definition of ‘fairness’

• CfE and BTC 5, and the principles of AiFL that ostensibly underpin both, challenge many of the ‘old’ paradigms assumptions.

• Secondary schools and teachers are looking for clarity and leadership: without this some more basic instincts and attitudes may prevail

Some basic instincts and attitudes• Assessment is primarily about measurement• ‘Grading and sorting’ is the first purpose of secondary

assessment• Involving the students (a la AiFL) doesn’t make sense in

these circumstances• External measures are required to prevent teachers and

students ‘cheating the system’• In a content-led curriculum, planning and teaching is largely

about ‘coverage’: how then to plan and deliver a skills-based curriculum in which the context is determined by the teacher?

• A ‘level’ is a line, not a large space within which students might learn non-sequentially

Questions

• What are the issues and concerns facing SQA at this stage in the roll-out of CfE and BTC5?

• What is SQA’s role as an essential provider of teachers’ professional learning over the next months/years?

• What are the key messages around assessment that SQA can help to disseminate, and how is this best achieved?

Thanks for the invitation to talk with you

All my views are personal and can therefore be completely disregarded

Additional ‘outsider’ views might provide necessary ‘triangulation’

Sutton.ruth@gmail.com