2011/10666

Assessment seminar2011

ENGLISHJohn Watson

2011/10666 2

WACOT professional learning requirements

• today’s seminar can contribute

– documentation required - keep your Reflective Journal as evidence.

2011/10666

Key messages 2011• The end of 2010 effectively saw the completion of the senior school

changes

• The Curriculum Council acknowledges the significant amount of work done by teachers to implement and assess their new courses

• In the main the WACE exams accurately and fairly assessed students’ achievements at Stage 2 or 3

• The approach now taken ensures that the Council can provide quality feedback to schools in all courses through the statistical and examination feedback reports and the Standards Guides

• We hope to have at least five years before the national curriculum is implemented in Years 11 and 12 – decisions about this have not been made yet

2011/10666

Seminar objectivesThat teachers:• understand the function and structure of the Standards Guides• are aware of the possible uses of the information provided in the

Standards Guides• know how the Council’s moderation processes will be implemented

for this course in 2011• understand how school marks and WACE examination marks are

adjusted• are aware of the 2010 state wide achievement and examination data

for this course• are aware of the range of student achievement data made available

to schools by the Council• are confident in the ability to analyse the student achievement data

provided in SIRS reports STS022, STS028 and STS036• are aware of the item analysis data provided in SIRS reports MF01 • are aware of and have the ability to access the range of assessment

support materials available to teachers on the Council’s extranet

2011/10666

Examinations and standards

2010 was:

• the fourth year this course has been examined

• the second time the standards setting process had been completed

2011/10666

WACE Course report

2011/10666

Page

72 in

WACE

manual

2011/10666

Raw WACE exam marks

Moderated school marks

Raw school marks

Standardised moderated

school marks

Standardised WACE exam

marks

Scaled scores

Combined marks

The marks adjustment process

WACE course scores

Curriculum Council processes

TISC process

2011/10666

The standards setting process

This is used to:

• set achievement band cut-offs which allow adjustment of the combined mark distribution to create the WACE course scores

• develop the achievement band descriptions to summarise what students demonstrated in the written examination

2011/10666

The WACE course score

<insert Alan Honeyman diagram>

2011/10666

Stage 3: Excellent Band Description

Excellent achievement (75 - 100)• Responds comprehensively using a variety of

texts; shows an excellent grasp of syllabus concepts.

• Demonstrates excellent command of writing conventions, including complex sentence structures, paragraphing, grammar and spelling. Uses sophisticated language to influence audiences. Sustains a lively personal voice to engage, entertain and communicate.

• Analyses codes and conventions to produce detailed, justified interpretations and texts.

2011/10666

Stage 3: Satisfactory Band Description

Satisfactory achievement (50 - 64)• Refers to texts to apply syllabus concepts.• Generally displays control over sentence

structure, paragraphing, grammar and spelling. Generally uses appropriate vocabulary, language conventions and writing forms. Inconsistently or ineffectively uses personal voice.

• Generally employs codes and conventions to analyse and produce texts.

2011/10666

The Standards Guides

• produced following the completion of the standards setting process

• separate guides for Stage 2 and Stage 3

2011/10666

To provide:

• the examination questions, marks and marking keys

• selected candidate responses and marker annotations

• question statistics

• examiners’ comments

Main function of the Standards Guide

2011/10666

Component 1: The examination questions, marks and marking keys

Question 2 (30 marks)Every aspect of a visual text is constructed to have an effect on its intended audience.Discuss this statement with reference to at least one of the images provided and at least one visual text you have studied.

Marking keyDescription

This question requires meaningful consideration of the relationship between the purpose of a text and the way it is produced in response to a target audience. Good answers will show some consideration of the form of the text (campaign poster, movie poster, film still image, photograph, magazine cover) and how this affects the text’s content and construction. The ideas directed at a text’s audience may range from efforts to ‘sell’ a ‘product’ (such as trying to convince people to see a movie, or give money to a charity), to more subtle communications of social expectations and norms, such as those to do with the distribution of power in terms of gender, race, class and sexuality. Responses may engage with the part of the question that stipulates that ‘every aspect’ of a visual text is constructed to have certain effects, commenting on the conscious and meticulous construction of texts that appear in print and visual media. Alternatively, they may disagree with the statement and attempt to argue that not all texts consist of this degree of artifice and may feature aspects that are ‘natural’ or incidental. It would be helpful to refer to specific details and conventions of texts such as those listed for Question 1.

2011/10666

Component 2: Selected candidate responses and marker annotations

Excellent response29/30 marks

Engaging and original opening establishes a strong sense of personal voice and brief response to the question. Texts to be referred to are clearly acknowledged.

2011/10666

Component 3: Question statisticsSection Three:

Viewing

30 marks

Note:Raw section total marks =

30

Weighted section total marks = 33.33

Weighted Section Statistics

Statistics ID = 23

Number of attempts = 8434

Highest mark achieved = 33.00

Lowest mark achieved = 0.00

Mean = 19.73

Standard deviation = 5.19

Correlation between section and exam total = 0.81

2011/10666

Component 3: Question statisticsQuestion statistics Statistics ID = 10

Number of attempts = 7594

Highest mark achieved = 30.00

Lowest mark achieved = 0.00

Mean = 17.81

Standard deviation = 4.63

Question difficulty = Moderate

Correlation between question part and section = 1.00

2011/10666

Component 4: Examiners’ commentsExaminers’ commentsSection Three: Viewing

Whilst most candidates used a formal essay style for their answer, a few did not allow themselves to be impeded by any attempt to structure their responses effectively and there were also too many who again wrote in an overly casual style. One marker noted: ‘…a lot of students discussed both texts in the same paragraphs or alternated, i.e. one paragraph on the static image, then one on the studied text, repeated a few times. This often……

Examiners’ commentsQuestion 2

Whilst most candidates obviously thought that this was an accessible and straightforward question, a disappointing number of them failed to discuss or define the intended audience and the effect of the construction. Only a few challenged the word ‘every’ even where they were discussing Images 3 or 5. However, many did write detailed…

2011/10666

Self reflection 1

Investigating the information in the Standards Guide

• Stage 3 – Section stats p75; Qu 2 page 91• Stage 2 – Section stats p75; Qu 2a page 90

2011/10666

Self reflection 2

Information in the Standards Guides can be used by:

• teachers and their students

• the examining panel

• the Curriculum Council

2011/10666

Moderation processes 2011

• statistical moderation of school marks

• consensus moderation, stage 3 July 25

• school moderation program

• small group moderation

2011/10666

Take a break

2011/10666

State examination statistics 2010English

Stage 3 mean standard deviation

Raw exam mark 59.00 11.47

Standardised exam mark 61.03 13.62

Difference + 2.03 2.15

Stage 2 mean standard deviation

Raw exam mark 57.00 13.35

Standardised exam mark 61.24 13.9

Difference + 4.24 0.55

2011/10666

Grade distributions forEnglish

  A (%) B (%) C (%) D (%) E (%) Total

Stage 2A 6.7 25.5 56.2 9.6 2 2745

Stage 2B 7.9 26.4 56.9 7 1.6 2569

Stage 2C 4.6 25.6 59.3 8 2.4 1927

Stage 2D 5.4 27.2 59.4 6.4 1.7 1862

Totals Stage 2 6.15 26.2 58 7.75 1.93  

Stage 3A 12.8 32.6 50.4 3.7 0.6 8506

Stage 3B 14.3 34.3 48.3 2.6 0.6 8438

TotalsStage 3 13.6 33.5 49.4 3.15 0.6  

2011/10666

Student achievement data – STS022

2011/10666

https://sirs.curriculum.wa.edu.au

2011/10666

Reports – other reports – statisticsSTS - 022

2011/10666

Student achievement data – STS022

2011/10666

2011/10666

2011/10666

The WACE course score

<insert Alan Honeyman diagram>

2011/10666

2011/10666

2011/10666

Your school in blue

2011/10666

Self reflection 3: a case study1. Describe what has happened to the school marks and

exam marks.2. Consider why marks adjustment is necessary across

the statewide examination population.3. Evaluate the relationship between the grade

distribution and the achievement band distribution.4. Evaluate the relationship between the rank order of the

students based on their school marks and the rank order based on their exam marks.

5. Identify possible causes for the assessment issues you identify and strategies that could be implemented.

2011/10666

Self reflection 4

1. Describe what has happened to the school marks and exam marks.

2. Evaluate the relationship between the grade distribution and the achievement band distribution.

3. Evaluate the relationship between the rank order of the students based on their school marks and the rank order based on their exam marks.

4. If issues are apparent, identify possible causes and strategies to address these.

2011/10666

Detailed examination feedbackformerly known as Maximising Feedback reports

• Provides an item analysis for your students

• Separate reports for written (MF01) and multiple choice

2011/10666

2011/10666

Council support for teachers• sample teaching programs

• sample assessment tasks

• sample assessment programs

• sample exams

• annotated student work samples

2011/10666 41

Council support for teachers

• assessment support materials on the Council extranet.

• register with WACOT number and work email address

• information in the WACE Manual 2011 and the WACE Circular each term (and more regularly in the eCircular)

• email any questions to info@curriculum.wa.edu.au

2011/10666

ATAR Using Stage 2 and 3 in 2010

2011/10666

Scaled Score Statistics – English 2010

2011/10666 44

Contact details• For clarification or further information about

assessment or moderation processes contact:

John Watson9273 6374john.watson@curriculum.wa.edu.au

• For syllabus clarifications and queries about teacher support materials contact:

• Gerard Morris 9273 6742

gerard.morris@curriculum.wa.edu.au

2011/10666

Seminar objectivesThat teachers:• understand the function and structure of the Standards Guides• are aware of the possible uses of the information provided in the

Standards Guides• know how the Council’s moderation processes will be implemented

for this course in 2011• understand how school marks and WACE examination marks are

adjusted• are aware of the 2010 state wide achievement and examination

data for this course• are aware of the range of student achievement data made

available to schools by the Council• are confident in the ability to analyse the student achievement data

provided in SIRS reports STS022, STS028 and STS036• are aware of the item analysis data provided in SIRS reports

MF01/02/03• are aware of and have the ability to access the range of

assessment support materials available to teachers on the Council’s extranetPLEASE COMPLETE THE SECOND

PART OF THE EVALUATION

SHEET

2011/10666

• use your Reflective Journal

• this information will be available as an online learning module accessible

Reflective Journal

2011/10666

• If teachers at your school could not attend a face-to-face assessment seminar and wish to access one or both of these modules, please email the following information to jennifer.wheatley@curriculum.wa.edu.au

• name• school name• the sector/system (AISWA, DoE or CEO)• your school email address• your WACOT number.

Online learning module

2011/10666

CEO• Frank Italiano   - Whole School Development Consultant Learning

and Teaching K-12 Team  -  6380 5362 - Italiano.Frank@cathednet.wa.edu.au 

 AISWA• Mark Newhouse  - Manager of Curriculum, Association of

Independent Schools of Western Australia  - 9441 1639 - mnewhouse@ais.wa.edu.au

•  DoE• Richard Lobb - Manager │K-12 Curriculum, Assessment and

Reporting - 9264 5076 - Richard.Lobb@det.wa.edu.au

ACARA - update info from: