Presentation Outline

Presentation Outline (a) Introduction

(b) Enrolment Statistics

(c) Enhanced Monitoring

(d) Target Success Indicators

(e) Progress - Examination Cycle

(f) Progress Chart

(g) Conclusion

2

BRIEFING ON EXAM AND ANA READINESS

Introduction

3

Introduction• Public Examinations in South Africa is considered to be

internationally comparable.

• Significant progress made over the last nineteen years.

• National Assessment is a fledgling system that is beginning to take root.

• The DBE continuously strives to improve and enhance examination and assessment functionality.

• Examinations and Assessment system is responsive to challenges and changes.

4

Change Factors

(a) The need to adjust the examinations and assessment landscape to the environmental changes:- the need to improve learning outcomes- understanding the different purposes of assessment.- need to develop 21st Century Skills

(b) The constant search for appropriate standards.

(c) The need to strengthen initiatives leading to the development of a national examination and assessment system.

(d) The need to identify the threats to examinations and assessment credibility and establish how best these risks can be mitigated.

(e) The need to institutionalize Annual National Assessment so that regular and meaningful feedback is provided to the system.

5

State of Examinations and Assessment in the Republic

Examinations

(a) Well established and rigorous business exam processes.(b) National question papers of a high standard have

contributed to improved teaching and learning.(c) Efficient administration of examinations.(d) Highly committed staff.(f) Significant strides made towards the establishment of a

single national examination system.(g) SBA system that is showing signs of improvement.(h) Marking system that is steadily improving.

6

State of Examinations and Assessment in the Republic

National Assessment

(a) Significant progress in the administration of Annual National Assessment. Still a developing system.

(b) Improved provincial capacity to manage both NSC exams and ANA.

(c) Improved quality of data (d) Printing outsourced and done in-house.(e) Dichotomy of test design being explored and will be piloted.(f) Data capture at 85% level.(g) Data processing and release completed in record time.(h) Diagnostics have added a new dimension to teaching and learning

7

The ChallengesThe Challenges(a) Question Papers: attaining international standards.

(b) Marking : inter-marker reliability.

(c) School based assessment: quality assessment at all levels.

(d) Standards: Grade 10 and 11

(e) Assessment purpose: diagnostic versus systemic.

(f) Feedback: effective data utilisation.

(g) Trend Analysis: National versus International and Year to Year

comparisons.

8

BRIEFING ON EXAM READINESS

NATIONAL SENIOR CERTIFICATE

9


enrolment

10

Scope and Size of 2014 NSC Exams• Total Candidates: 689 625• Full Time Candidates: 551 588• Part Time Candidates : 138 037• Question Papers: 258• Printed Question Papers: 7.2 million• Scripts: 7.0 million• Examination Centres: 6 740• Invigilators: 65 000• Markers: 41 564• Marking Centres: 118

11

12

Province Name Total Entered

EASTERN CAPE 69 860FREE STATE 26 899GAUTENG 101 340KWAZULU-NATAL 148 925LIMPOPO 73 546MPUMALANGA 46 187NORTH WEST 26 382NORTHERN CAPE 9 556WESTERN CAPE 48 893

NATIONAL 551 588

NSC Full-Time Enrolments per Province 2014

12

14

ProvinceEntered

2010Entered

2011Entered

2012Entered

2013Entered

2014

Eastern Cape 68 294 68 174 69 199 74 998 69 860

Free State 28 448 26 510 24 760 28 019 26 899

Gauteng 94 642 87 631 91 475 99 504 101 340

KwaZulu-Natal 130 302 127 053 132 272 150 154 148 925

Limpopo 95 869 74 669 78 182 83 594 73 546

Mpumalanga 54 542 49 592 48 989 51 206 46 187

North West 29 601 25 931 27 575 29 539 26 382

Northern Cape 10 406 10 426 9 281 10 693 9 556

Western Cape 47 062 41 273 45 602 48 783 48 893

National 559 166 511 259 527 335 576 490 551 588

NSC Full-Time Enrolments per Province 2010 to 2014

14

16

Province

2013 2014Difference

in FT

2014-2013Full Time Part time Total Full Time Part time Total

Eastern Cape 74 998 20 536 95 534 69 860 21 359 91 219 -5 257

Free State 28 019 3 208 31 227 26 899 3 018 29 917 -558

Gauteng 99 504 41 408 140 912 101 340 42 355 143 695 1 860

KwaZulu-Natal 150 154 24 888 175 042 148 925 26 192 175 117 -1 029

Limpopo 83 594 16 993 100 587 73 546 19 673 93 219 -10 015

Mpumalanga 51 206 7 391 58 597 46 187 7 761 53 948 -4 968

North West 29 539 3 604 33 143 26 382 3 799 30 181 -3 152

Northern Cape 10 693 1 760 12 453 9 556 2 110 11 666 -1 014

Western Cape 48 783 10 858 59 641 48 893 11 770 60 663 213

Total 576 490 130 646 707 136 551 588 138 037 689 625 -23 920

NSC Enrolments - 2013 and 2014 comparison

18

Subject Entered

Accounting 129 352Afrikaans First Additional Language 83 817Agricultural Sciences 80 476Business Studies 212 630Economics 141 339English First Additional Language 444 689Geography 241 921History 118 758Life Sciences 291 317Mathematical Literacy 319 191Mathematics 232 432Physical Sciences 172 259

NSC Full-Time Subject Enrolments 2014

19

SubjectEntered

2010Entered

2011Entered

2012Entered

2013Entered

2014

Accounting 165 522 140 849 137 527 148 374 129 352

Afrikaans First Additional Language 78 502 69 287 76 797 88 727 83 817

Agricultural Sciences 88 075 79 680 79 828 85 098 80 476

Business Studies 206 625 191 850 199 377 223 467 212 630

Economics 151 911 136 652 137 524 153 644 141 339

English First Additional Language 462 959 424 346 430 521 464 997 444 689

Geography 215 815 203 805 217 875 244 457 241 921

History 90 595 88 290 96 481 111 568 118 758

Life Sciences 292 865 270 540 283 611 307 303 291 317

Mathematical Literacy 288 370 281 613 297 074 330 789 319 191

Mathematics 270 598 229 371 230 194 245 663 232 432

Physical Sciences 210 168 184 052 182 083 187 170 172 259

NSC Full-Time Subject Enrolments 2010 to 2014

20

270 598

229 371 230 194

245 663

232 432

200 000

210 000

220 000

230 000

240 000

250 000

260 000

270 000

280 000

2010 2011 2012 2013 2014

Mathematics Full-Time Enrolments

23

Enrolments NSC 2014 in terms of Gender

BRIEFING ON EXAM AND ANA READINESS

enhanced monitoring

approach

Monitoring Approach(a) Key focus: building a credible, national examination and

assessment system

(b) Five pillars of the enhanced monitoring approach:

- Development of Norms and Standards

- Mediation of Norms and Standards

- Coordination of examination processes

- Monitoring and support of PEDs

- Evaluation and feedback

Monitoring Approach

(c) Monitoring and support entailed the following:• identifying the risks.• collaborative review.• four-tiered approach.• part-time monitors.• state of readiness (d) Final State of Readiness visit in September 2014.

Target Success IndicatorsTarget Success Indicators

a) High quality question papers are set and moderated.

b) Examination centre and candidates are accurately registered on the examination computer system.

c) Question papers are accurately printed.

d) Question papers are distributed on time to all schools and candidates and strict security measures adhered to in the distribution process

Target Success Indicatorse) SBA is completed in accordance with policy prescripts and

moderated.

f) All examination and SBA marks are accurately captured.

g) All examination irregularities are appropriately managed.

h) All candidates are accurately and timeously resulted.

i) Effective feedback provided to schools.

j) All eligible candidates receive certificates.

Target Success Indicatorsk) All candidates including learners with special needs are

accommodated in the examination.

l) Examinations are conducted under controlled conditions across all examination centres..

m) Answer scripts accurately marked and moderated.

n) All examination marks are accurately captured on the examination computer system.


registration

Registration of Candidates/Centresa) All candidates have been registered on the Examination Computer

System.

b) Most PEDs registered candidates in Grade 10 and data is rolled over each year.

c) Two schedule of entries have been sent to all schools to verify accuracy of registration data.

d) Subject changes, immigrant candidates and special concessions have been appropriately managed.

e) DBE to conducting an audit of the provincial registration data.

Registration of Candidates/Centresf) All enhancements on the examination computer system will be

completed by 30 September 2014.

g) A complete dry run on the system will be commence by 30 September 2014. Dry run done in conjunction with Umalusi

h) All independent centres have been evaluated to ensure that they

satisfy the criteria for registration.

i) In cases where there has been some doubt about the integrity of the centre, the examination will be administered by the PED, or will be closely monitored.

Registration of Candidates/Centresk) Concessions granted to candidates with special needs and include:

• Additional time• Amanuensis• Scribe

l) Special extension on the SBA validity period granted by the Minister to repeat candidates, where the validity period had expired.


question papers

Question Papers a) All 258 question papers for November 2014 and March

2015 examinations have been set and moderated by the DBE to ensure a national standard

b) Question papers have been approved by Umalusi, edited, quality assured and handed over to PEDS based on their printing plans.

c) Adaptation of the question papers for blind, partially sighted and deaf candidates is complete

d) Brailing of the adapted papers for the blind is in progress

36

Question PapersChanges based on CAPS:

•Mathematics – two papers.•Economics – two papers.•All Home Language papers – one short text has been

replaced with a long text.•History – one additional essay.•Consumer Studies – one additional question.

Preparation for CAPS:•Exemplars developed for subjects with significant

changes.•Examination Guidelines reviewed for all subjects.

37

Question Papers

Other Changes:• Separate panels for Paper 1 and Paper 2 in Maths and Physical

Science.• Fairness review of question papers. • Additional tier of editing

38


printing, packing and distribution

Printing, Packing and Distributiona) Printing and packing progressing according to plan. For

security reasons question papers made available to PEDS only a week before printing commences.

b) Earliest that printing could commence was 1 August 2014.

c) Storage facilities across all PEDs have been inspected and security has been improved at distribution points

d) Question papers will be distributed to examination centres on the morning of the examination, except in the Western Cape.

40

Printing, Packing and Distribution

e) In the Northern Cape, only in schools that are far from the districts, will question papers be stored at schools.

f) Western Cape will deliver to schools in weekly consignments.

41


writing

Writing of the examinationWriting of the examination

a) Examination commences on 27 October 2014 and concludes on 28 November 2014.

b) Invigilators who supervise the writing of exams are in the process of being trained across all PEDs.

c) A common national Manual on Invigilation is used across all examination centres.

d) To inculcate a sense of commitment to complying with the examination code of conduct, all learners will sign a Pledge on 17 October 2014, at a Pledge Signing Ceremony.

43

Writing of the examinationWriting of the examination

e) The writing of the examinations will be monitored by DBE, PEDs and Umalusi. DBE has appointed 30 independent monitors who will be deployed to the provinces.

f) Schools with previous irregularities will be closely monitored.

44


script collection

Collection of Answer Scriptsa) All PEDs have a clear control process for the management of

Answer books.

b) All scripts to be returned to the district office the same day.

c) Scripts to enjoy the same attention and security as the question paper.

d) Four PEDs will be using bar coded labels in the 2014 NSC examination. This will enable PEDs to scan scripts and ensure better script control.

e) All scripts must be correctly labelled with centre number, subject and must be sealed at the school level.

46

Collection of Answer Scripts

f) All scripts to be controlled and checked at each stage of the process.

g) All PEDs will follow specific norm times for the return of scripts between the different collection points.


marking

MarkingMarkinga) Markers have been appointed based on the PAM criteria

and most PEDs have added learner performance as an additional criteria.

b) National Marking Guideline discussions will be hosted in Pretoria for all subjects, to ensure standardisation of marking across all PEDs.

c) Internal moderators to be appointed for each subject at each marking centre.

d) DBE will appoint external moderators to evaluate the marking at each centre.

49

• Tolerance range (TR) introduced for the first time to control differences in moderation.

• Tolerance range (TR) is the agreed degree of deviation between the marked and moderated mark.

• Tolerance range should not exceed 2 – 3% - at the total mark level and at the question level

• TR exceeded:- in more than 50% of the scripts – remark batch.- in 50% of the scripts – moderate additional scripts.

- less that 50% of the scripts - accept

Tolerance Range

• Chief markers (CM) and Internal moderators (IM) during the national training session will be evaluated on their compliance to the TR.

• After multiple opportunities if the TR is not reached, then the CM or the IM will not be authorised.

• Given that the CMs and IMs have already been appointed, their services cannot be terminated.

• Therefore, DBE will arrange for support for these officials at the marking centre.

Authorisation of CMs and IMs

• Prior to the commencement of marking, each marker will be given a batch of 10 scripts to mark.

• If 50% of the scripts are within the TR, marker is authorised to mark.

• If less than 50% of the scripts are within the TR, the marker is given a second batch of scripts.

• If the marker is still outside the TR, then he is called to a consultative meeting where his future involvement in the marking process is determined.

• Need for a reserve pool of markers.

Authorisation of Markers

Centralised Marking• Centralization of the small enrolment subjects, where the pooling of scripts

from across the nine provinces and allocating the marking of the scripts from all nine provinces to one province, will be piloted in 2014.

• The marking of these small subjects will serve as a springboard to experiment with other subjects with larger enrollments at a later stage.

• A province that is allocated a specific subject/subjects will take full responsibility for the marking of the subject/s. This responsibility would entail: – Appointment of markers, senior markers and chief marker and internal moderator– Training of the markers– Venue for marking– Accommodation of the markers– Management of the marking process– Control and security of the scripts

• Bilateral arrangement between PEDS where the number of scripts is less than 1000.

• The DBE will coordinate and monitor the process nationally to ensure that there are no glitches.

• No PED will be allowed to mark subjects where all the QA levels cannot be complied.

• The DBE will mark centrally - Dance Studies and Agricultural Technology.

• Centralised Marking of Deaf and Braille scripts by WCED and GDE

Centralised Marking - 2014


school based assessment

School Based Assessment (SBA)School Based Assessment (SBA)

a) DBE conducted Audit of provincial SBA moderation systems focused moderation of assessment tasks in 18 districts across all

PEDS, in July 2014.

b) Second national moderation will be conducted in October 2014 and will evaluate the assessment tasks and the learner evidence.

c) Provincial moderation to be completed by 16 October 2014.d) All SBA marks will be captured by 15 November 2014.

56

School Based Assessment (SBA)School Based Assessment (SBA)

f) Common Assessment Task developed by the DBE was administered across PEDs, in Life Orientation, on 5 September 2014.

g) National Marking Guideline discussions will be hosted in Pretoria for Life Orientation, to ensure standardisation of marking across all PEDs

57


processing, release and appeals

Processing, Release and Appealsa) Marking will be completed by 14 December across all PEDs.

b) Mark Capture will be completed by 18 December 2014.

c) Proposals in respect of adjustments for the pre-standardization

meeting – 20 December 2014.

d) Pre-standardization meetings will be held on 21 & 22 December.

e) Umalusi standardisation meeting to be held on 23 December 2014.

f) Results will be thoroughly checked by DBE, PEDs and Umalusi from

23 – 30 December.

g) Umalusi approval meeting to be held on 30 December 2014.

59

Processing, Release and Appeals h) Ministerial announcement on the 2014 NSC Results: 5 January

2015. i) Release of results to candidates by schools : 6 January 2014

j) Remark and re-check will take place early in February 2014

k) Candidates given 14 days after the release of the results, to apply for a re-mark or re-check.

l) If candidates are still not satisfied, with the outcome, candidates will be allowed to view his/her script.

m) Based on specific criteria, candidates are allowed to write the supplementary examination.

60


analysis and feedback

Analysis and FeedbackAnalysis and Feedback

a) Detailed analysis of results per school, per district, per province, per

subject will be made available.

b) Qualitative analytical reports from chief markers/internal moderators will

be consolidated for distribution to schools.

c) Outcomes of the Umalusi standardisation meetings will be made available

to curriculum specialists and examining panels.

d) Workshops will be conducted with teachers and subject advisors.

e) Under-performing schools will be brought to account and improvement

plans will be designed.

f) Curriculum specialists to develop subject improvement plans.


certification

Certification

a) The issue of the certificate signifies the culmination of the examination cycle.

b) The certificates for candidates that wrote the examination in 2013 was distributed to candidates at the end of June 2013.

c) The 2014 certificates are scheduled to be released to

candidates by end May 2015.


irregularities

Irregularities

The serious irregularities experienced in 2013 and the action taken:(a)Dramatic Arts – question on rape.(b)Marker appointments in KZN.(c)Lost scripts(d)Copying (reduced)


progress chart

Progress ChartTARGET INDICATORS PROGRESS

1. High quality question papers are set and moderated

2. Examination centre and candidates are accurately registered on the examination computer system.

3. Question papers are accurately printed. In progress

4. Question papers are distributed on time to all schools and candidates and strict security measures adhered to in the distribution process

All preparations in place

5. All candidates including learners with special needs are accommodated in the examination.

6. Examinations are conducted under controlled conditions across all examination centres.

All preparations in place

Progress ChartTARGET INDICATORS PROGRESS

7. Answer scripts accurately marked and moderated. Marker appointments being finalised; marking centres established.

8. All examination marks are accurately captured on the examination computer system.

Data capturers being appointed; capture centres established

9. SBA is completed in accordance with policy prescripts and moderated.

In progress

10. All examination and SBA marks are accurately captured. Data capturers being appointed; capture centres established.

11. All examination irregularities are appropriately managed. Structures and training in process

12. All candidates are accurately and timeously resulted. System enhancements being finalised

13. Effective feedback provided to schools. Diagnostic analysis

14. All eligible candidates receive certificates. Systems in place


ANNUAL NATIONAL ASSESSMENT

Introduction(a) DBE, working together with PEDs, will administer ANA tests on all

learners in grades 1-6 & 9, plus a pilot study on representative samples of schools and learners at grade 7&8 levels from 16 to 19 September 2014

(b) Time table for the assessment was sent to PEDs and schools in the first quarter of the 2014 school year

(c) Key systems for continuous monitoring of preparations and processes of ANA include bi-monthly meetings with Heads of Exams from PEDs and at least one monitoring visit to individual PEDs by the DBE before the tests are written

(d) DBE 2014 visited all PEDs between May & June and the focus was on assessing the state of readiness for ANA 2014.

Key elements of the ANA process

1. Test development & quality assurance

2. Registration of learners on the GET system.

3. Printing, packing and distribution of tests to schools

4. Administration of tests in schools

5. Marking and moderation of learner responses

6. Capturing, cleaning and analysis of data

7. Reporting

8. Monitoring of the processes - ongoing

9. Independent verification if the processes and outcomes of the assessment

Test development & quality assurance

• Language and mathematics tests for all the affected grades were developed by teams of selected educators and subject advisors

• For quality assurance, separate teams moderated and edited the tests

• Tests for grades 3, 6 & 9 were piloted in selected schools across the provinces in February 2014.

• In addition, the tests for grades 3, 6 & 9 were submitted to experts who serve on the ANA Advisory Committee for comments and all received inputs were used to finalise the tests

Test development & quality assurance

• For the Foundation Phase all tests are versioned into the eleven official languages.

• All tests were print-ready by end of May 2014 and adaptations for learners with special learning needs have been completed which include Brailling for learners who are blind, large font size for those who experience sight challenges and videos for learners who are deaf

Registration of learners

(a) Registration of learners includes capturing essential information that will be used in:-

• Determining the numbers of test booklets to print, pack and distribute per grade, subject and language

• Preparing reporting templates so that results can be reported at national, provincial, district, school and individual learner levels

(b) To date around 7 million learners have been registered on the GET platform

Registered learners for ANA 2014

Prov.Total Number of learners Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 9

EC 1,043,043 183,372 167,624 156,199 147,452 133,595 126,375 128,426

FS 372,779 57,856 57,075 53,842 52,536 45,904 44,517 61,049

GT 1,270,418 197,264 211,414 195,193 186,866 169,975 159,142 150,564

KZN 1,583,300 272,753 245,297 230,632 220,965 198,261 195,754 219,638

LP 910,142 143,930 137,420 128,029 122,333 106,622 106,625 165,183

NC 172,308 28,909 26,277 25,005 25,241 23,313 22,087 21,476

NW 351,299 69,290 63,688 60,324 58,642 51,369 47,986 0

MP 557,963 96,866 87,994 81,888 78,535 70,032 68,131 74,517

WC 583,814 100,078 91,999 84,032 85,187 76,296 71,605 74,617

SA 6,845,066 1,150,318 1,088,788 1,015,144 977,757 875,367 842,222 895,470

Printing, packing & distribution

• Eastern Cape, Mpumalanga, Northern Cape and Western Cape opted to print their tests using their existing facilities. Three of the PEDs have completed the printing and packing. Only Northern Cape is still busy with the printing.

• Tests for Free State, Limpopo and North West are under print by CTP in Cape Town. KwaZulu-Natal & Gauteng test materials are printed by Lebone Litho Printers in Gauteng. Both service providers must complete distribution to Nodal Points by 12 September

Administration of tests in schools

• Tests will be written in all schools from 16 to 19 September

• Principals will allocate teachers to invigilate classes that they do not teach

• Independent schools that participate in ANA for purposes of being evaluated for subsidies will be invigilated by a district official

• DBE will employ independent monitors to reinforce monitoring that will be done by DBE, PED & District officials

Marking & moderation of scripts

• Marking will be preceded by memo discussions at a national and district level

• Teachers will mark scripts of their learners after writing

• SMTs will moderate the marking before schools issue reports to parents

• Samples of grade 3, 6 & 9 scripts from all schools will be moderated by appointed markers at centres managed by PEDs

• Marking centres will provide feedback to schools on the findings of moderation

Marking & moderation of scripts

• All participating independent schools will be marked

by appointed markers at the centres that have been

selected and will be managed by the PEDs

Capturing, cleaning & analysis of data

• All marks and relevant assessment data per learner will be captured on existing IT systems such as SA-SAMS, CEMIS and others but will eventually be uploaded onto the national GET system.

• Trained Systems Administrators will clean the data and prepare it for analysis

Reporting

• DBE professionals will compile reports – a national report that reflects provincial and district results & a diagnostic report that identifies the skills and knowledge that learners were able or not able to demonstrate in the assessment

• The Minister will release the national report in December

• The diagnostic report will be disseminated to schools in January so that teachers can plan how they will address the identified learning deficiencies in the new year

Verification of ANA

• For independent verification of all the processes as well as the outputs of ANA the DBE has appointed a service provider to conduct the verification.

• Service provider will monitor the administration of the tests in a random sample of 125 schools per province during the week of 16 to 19 September

• They will independently mark, process and report performance from the sampled schools at grades 3, 6 & 9

Conclusion

• DBE, together with PEDs, have monitored the preparations for ANA 2014 through bi-monthly meetings. All reports from PEDs indicate that PEDS are ready for ANA 2014

• Provincial visits that were conducted in May/June by the DBE verified the state of readiness of each province

• All schools have received the time table for ANA 2014

• Based on an evaluation of the key components of ANA, the system is ready for ANA 2014

Conclusion

• The DBE and the PEDs are ready to administer the 2014 NSC examination and ANA 2014

• The DBE will vigorously pursue its agenda of building a national examination and assessment system, that is internationally comparable and credible.

• DBE is harnessing the untapped potential of learner assessment.

• Data analysis and data utilisation to enhance teaching and learning is the focus.

Presentation Outline

Documents

Transcript of Presentation Outline