Monitoring SQA Qualifications over time
description
Transcript of Monitoring SQA Qualifications over time
![Page 1: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/1.jpg)
![Page 2: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/2.jpg)
Monitoring SQA Qualifications over time
Dr Rob van Krieken
![Page 3: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/3.jpg)
Another bridge in Arnhem
![Page 4: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/4.jpg)
![Page 5: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/5.jpg)
Content
Why is SQA monitoring over time? What are the standards SQA is monitoring? SQA’s monitoring programme Implementation Reports and actions
![Page 6: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/6.jpg)
Why is SQA monitoring over time?
to avoid
‘creep’ within a stable system
shock when the system changes
![Page 7: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/7.jpg)
Example of “creep”
Year 1 Year 2
Cut-off % Pass Cut-off % Pass
49 86% 49 87%
50 80 % 50 81 %
51 72% 51 73%
![Page 8: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/8.jpg)
Example of “creep”Year 2 Year 3
Cut-off % Pass Cut-off % Pass
49 87% 49 88%
50 81 % 50 82 %
51 73% 51 74%
![Page 9: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/9.jpg)
Example of “creep”Year 3 Year 4
Cut-off % Pass Cut-off % Pass
49 88% 49 89%
50 82 % 50 83%
51 74% 51 75%
![Page 10: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/10.jpg)
Example of “creep”
Year 1 Year 4
Cut-off % Pass Cut-off % Pass
49 86% 49 89%
50 80 % 50 83%
51 72% 51 75%
![Page 11: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/11.jpg)
System changes Vocational qualifications are reviewed
systematically at the end of their duration O-grade to Standard Grade (1984 – 1990s) SCE Higher – Revised Higher (1992 - ) (Revised) Higher - Higher (Still) (2000) CSYS – Advanced Higher (2001) New National Qualifications (2014-2016)
![Page 12: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/12.jpg)
What are the standards SQA is monitoring?
‘Minimum level of demand’– Laid down in specifications (“standards”)– Exemplified in assessments/marking/boundary
Within the educational context of– SCQF levels, similar qualifications– Quality and quantity of teaching and learning– Cohort
![Page 13: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/13.jpg)
Monitoring programmes
Different programmes for National Courses and Vocational Qualifications (HN and SVQ)
Sampling by subject area and level
Cycle depends on qualification / level
![Page 14: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/14.jpg)
Sample Higher National Diplomas2014 • 2015 2016 2017 2018 2019
Beauty Therapy Administration and
Information Technology
Early Education and Childcare: Childhood Practice
Social Care Social Service
Hospitality Management
Computing: Software Development
Travel and Tourism
Nautical Science
Early Education and Childcare
Beauty Therapy Administration
and Information Technology
Social Care
Hospitality Management
Computing: Software Development
Social Service
Travel and Tourism
Nautical Science
C&ACollect and archive
C&CCollect and Compare
![Page 15: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/15.jpg)
Sample Higher National Qualifications2013 2014 2015 2016 2017 2018 2019 2020
English C&A C&C C&C
Mathematics C&A C&C C&C
Art and Design C&A C&C
Business Management C&C
Chemistry C&A C&C
Computing C&C
Engineering Science C&C C&C
French C&A C&C
Geography C&A C&C
RMPS C&A C&C
![Page 16: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/16.jpg)
Implementation: Panels, materials, and tasks
Panel: – 3 subject experts
Materials:– Specification– Assessments, Marking Instructions– Evidence– Judgements
Task: – Questionnaire
![Page 17: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/17.jpg)
Questionnaire
1 Educational Context:
Are there any general changes that may have influenced the general ability of candidates in one of the years, in particular the way they meet the demands made by the syllabus and exam?
![Page 18: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/18.jpg)
Questionnaire
2 Unit Specification or Standards and Guidance2.1 Are there differences in the Unit Specifications and Guidance that affect the level of demand? 2.2 Were the Unit Specification Standards in
2011 on the whole more / no more / less demanding than before?
![Page 19: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/19.jpg)
Questionnaire
3 Assessment
3.1 Are there differences in Assessment practice that affect the level of demand? Please tick the relevant boxes and add if necessary.
3.2 Was the Assessment in 2011 on the whole more / no more / less demanding than before?
![Page 20: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/20.jpg)
Questionnaire
4 Quality and Judgement of Evidence
4.1 Are there differences in the quality of the evidence and in the way this has been
judged? Please tick the relevant boxes and add if necessary.
![Page 21: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/21.jpg)
Questionnaire
5 Overall judgement
A reasoned overall judgement of the level of demand of the qualification as a whole, taking into account standards, assessments, and evidence.
![Page 22: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/22.jpg)
Reports and actions
Overall report
Summary of conclusions
Actions SQA undertakes
![Page 23: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/23.jpg)
Issues and answers 1
Too much change
Overall judgement
Quantity and type of evidence
Panel membership
No looking Backward
![Page 24: Monitoring SQA Qualifications over time](https://reader035.fdocuments.net/reader035/viewer/2022062410/5681666b550346895dda0423/html5/thumbnails/24.jpg)