Post on 25-Dec-2014
description
Students as Partners in creating our public reputation
Contents
• What are league tables for?• Student input to league tables.• How league tables are
constructed.• What do they say about us?• How should we respond?
Where do you think we should be in a university league table?
Why are they important?
• Used in recruitment decisions – maybe.• Reputation – definitely• How our students see us• Our own self-worth
What are the problems with League tables?• League tables use the most recent data sets available
at the time of publication. • There is a time-lag when using publishing data sets and
time series data.• Different statistical analysis, methodology and
weightings are used. In the different tables• There can be changes to criteria and methodology• Data sets can change• Difficult to make comparisons between subjects and
between competitors at this lower level.
More problems....
• Categorisation of courses and subjects are different. And only concentrate on FT UG provision – this University is about so much more than that
• The editorial that appears alongside each University entry is not always available to the University to update or influence prior to publication.
• Information is often not the same as we might use internally
• Newspapers love to trumpet obvious, but wildly misleading headlines
However – we are stuck with them…..
....so, let’s make sure we understand what they say about us, and try to make them work for us
Over-riding importance – Improving Student Outcomes
• We need to allow students to reach their full potential– Celebrating individual success– Maximising individual rewards– Maximising contribution to society– How do we do this in areas of low aspiration?
• Students are increasingly consumers– Access price comparisons – is there price
sensitivity?– Access to performance comparisons
Impact of student outcomes on the University
• KIS• Unistats• National Student Survey• League Tables• Which? University Guide• College and University websites• The Complete University Guide• ……etc etc
Student Outcomes in Public Information
• Inputs– Spend per student– Staff student ratios– Entry standards– Research ratings– Cost of living– Spend on services– Faculty spend
• Outputs– Number of “good”
degrees– National Student
Survey results– Retention rates– Employability
Student Input to Public Information
• National Student Survey– Measure of final year
students– Questions on
satisfaction with course, feedback, teaching
• Used in:– League tables/KIS
(external)– Portfolio performance
review (internal)– Award annual
monitoring (internal)
Ones to watch
• Complete University Guide (May)• Guardian University Guide (June) *• Times/Sunday Times Good University Guide
(August)
• * - in University strategic plan…..aim to be in top 50
What do they say about us now?
YEAR THE COMPLETE
UNIVERSITY GUIDE
THE GUARDIAN
THE TIMES THE SUNDAY TIMES
2011 80 69 77 96
2012 99 77 89 105
2013 108 96 100 107
2014 113 92 ??? ???
SU positions in Complete University Guide for each factorEntry Standards 111/124
Student Satisfaction 54/124 <- good!!
Research Assessment 111/124
Graduate Prospects 116/124
Staff student ratio 87/124
Academic Services Spend 94/124
Facilities Spend 62/124
Good Honours 103/124
Degree Completion 108/124
Overall 113th
150 200 250 300 350 400 450 500 550 600 6500
10
20
30
40
50
60
70
80
90
100
% Good Degrees (56.3) vs Entry Standards (251)
entry standard
% g
ood
degr
ees
150 200 250 300 350 400 450 500 550 600 6500
10
20
30
40
50
60
70
80
90
100
Graduate Prospects (48.9) vs Entry Standards (251)
entry standards
grad
uate
pro
spec
ts
0.5 1 1.5 2 2.5 3 3.50
200
400
600
800
1000
1200
Overall Score (453) vs Research (1.62)
research assessment
over
all s
core
To be successful in the CUG league table :
• increase the number of good degrees awarded – how?
• recruit better qualified students who are more likely to get good degrees – WP? Value added?
• with better degrees, more graduates will get graduate entry jobs
• increase research assessment scores – limit number submitted to REF
Factor Source Weighting% Satisfied with Teaching NSS 2012 10%% Satisfied overall with
courseNSS 2012 5%
Expenditure per student (FTE)
HESA data for 2010–11, and 2011–12
15%
Student:staff ratio HESA data for 2010–11, and 2011–12
15%
Career prospects 2010-11 HESA/DLHE data 15%Value added score/10 HESA data (ie the cohort who
graduated in 201215%
Average Entry Tariff typical UCAS scores of students aged 20 or under on
entry (HESA)
15%
% Satisfied with Assessment NSS 2012 10%
Guardian league table factors
2008 2009 2010 2011 2012 2013 2014
Staffordshire 74 67 55 69 77 96 9210
30
50
70
90
110
Overall Position in Guardian Guide
Ovw
eral
l Pos
itopn
2008 2009 2010 2011 2012 2013 2014
Coventry 65 75 75 87 63 46 33
Huddersfield 94 89 96 54 53 48 42
Keele NaN NaN 50 44 54 50 45
Sunderland 82 72 62 52 48 57 65
BCU 61 56 63 66 90 60 61
Plymouth 35 48 51 49 47 61 59
Derby 93 98 105 91 94 70 79
UCLAN 68 62 82 69 70 71 87
Teeside 70 81 69 79 60 78 74
Stafford-shire
74 67 55 69 77 96 92
Liverpool JM 49 109 112 109 111 98 69
Glyndwr 118 103 92 95 102 106 108
10
30
50
70
90
110
130
Comparison with Competitorspo
sition
2010 2011 2012 2013 2014
NSS Teaching (%) 81.5632889924289
80.4511278195489
81.9548872180451
83.25 86
77.5
78.5
79.5
80.5
81.5
82.5
83.5
84.5
85.5
86.5
NSS Teaching (%)
year 2010 2011 2012 2013 2014
NSS Overall (%) NaN 78 81.8181818181818
80 85 NaN
75.0
77.0
79.0
81.0
83.0
85.0
NSS Overall (%)
2010 2011 2012 2013 2014
NSS Feedback (%) 63.3357760830865 65.4 66.2674650698603 68.2634730538922 72
59.0
61.0
63.0
65.0
67.0
69.0
71.0
73.0
NSS Feedback (%)
2010 2011 2012 2013 2014
Entry Tariff 249.418065547561 235.770276900066 238.5 245.50650229837 254
227.5
232.5
237.5
242.5
247.5
252.5
257.5
Entry Tariff
2010 2011 2012 2013 2014
Career prospects (%)
65.1504051666276
63.2253202712886
56.4021995286724
55.1157580283794
48
5.0
15.0
25.0
35.0
45.0
55.0
65.0
Career prospects (%)
2010 2011 2012 2013 2014
Value added score/10
5.24246243984631
5.16113615713964
4.71583068570222
3.97697105816477
3.8
0.5
1.5
2.5
3.5
4.5
5.5
Value added score/10
Are we trying to improving position or performance?• Clearly, we can try to play the game of moving
our league table position• What we really want to do is improve our
performance in each of the key areas to make sure there is a sustainable and genuine change
What steps can we take?
Guardian criteria Suggested Action
Entry standards Review all current standard offers, are we pitching ourselves properly against competitors? Average A level scores have gone up, have our offers?
Student/staff ratio Reviewing more thoroughly the data we submit to HESADeveloping better models of SSR to identify where investment is most needed
Spend per student Reviewing more thoroughly the data we submit to HESAIdentifying capital spend neededIncreased recent spend on libraries and IT will have an impact
Guardian criteria Action
Value added Increase number of “good” degrees awarded. Reviewing all level modules with low pass rates and average marks. Reviewing degree classification rules as part of change to % calculationIdentifying through portfolio review awards with consistently poor progression and attainment
NSS teaching, assessment and feedback and overall satisfaction
Faculty action plans, and award level plansIncreased student engagement with surveySeven principles of feedbackOnline assessment and feedback project
employment Encouraging more students to complete DLHEStaffordshire Graduate – improving our students’ chances of success
Keeping track of how we are doing• Portfolio Performance Measurement
– Provides an internal review mechanisms for award performance
– Records market attractiveness– Retention, progression and “good” degrees– National Student Survey, DLHE
• Future performance measures– Value added (difficult to get the raw data)– SSR at School and subject level– REF
Conclusions
• League Tables are a necessary evil, and a part of the HE landscape
• One of our KPIs is to ‘to be amongst the top 50 institutions in The Guardian league table’
• We all have a part to play, in explaining to students and parents what they really measure,
• Central work on-going on data returns and strategy• Schools working in partnership with our students
on supporting their experience and attainment
@mikehamlyn
blogs.staffs.ac.uk/mgh1/