Large scale surveys as campaigns
-
Upload
sonia-whiteley -
Category
Education
-
view
80 -
download
4
Transcript of Large scale surveys as campaigns
Large scale surveys as campaigns: social media and the
Student Experience SurveySonia Whiteley
The Social Research Centre Seminar Series
Overview
1. About the QILT surveys
2. QILT & the QILT brand
3. Surveys as campaigns
4. Initial outcomes
QILT survey campaigns 2
About the QILT surveys
• The QILT survey suite consists of: The Student Experience Survey (SES), formerly
the University Experience Survey (UES)
The Graduate Outcomes Survey (GOS), and
The Employers Satisfaction Survey (ESS)
• All are cross sectional, point-in-time surveys except the GOS, which is longitudinal.
• All higher education providers are in-scope for QILT.
QILT survey campaigns 3
The Student Experience Survey
• The University Experience Survey (UES) / Student Experience Survey (SES) involves: Undergraduate higher education students
Who are onshore in Australia, and
Commencing their studies or in the final year of their degrees• More than 100,000 surveys completed each year• Data is collected online.
QILT survey campaigns 4
The Graduate Outcomes Survey
• The in-scope population is undergraduate and postgraduate higher education students who completed their qualification in the six months before
data collection
as an onshore student• Graduates are surveyed twice a year, approximately four
months after they have completed their qualification• Data is collected online using a similar approach to the
SES.
QILT survey campaigns 5
The Employers Satisfaction Survey
• All supervisors of graduates eligible for the GOS are in-scope for the ESS
• Two trials of the ESS were undertaken: a methodological trial in conjunction with the October 2014 AGS
a content trial (revised Employer Satisfaction Questionnaire – ESQ) in conjunction with the April 2015 AGS
• ‘Encouraging’ graduates to provide contact information for their supervisor remains a ‘challenge’ for the live implementation of the survey
QILT survey campaigns 6
QILT data collection periods
QILT survey campaigns 7
Domain 2015 2016 2017
Student Experience Survey August/September August/September August/September
Graduate Outcomes Survey
May/June & November/December
May/June & November/December
May/June & November/December
Employer Satisfaction Survey
May/June & November/December
May/June & November/December
May/June & November/December
Graduate Outcomes SurveyLongitudinal
- March/April March/April
The QILT ‘brand’
• QILT is the overarching or ‘corporate’ brand and is used as the anchor point to integrate the ‘brand family’ through The website Facebook Official communication Media releases Workshops and fora, and Conference sponsorship.
QILT survey campaigns 8
The QILT brand (2)
• The QILT brand is intended to be a constant, recognisable presence throughout the student’s higher education experience and their outcomes post-graduation
• QILT is positioned as an independent ‘feedback friend’ students and graduates expect to encounter throughout and after their studies
• Even if a particular survey/survey invitation is not immediately recognised by the recipient, the look, feel and tone should be familiar and reassuring while retaining a sense of ‘official importance’
QILT survey campaigns 9
The QILT brand family
• Student Experience Survey
• Graduate Outcomes Survey
• Employer Satisfaction Survey
QILT survey campaigns 10
The QILT brand family (2)
• The brand family or ‘products’ are the three individual surveys that have their own logos, colour schemes and brand guidelines.
• The SES brand is more informal, typically targeted at younger students / recent school leavers who may be less likely to engage with QILT
• The GOS brand has a more formal remit and has been designed to appeal to a broader group that includes more mature postgraduates
• The ESS is the closest in tone to the QILT brand with more of a sense of official, quality assurance
QILT survey campaigns 11
Surveys as ‘campaigns’
• Annual frameworks include: Institutional Liaison Strategy
Student/Graduate Engagement Strategy
Employer Engagement Strategy
• Each survey is supported by a: SES or GOS Communications & Marketing Plan, and an
Institutional specific plan
QILT survey campaigns 12
Wait a minute - Why bother?
• Combination of push and pull factors
• Pushed towards trying a new approach by Low online response rates
Declining online response rates
Problems with representativeness
• But also pulled into innovative activities by A large in-scope population for the surveys
Engaged (mostly) staff at institutions
Clients with vision and a flexible budget
QILT survey campaigns 13
Surveys as ‘campaigns’
• The QILT website is the start and the end point for all survey activity
• A background campaign runs throughout the year
• Awareness raising campaigns run two weeks prior to each survey
• Survey specific campaigns with advertising and sharable social media content are run during fieldwork
• Surveys are also supported by more traditional approaches to response maximisation
QILT survey campaigns 14
QILT website
• QILT website informs student choice through provision of survey results by institution and study area.
The initial indicators include:
• SES – Overall Educational Experience item, plus five scale scores
• CEQ – Overall Satisfaction Indicator, Good Teaching Scale and Generic Skills Scale
• GDS – employment rates, further study rates, and median salaries
QILT survey campaigns 15
QILT Home Page
QILT survey campaigns 16
Comparison chart
QILT survey campaigns 17
Table view
QILT survey campaigns 18
QILT Facebook Home
QILT survey campaigns 19
QILT Facebook Winners
QILT survey campaigns 20
QILT sharable content
QILT survey campaigns 21
QILT sharable content (2)
• https://www.youtube.com/watch?v=uvHBBGPOzUg
• https://www.youtube.com/watch?v=LdFFKYHUqKc
QILT survey campaigns 22
QILT coverage
• AFR launch - http://www.afr.com/news/policy/education/for-the-first-time-new-qilt-website-tells-students-about-salaries-after-graduation-20150915-gjnczj
• Australian, league tables under fire - http://www.theaustralian.com.au/higher-education/qilt-website-league-tables-under-fire/story-e6frgcjx-1227569138980
• AFR, lesser known job outcomes - http://www.afr.com/leadership/careers/jobs/lesserknown-universities-achieve-best-job-outcomes-20151012-gk6x16
• Queensland Times, USQ winning - http://www.qt.com.au/news/uni-tops-employment-stats-for-graduates/2808553/
• Business news, WA grads - https://www.businessnews.com.au/article/WA-grads-record-second-highest-median-salaries
• PS News - http://www.psnews.com.au/tvic/473/news/qilt-weaves-quality-comparisons
• AFR, sandstone Unis - http://www.afr.com/leadership/oops-graduates-of-sandstone-unis-dont-earn-less-melbourne-institute-admits-20150916-gjodvr
SRC Board 23
QILT & social media
QILT survey campaigns 24
QILT & social media (2)
QILT survey campaigns 25
QILT & social media (3)
QILT survey campaigns 26
Survey response maximisation
• Rolling prize draw for each of the four weeks of fieldwork• Seven email reminders • Reminder letter or postcard (to coincide with the second
email reminder)• SMS reminder(s)• Institution specific reminders• Survey links in Learning Management Systems• Promotional materials – website tiles, PowerPoint slides for
lectures, handouts & posters
QILT survey campaigns 27
Survey response ‘cycle’
QILT survey campaigns 28
Initial outcomes
• Still early days and challenging to identify what works best for each survey (ie SES vs GOS)
• Difficult to disentangle which strategy is having the most impact
• Very different outcomes for individual institutions – different levels of student and staff engagement
• QILT website not launched early enough to support the 2015 SES
QILT survey campaigns 29
Initial outcomes (2)
• Online fieldwork completed in four weeks
• 79 participating institutions
• More than 140,000 responses
• 38% response rate, up from 30% in 2014, 29% in 2013, and 21% in 2012
• Increase in the amount of reportable data on the website but…still issues with representativeness
QILT survey campaigns 30
Where to for 2016?
• Same prize draw and reminder schedule• Postcard rather than letter reminders• Main QILT website live (and hopefully updated)• New & refreshed videos• Paid and free social media activity• Engagement with mainstream media• Campaign website to support more effective tracking of
social media strategies• Early work to develop institutional level communication
strategies targeting underperforming unis/NUHEIs and study areas
QILT survey campaigns 31