Using Tracer Surveys to Track Impact of Training & … Tracer Surveys to Track Impact of Training &...

Using Tracer Surveys to Track

Impact of Training & Skills Programs

Presentation at the worshop

“ Let’s Work - Pillar 2: Measuring the Impact of Private Sector Investments on Jobs”

London 16-17 September, 2015 (hosted by CDC)

By Harald Schomburg

1. Background – INCHER Kassel 2. Theoretical framework 3. Key aspects of the methodology of tracer studies 4. Conclusions

Outline

Background: International Tracer Study Cooperation

INCHER-Kassel

Africa Cameroon, Ghana, Kenya,

Malawi, Namibia, Nigeria, South Africa, Uganda, Tanzania

Asia Indonesia, Iran,

Japan, Kyrgyzstan, Oman, Philippines,

Vietnam

Europe Austria, Belgium, Czech Republic, Estonia, Finland,

France, Italy, Netherlands, Norway, Portugal, Romania, Serbia, Slovenia, Spain, Sweden,

Switzerland, United Kingdom

Central America Costa Rica, El Salvadore, Honduras, Guatemala,

Mexico, Nicaragua, Panama

11/09/2015

INCHER - University Kassel: in the Heart of Germany, in the State of Hesse

11/09/2015

Located in the center of

Germany and Europe

Unique unity of urban

culture and landscape

City of the documenta

Parks and recreational areas

Center for industry, services

and civil administration

of nothern Hesse

1. Background – INCHER Kassel

2. Theoretical framework 3. Key aspects of the methodology of tracer studies 4. Conclusions

Outline

What are (Graduate) Tracer Studies in Education?

Similar terms Graduate survey

Tracer study

Alumni research

Follow-up study

Graduate career tracking

Target population Graduates of an institution of VET/HE (regardless the kind of degree)

Surveyed after graduation

11/09/2015

High Demand on Tracer Studies

Tracer Studies are very often a key instrument for monitoring and evaluation in projects of donor agencies (like ADB, GIZ, JICA) in the VET and HE area

Accreditation and accountability

Labor market relevance as a key indicator of quality

Assessment and further development of study/traing programmes based on LM feedbacks

11/09/2015

2. To evaluate

the relevance of

VET/HE (assessment)

3. To contribute to the

accreditation process

(accountability)

Objectives of Institutional VET/HE Tracer Studies

11/09/2015

1. Development of the

VET/HE institution

(e.g. curriculum)

4. To inform students,

parents, lecturers and

administrators

The Conceptual Framework of Tracer Studies 10

Student Input Students‘ bio

data, experiences, motives

Output Kowledge, skills,

competences, motivation,

grades

VET/HE Input Study conditions and provisions

Process Teaching and learning

Tracer study

Context: Labour market – region – country; socio-economic development and personal development

11/09/2015

Labor market signals

Outcomes Transition,

employment, work, professional success

service to society

11/09/2015

11 Socio-biographic background

Parents‘ education Gender

Motives and abilities Prior

education

VET/higher education Structures Study conditions Curricula

Study behaviour

Labour market

conditions

Socio-cultural

conditions

Region

Country

Competences

Employment and work

Globalisation, internationalisation

Knowledge society

Life course

Transition process

Key Theoretical Problems of Graduate Surveys

1. Which indicators do we need?

2. How to measure the labour market signals?

3. Which factors are relevant to explain the success of the graduates besides education?

4. To what extent is the professional success/career caused by VET/HE?

5. Which elements of VET/HE (study conditions and provisions, etc) have an impact?

11/09/2015

Common Indicators of Professional Success – Labor Market Signals – Employment Outcomes

Objective indicators Short search duration Employed or further study High income Appropriate position regarding level of education (job title; ISCO) Work tasks closely related to field of study (description of work tasks)

Subjective indicators Perceived links of kind of job and own level of education Perceived high use of competences Perceived high status (income, career perspectives) Perceived high work autonomy Perceived high job satisfaction

11/09/2015

Professional success:

A multi-

dimensional concept

Outline

Traditional and New Type of Graduate Surveys

Traditional type – representative sample survey to inform the Ministry and other key stakeholders

Since more than 30 years E.g. France, Germany, Italy, Norway, Switzerland Irrelevant for individual institution

New type – institutional graduate surveys (all graduates from one institution of VET/HE)

(France), Germany, Hungary, Italy, Indonesia, the Netherlands, Romania Combination of a National Monitor and a feedback instrument for VET/HE institutions Relevant for individual institutions (reports for individual institutions, study program level breakdown of results)

11/09/2015

The Old Type of (National) Tracer Studies: the School/University Delivers Data (e.g. Addresses)

Consultants; Research institute

University A University B University C

Researcher

Ministry of HE: Donor agency

11/09/2015

The New Type of Graduate Surveys: the Network Approach (KOAB)

INCHER-Kassel

Uni …

Uni 80

2012: About 80 institutions of HE in Germany are cooperating in conducting regular tracer studies

11/09/2015

The Bridge Concept – Institutional Development and Research

Institutional Tracer Study

Vertical match – overeduction/

undereducation

Skills mismatch - study and

Study conditions and

professional success

Evaluation, information,

marketing

Alumni and career service

Quality assurance

(accreditation)

11/09/2015

Research Institutional Development

The New Approach of Tracer Studies

Long term perspective: implementation of regular tracer studies

Capacity buildung • Tracer Studies should be organized as a learning process

• Training and cooperation (network) is needed

High quality of data (standardisation of processes)

International standards (design, use of instruments, comparison of results)

11/09/2015

Key Issues of the KOAB Tracer Studies

Every university has an individual questionnaire (2009: 160 surveys were administrated at the same time by INCHER-Kassel)

Every university has a project coordinator (= researcher)

At least 6 training workshops per survey + national and international conferences

Response rate: 50 %

11/09/2015

The New Tracer Study Network Approach: Capacity Building for a Learning Organisation

Staff training

Field phase/data collection

Staff training

Data analysis and

interpretation

Staff training

Concept and planning

11/09/2015

A cycle of 12 – 24 months

Quality assurance (and increasingly accreditation)

Alumni-tracking

National monitoring of transition from higher education to work

Study choice information for school-leavers (both at national and institution level)

Input for labour market forecasting

Scientific and policy analyses

11/09/2015

22 Multifunctional Tracer Surveys – the Netherlands

Source: Jim Allen (ROA, University of Maastricht, Netherlands), Kassel 2008

Information for political authorities (education/employment sectors)

Benchmarking (system of indicators of Swiss higher education institutes)

Evaluation and information tool for higher education institutes

Information for career/study counselling services

11/09/2015

23 Multifunctional Tracer Surveys - Switzerland

Source: Andrea Witmer (Swiss Federal Statistical Office), Kassel 2008

Design of Graduate Surveys

11/09/2015

1- 2 years 4 - 5 years

Time After Graduation

Seldom

Exit Survey

Most frequent

Graduate Survey I

Student survey Graduate survey

Less frequent

Graduate Survey II

11/09/2015

25 University of Indonesia:

High Response Rate – High Use of the Results

Source: Fikawati/Syafiq: Sustaining Tracer Study at University Level: Universitas Indonesia Experience, EXLIMA Conference Bali, 2012

11/09/2015

26 University of Indonesia Tracer Study

11/09/2015

27 Low Budget Example Guatemala (UNITRACE):

Successful Methodology

1. Background – INCHER Kassel 2. Theoretical framework 3. Key aspects of the methodology of tracer studies

4. Conclusions

Outline

Low Quality of Many Tracer Studies

Tracer Studies are very often done by local consultants with little experiences

High costs and low quality low response rate

misinterpretation of findings

Local consultants are often not following international standards – e.g. they do not compare their study with others

Only a few studies are published

11/09/2015

Lessons Learned from Tracer Studies in Different Countries

High response rates are possible to achieve in almost all countries

High quality of the data is a pre-requisite of the relevance of the findings for research and institutional development

Availability and quality of addresses (methods used to update addresses) seems not to be a real problem

Documentation of survey procedures and data editing is necessary – culture of critique must be developed

11/09/2015

Open Questions

How to ensure comparability of Tracer Studies?

Who collects the data and is responsible for data quality?

How to build a system of Tracer Studies which is flexible and relevant for the institutions/sectors and for researchers (and the Ministry of Education/Employment) at the same time?

How to exchange systematically the experiences to improve the future Tracer Studies (building a learning culture)?

11/09/2015

Tracer Study Researchers from Africa, America, Asia and Europe

International Conference „Experiences with Link and Match in Higher Education: Results of Tracer Studies Worldwide“ (Bali, 2012)

11/09/2015

www.exlima.org

Thank you very much for your attention

Further information and tools: Web site http://ingradnet.org

Harald Schomburg (2014): Carrying out tracer studies.

In: ETF/CEDEFOP/ILO (eds.): Guide to anticipating and matching skills and jobs. Volume 6. Torino: ETF (in print).

Using Tracer Surveys to Track Impact of Training & … Tracer Surveys to Track Impact of Training &...

Documents

Transcript of Using Tracer Surveys to Track Impact of Training & … Tracer Surveys to Track Impact of Training &...

Clinical Tracer for department 2010 update [โหมดความเข้า ...medinfo2.psu.ac.th/qc/file_topic/HA/Clinical Tracer... · Clinical Tracer มมประโยชนอยางไรีประโยชน

Tracer Study Concepts and Principles - ingradnet.orgingradnet.org/surveys/INDONESIA_UI_2012_syafiq_fikawati.pdf · One of main indicators of graduate employability is the ... Psychology,

THE STATISTICAL MEASUREMENT OF BUSINESS ...2017/02/17 · Keywords: small entrepreneurship, business conditions, composite indicators, cycle tracer, business conditions surveys JEL:

Cisco Packet Tracer Overview - Freedemay.iut.lr.free.fr/doc/Packet Tracer/Packet Tracer 6.1.1 Overview...• Cisco Packet Tracer Overview ... network “world” Packet Tracer is an

"Review of tracer surveys"

OASIS Facilitators Guide · Web viewTools and methods include tracer studies, post-award surveys, alumni impact assessments, case studies and other small-scale evaluation studies.

Cisco Packet Tracer Overview - WikispacesPacket+Tracer... · Cisco Packet Tracer Overview ... Packet Tracer Practice SBA ... •Each CCNA course has a final hands-on skills

Technical and Vocational Education and Training in the ......10.3 Sample Identified from the Individual Graduate Tracer Surveys, 2013–2014 and 2017 179 A10.1 Coefficient Estimates

WILDLIFE CORRIDOR MONITORING STUDY...methods. These sign surveys along the track station transects are thus similar to surveys conducted by the SDTT. The SDTT monitored wildlife use

Tracer Controls, Tracer VV550/551 VAV Controllers · October 2005 CNT-PRC003-EN Tracer™ Controls Tracer VV550/551 VAV Controllers ®

ACKNOWLEDGEMENTS - NHSSP · This tracer list would ideally be integrated in the existing Logistics Management Information System (LMIS) and also be used by health facility surveys

Line Tracer

Packet Tracer Network Simulator Book Amazon | Cisco Packet Tracer

Israel Country Profile 2013 - Enterprise Surveys/media/GIAWB/Enterprise...The surveys are repeated over time to track changes and benchmark the effects of reforms on firms performance.

Global Repeat Hydrographic/CO 2/Tracer Surveys in Support ...cpo.noaa.gov/sites/cpo/COD2/2014 Progress Reports/FY14_Progress… · AOML technicians facilitated the fabrication of

Workshop Tracer Studies in Central America: Findings …ingradnet.org/surveys/Nicaragua_UNAN_2013_ruiz_rivera.pdf · (CISE) Rising institutional Consciousness of the strategic importance

Tracer un triangle Tracer un triangle 2 Fiches MHM

VOLUNTEER CARNIVORE TRACK SURVEYS - Part of an Integrated Approach to Monitoring Wisconsin Wolves by Jane E. Wiedenhoeft.

TRACER STUDY (TS) ONLINE KEMRISTEKDIKTI · Introduction • Pencatatan Tracer Study : o Pemetaan hasil tracer secara nasional o Pengelolaan data tracer study institusi / PT • Periode

Australia Awards Global Tracer Facility Tracer Survey ...dfat.gov.au/people-to-people/australia-awards/Documents/australia... · Australia Awards Global Tracer Facility Tracer Survey: