2014 1

Executive Summary

Measuring the impact of university-business cooperation

2014 2

Europe Direct is a service to help you find answers

to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some

operators phone boxes or hotels may charge you)

More information on the European Union is available on the Internet

(httpeuropaeu)

Luxembourg Publications Office of the European Union 2014

Catalogue Number NC-01-14-299-EN-N

Project Number 20143255

copy European Union 2014

Reproduction is authorised provided the source is acknowledged

2014 2

Europe Direct is a service to help you find answers

to your questions about the European Union

Freephone number ()

00 800 6 7 8 9 10 11

() The information given is free as are most calls (though some

operators phone boxes or hotels may charge you)

More information on the European Union is available on the Internet

(httpeuropaeu)

Luxembourg Publications Office of the European Union 2014

Catalogue Number NC-01-14-299-EN-N

Project Number 20143255

copy European Union 2014

Reproduction is authorised provided the source is acknowledged

2014 3

Measuring the Impact of

University Business Cooperation

(EAC232012)

Authors

Cardiff University Dr Adrian Healy

Imperial Consulting Dr Markus Perkmann

Newcastle University Prof John Goddard

Louise Kempton

Disclaimer

This document has been prepared for the European Commission however it reflects

the views only of the authors and the Commission cannot be held responsible for any

use which may be made of the information contained therein

2014 4

Table of Contents

Introduction 5

Types and drivers of cooperation 6

Benefits of cooperation 8

Assessing the benefits of cooperation 10

Implications for policy and practice 13

2014 5

Introduction

Cardiff University in association with Newcastle University and Imperial Consultants

is pleased to present this report on Measuring the Impact of University-Business

Cooperation to DG Education and Culture (DG EAC)

Promoting and developing cooperation between higher education and business is a

core element of the EUrsquos Agenda for Modernising Higher Education which stresses the

importance of this for pedagogical skills and the development of courses relevant to

social and labour market needs1 The potential to enhance this contribution further

through increased levels of collaboration is now firmly recognized within EU policy

circles and in Member States This is highlighted in the publication of Europe 2020 and

the related Flagship Initiative promoting a new agenda for skills and jobs which

identifies the value of ventures and networks between business and higher educational

establishments to address new skills requirements and labour market needs2

In recognition of the potential benefits to be realized through increased cooperation

between businesses and Higher Educations Institutions (HEIs) in the field of education

the European Commission launched the Knowledge Alliance pilot initiative to create

new multidisciplinary curricula to promote entrepreneurship within education as well

as developing other transferable skills This initially funded a small number of pilot

projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning

Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+

programme

There are many other examples of university-business collaboration in the field of

education across the EU with no involvement of public sector bodies many of which

have been underway for several decades Examples cited in this study include the

Masters Course in Banking Management established in 1989 in Valencia Spain and

the course in Commercial Communications launched in Prague Czech Republic in

1994 As these examples illustrate many cooperation projects are the result of

bilateral agreements between individual companies and HEIs The focus of these

activities can vary widely depending upon the aims and objectives of each particular

example (as identified in the Wilson Review in the UK3)

In contrast to university-business cooperation in the world of research and innovation

the profile of business-university collaboration in the field of education has been

relatively limited This is unfortunate as it is at this level that knowledge exchange

can often most effectively be secured relevant skills developed and the conditions for

future innovation and economic growth laid This low profile has resulted in a limited

literature on the topic and prompted DG EAC to commission this study amongst

others The study had three specific objectives

1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration

2014 6

To analyse existing and identify emerging types of university-business

cooperation in Europe and their drivers

To identify demonstrate and assess tangible and intangible effects of

university-business-cooperation

To develop an assessment methodology and relevant qualitative and

quantitative indicators for measuring the outcomes and impact of university-

business cooperation and in particular of Knowledge Alliances

The study has analysed the following ten cases

Educckate multinational - led by the University College London in

partnership with other EU Universities and their local companies to

stimulate entrepreneurship in cultural and creative industries

EUEN multinational ndash led by Coventry University with transnational

academic and industry partners to promote entrepreneurial education

KnowFact multinational ndash led by the University of Patras with industry

partners from other EU countries to develop the Teaching Factory paradigm

in manufacturing education

AppCampus Finland ndash an innovative initiative between Aalto University and

NokiaMicrosoft involving app Development National

Qatar Carbonates UK ndash a collaborative arrangement between Qatar

PetroleumShell and Imperial College London promoting post-graduate

education International

Newcastle University Subsea UK ndash cooperation between Newcastle

University and regional businesses developing sector relevant skills

Regional

Masters in Banking Valencia Spain ndash collaboration between Valencia

University and regional banking bodies developing sector relevant skills

Regional

HP Bulgaria ndash Cooperation between Hewlett Packard and various

Universities whereby HP trained university lecturers in specialised

technologies and created the project curriculum National

University of Merseburg Germany ndash regional businesses endow professorial

chairs at University of Merseburg to ensure industry relevant education

AKA Prague ndash professional qualification (Certificate of Communication

Agencies) developed by industry and delivered by University of Economics

Prague

Types and drivers of cooperation

The first decade of the 21st Century has seen an increasing emphasis on the role of

universities in explicitly contributing to social and economic development This can be

attributed to a number of factors including

2014 7

The emergence of global lsquogrand challengesrsquo (eg climate change ageing

terrorism etc) which cannot be solved by government or business alone

but requires a multi-disciplinary and collaborative approach

Increased marketisation of higher education leading to greater competition

between universities and an emphasis on the lsquostudent experiencersquo and

student outcomes (including entrepreneurship)

The global economic crisis which has ushered in an era of austerity in public

finances in many countries

The recognition by many firms that working with Universities provides an

advantage for their future recruitment pipelines and ancillary profile

benefits

This has resulted in a range of policies designed to encourage universities to build

stronger links with business suggesting that businesses should be (more) involved in

designing curricula (including undergraduate postgraduate as well as CPD and other

training) and that universities should work more closely with industry partners to

promote entrepreneurship mobility (between business and academia) and lifelong

learning

In practice many universities and businesses have recognized the value of

cooperation particularly in terms of how this can strengthen student recruitment for

universities improve the educational experience for students and provide a supply of

more experienced labour for companies Some companies and universities also

recognize the wider benefits that this cooperation can bring in the form of higher

levels of innovation the upskilling and retention of existing staff and the higher profile

that successful collaboration activities can bring

A number of reports have usefully set out different lsquotypesrsquo of cooperation between

businesses and universities including student and academic mobility programmes

curriculum development and delivery lifelong learning and entrepreneurship4

However most cooperation activities examined for this work incorporated more than

one of these cooperation types This suggests that it is the purpose of cooperation

that is more important rather than the type of cooperation undertaken

Each cooperation partner (business university student) has their own reason to

engage in the cooperation activity but a number of common themes can be identified

These are illustrated in Figure ES1 Three of these relate to strengthening levels of

human capital either of students or of existing workers within companies and

universities to promote a stronger labour supply enhance individual employability

and heighten levels of entrepreneurship These were common across all the projects

examined for this study Three are more related to innovation primarily through

collaborative working on company projects as part of the educational experience or

through the co-production of new courses and course material these were visible in

each of the cases studied but to varying degrees Finally the remaining three

4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 4

Table of Contents

Introduction 5

Types and drivers of cooperation 6

Benefits of cooperation 8

Assessing the benefits of cooperation 10

Implications for policy and practice 13

2014 5

Introduction

Cardiff University in association with Newcastle University and Imperial Consultants

is pleased to present this report on Measuring the Impact of University-Business

Cooperation to DG Education and Culture (DG EAC)

Promoting and developing cooperation between higher education and business is a

core element of the EUrsquos Agenda for Modernising Higher Education which stresses the

importance of this for pedagogical skills and the development of courses relevant to

social and labour market needs1 The potential to enhance this contribution further

through increased levels of collaboration is now firmly recognized within EU policy

circles and in Member States This is highlighted in the publication of Europe 2020 and

the related Flagship Initiative promoting a new agenda for skills and jobs which

identifies the value of ventures and networks between business and higher educational

establishments to address new skills requirements and labour market needs2

In recognition of the potential benefits to be realized through increased cooperation

between businesses and Higher Educations Institutions (HEIs) in the field of education

the European Commission launched the Knowledge Alliance pilot initiative to create

new multidisciplinary curricula to promote entrepreneurship within education as well

as developing other transferable skills This initially funded a small number of pilot

projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning

Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+

programme

There are many other examples of university-business collaboration in the field of

education across the EU with no involvement of public sector bodies many of which

have been underway for several decades Examples cited in this study include the

Masters Course in Banking Management established in 1989 in Valencia Spain and

the course in Commercial Communications launched in Prague Czech Republic in

1994 As these examples illustrate many cooperation projects are the result of

bilateral agreements between individual companies and HEIs The focus of these

activities can vary widely depending upon the aims and objectives of each particular

example (as identified in the Wilson Review in the UK3)

In contrast to university-business cooperation in the world of research and innovation

the profile of business-university collaboration in the field of education has been

relatively limited This is unfortunate as it is at this level that knowledge exchange

can often most effectively be secured relevant skills developed and the conditions for

future innovation and economic growth laid This low profile has resulted in a limited

literature on the topic and prompted DG EAC to commission this study amongst

others The study had three specific objectives

1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration

2014 6

To analyse existing and identify emerging types of university-business

cooperation in Europe and their drivers

To identify demonstrate and assess tangible and intangible effects of

university-business-cooperation

To develop an assessment methodology and relevant qualitative and

quantitative indicators for measuring the outcomes and impact of university-

business cooperation and in particular of Knowledge Alliances

The study has analysed the following ten cases

Educckate multinational - led by the University College London in

partnership with other EU Universities and their local companies to

stimulate entrepreneurship in cultural and creative industries

EUEN multinational ndash led by Coventry University with transnational

academic and industry partners to promote entrepreneurial education

KnowFact multinational ndash led by the University of Patras with industry

partners from other EU countries to develop the Teaching Factory paradigm

in manufacturing education

AppCampus Finland ndash an innovative initiative between Aalto University and

NokiaMicrosoft involving app Development National

Qatar Carbonates UK ndash a collaborative arrangement between Qatar

PetroleumShell and Imperial College London promoting post-graduate

education International

Newcastle University Subsea UK ndash cooperation between Newcastle

University and regional businesses developing sector relevant skills

Regional

Masters in Banking Valencia Spain ndash collaboration between Valencia

University and regional banking bodies developing sector relevant skills

Regional

HP Bulgaria ndash Cooperation between Hewlett Packard and various

Universities whereby HP trained university lecturers in specialised

technologies and created the project curriculum National

University of Merseburg Germany ndash regional businesses endow professorial

chairs at University of Merseburg to ensure industry relevant education

AKA Prague ndash professional qualification (Certificate of Communication

Agencies) developed by industry and delivered by University of Economics

Prague

Types and drivers of cooperation

The first decade of the 21st Century has seen an increasing emphasis on the role of

universities in explicitly contributing to social and economic development This can be

attributed to a number of factors including

2014 7

The emergence of global lsquogrand challengesrsquo (eg climate change ageing

terrorism etc) which cannot be solved by government or business alone

but requires a multi-disciplinary and collaborative approach

Increased marketisation of higher education leading to greater competition

between universities and an emphasis on the lsquostudent experiencersquo and

student outcomes (including entrepreneurship)

The global economic crisis which has ushered in an era of austerity in public

finances in many countries

The recognition by many firms that working with Universities provides an

advantage for their future recruitment pipelines and ancillary profile

benefits

This has resulted in a range of policies designed to encourage universities to build

stronger links with business suggesting that businesses should be (more) involved in

designing curricula (including undergraduate postgraduate as well as CPD and other

training) and that universities should work more closely with industry partners to

promote entrepreneurship mobility (between business and academia) and lifelong

learning

In practice many universities and businesses have recognized the value of

cooperation particularly in terms of how this can strengthen student recruitment for

universities improve the educational experience for students and provide a supply of

more experienced labour for companies Some companies and universities also

recognize the wider benefits that this cooperation can bring in the form of higher

levels of innovation the upskilling and retention of existing staff and the higher profile

that successful collaboration activities can bring

A number of reports have usefully set out different lsquotypesrsquo of cooperation between

businesses and universities including student and academic mobility programmes

curriculum development and delivery lifelong learning and entrepreneurship4

However most cooperation activities examined for this work incorporated more than

one of these cooperation types This suggests that it is the purpose of cooperation

that is more important rather than the type of cooperation undertaken

Each cooperation partner (business university student) has their own reason to

engage in the cooperation activity but a number of common themes can be identified

These are illustrated in Figure ES1 Three of these relate to strengthening levels of

human capital either of students or of existing workers within companies and

universities to promote a stronger labour supply enhance individual employability

and heighten levels of entrepreneurship These were common across all the projects

examined for this study Three are more related to innovation primarily through

collaborative working on company projects as part of the educational experience or

through the co-production of new courses and course material these were visible in

each of the cases studied but to varying degrees Finally the remaining three

4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 5

Introduction

Cardiff University in association with Newcastle University and Imperial Consultants

is pleased to present this report on Measuring the Impact of University-Business

Cooperation to DG Education and Culture (DG EAC)

Promoting and developing cooperation between higher education and business is a

core element of the EUrsquos Agenda for Modernising Higher Education which stresses the

importance of this for pedagogical skills and the development of courses relevant to

social and labour market needs1 The potential to enhance this contribution further

through increased levels of collaboration is now firmly recognized within EU policy

circles and in Member States This is highlighted in the publication of Europe 2020 and

the related Flagship Initiative promoting a new agenda for skills and jobs which

identifies the value of ventures and networks between business and higher educational

establishments to address new skills requirements and labour market needs2

In recognition of the potential benefits to be realized through increased cooperation

between businesses and Higher Educations Institutions (HEIs) in the field of education

the European Commission launched the Knowledge Alliance pilot initiative to create

new multidisciplinary curricula to promote entrepreneurship within education as well

as developing other transferable skills This initially funded a small number of pilot

projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning

Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+

programme

There are many other examples of university-business collaboration in the field of

education across the EU with no involvement of public sector bodies many of which

have been underway for several decades Examples cited in this study include the

Masters Course in Banking Management established in 1989 in Valencia Spain and

the course in Commercial Communications launched in Prague Czech Republic in

1994 As these examples illustrate many cooperation projects are the result of

bilateral agreements between individual companies and HEIs The focus of these

activities can vary widely depending upon the aims and objectives of each particular

example (as identified in the Wilson Review in the UK3)

In contrast to university-business cooperation in the world of research and innovation

the profile of business-university collaboration in the field of education has been

relatively limited This is unfortunate as it is at this level that knowledge exchange

can often most effectively be secured relevant skills developed and the conditions for

future innovation and economic growth laid This low profile has resulted in a limited

literature on the topic and prompted DG EAC to commission this study amongst

others The study had three specific objectives

1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration

2014 6

To analyse existing and identify emerging types of university-business

cooperation in Europe and their drivers

To identify demonstrate and assess tangible and intangible effects of

university-business-cooperation

To develop an assessment methodology and relevant qualitative and

quantitative indicators for measuring the outcomes and impact of university-

business cooperation and in particular of Knowledge Alliances

The study has analysed the following ten cases

Educckate multinational - led by the University College London in

partnership with other EU Universities and their local companies to

stimulate entrepreneurship in cultural and creative industries

EUEN multinational ndash led by Coventry University with transnational

academic and industry partners to promote entrepreneurial education

KnowFact multinational ndash led by the University of Patras with industry

partners from other EU countries to develop the Teaching Factory paradigm

in manufacturing education

AppCampus Finland ndash an innovative initiative between Aalto University and

NokiaMicrosoft involving app Development National

Qatar Carbonates UK ndash a collaborative arrangement between Qatar

PetroleumShell and Imperial College London promoting post-graduate

education International

Newcastle University Subsea UK ndash cooperation between Newcastle

University and regional businesses developing sector relevant skills

Regional

Masters in Banking Valencia Spain ndash collaboration between Valencia

University and regional banking bodies developing sector relevant skills

Regional

HP Bulgaria ndash Cooperation between Hewlett Packard and various

Universities whereby HP trained university lecturers in specialised

technologies and created the project curriculum National

University of Merseburg Germany ndash regional businesses endow professorial

chairs at University of Merseburg to ensure industry relevant education

AKA Prague ndash professional qualification (Certificate of Communication

Agencies) developed by industry and delivered by University of Economics

Prague

Types and drivers of cooperation

The first decade of the 21st Century has seen an increasing emphasis on the role of

universities in explicitly contributing to social and economic development This can be

attributed to a number of factors including

2014 7

The emergence of global lsquogrand challengesrsquo (eg climate change ageing

terrorism etc) which cannot be solved by government or business alone

but requires a multi-disciplinary and collaborative approach

Increased marketisation of higher education leading to greater competition

between universities and an emphasis on the lsquostudent experiencersquo and

student outcomes (including entrepreneurship)

The global economic crisis which has ushered in an era of austerity in public

finances in many countries

The recognition by many firms that working with Universities provides an

advantage for their future recruitment pipelines and ancillary profile

benefits

This has resulted in a range of policies designed to encourage universities to build

stronger links with business suggesting that businesses should be (more) involved in

designing curricula (including undergraduate postgraduate as well as CPD and other

training) and that universities should work more closely with industry partners to

promote entrepreneurship mobility (between business and academia) and lifelong

learning

In practice many universities and businesses have recognized the value of

cooperation particularly in terms of how this can strengthen student recruitment for

universities improve the educational experience for students and provide a supply of

more experienced labour for companies Some companies and universities also

recognize the wider benefits that this cooperation can bring in the form of higher

levels of innovation the upskilling and retention of existing staff and the higher profile

that successful collaboration activities can bring

A number of reports have usefully set out different lsquotypesrsquo of cooperation between

businesses and universities including student and academic mobility programmes

curriculum development and delivery lifelong learning and entrepreneurship4

However most cooperation activities examined for this work incorporated more than

one of these cooperation types This suggests that it is the purpose of cooperation

that is more important rather than the type of cooperation undertaken

Each cooperation partner (business university student) has their own reason to

engage in the cooperation activity but a number of common themes can be identified

These are illustrated in Figure ES1 Three of these relate to strengthening levels of

human capital either of students or of existing workers within companies and

universities to promote a stronger labour supply enhance individual employability

and heighten levels of entrepreneurship These were common across all the projects

examined for this study Three are more related to innovation primarily through

collaborative working on company projects as part of the educational experience or

through the co-production of new courses and course material these were visible in

each of the cases studied but to varying degrees Finally the remaining three

4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 6

To analyse existing and identify emerging types of university-business

cooperation in Europe and their drivers

To identify demonstrate and assess tangible and intangible effects of

university-business-cooperation

To develop an assessment methodology and relevant qualitative and

quantitative indicators for measuring the outcomes and impact of university-

business cooperation and in particular of Knowledge Alliances

The study has analysed the following ten cases

Educckate multinational - led by the University College London in

partnership with other EU Universities and their local companies to

stimulate entrepreneurship in cultural and creative industries

EUEN multinational ndash led by Coventry University with transnational

academic and industry partners to promote entrepreneurial education

KnowFact multinational ndash led by the University of Patras with industry

partners from other EU countries to develop the Teaching Factory paradigm

in manufacturing education

AppCampus Finland ndash an innovative initiative between Aalto University and

NokiaMicrosoft involving app Development National

Qatar Carbonates UK ndash a collaborative arrangement between Qatar

PetroleumShell and Imperial College London promoting post-graduate

education International

Newcastle University Subsea UK ndash cooperation between Newcastle

University and regional businesses developing sector relevant skills

Regional

Masters in Banking Valencia Spain ndash collaboration between Valencia

University and regional banking bodies developing sector relevant skills

Regional

HP Bulgaria ndash Cooperation between Hewlett Packard and various

Universities whereby HP trained university lecturers in specialised

technologies and created the project curriculum National

University of Merseburg Germany ndash regional businesses endow professorial

chairs at University of Merseburg to ensure industry relevant education

AKA Prague ndash professional qualification (Certificate of Communication

Agencies) developed by industry and delivered by University of Economics

Prague

Types and drivers of cooperation

The first decade of the 21st Century has seen an increasing emphasis on the role of

universities in explicitly contributing to social and economic development This can be

attributed to a number of factors including

2014 7

The emergence of global lsquogrand challengesrsquo (eg climate change ageing

terrorism etc) which cannot be solved by government or business alone

but requires a multi-disciplinary and collaborative approach

Increased marketisation of higher education leading to greater competition

between universities and an emphasis on the lsquostudent experiencersquo and

student outcomes (including entrepreneurship)

The global economic crisis which has ushered in an era of austerity in public

finances in many countries

The recognition by many firms that working with Universities provides an

advantage for their future recruitment pipelines and ancillary profile

benefits

This has resulted in a range of policies designed to encourage universities to build

stronger links with business suggesting that businesses should be (more) involved in

designing curricula (including undergraduate postgraduate as well as CPD and other

training) and that universities should work more closely with industry partners to

promote entrepreneurship mobility (between business and academia) and lifelong

learning

In practice many universities and businesses have recognized the value of

cooperation particularly in terms of how this can strengthen student recruitment for

universities improve the educational experience for students and provide a supply of

more experienced labour for companies Some companies and universities also

recognize the wider benefits that this cooperation can bring in the form of higher

levels of innovation the upskilling and retention of existing staff and the higher profile

that successful collaboration activities can bring

A number of reports have usefully set out different lsquotypesrsquo of cooperation between

businesses and universities including student and academic mobility programmes

curriculum development and delivery lifelong learning and entrepreneurship4

However most cooperation activities examined for this work incorporated more than

one of these cooperation types This suggests that it is the purpose of cooperation

that is more important rather than the type of cooperation undertaken

Each cooperation partner (business university student) has their own reason to

engage in the cooperation activity but a number of common themes can be identified

These are illustrated in Figure ES1 Three of these relate to strengthening levels of

human capital either of students or of existing workers within companies and

universities to promote a stronger labour supply enhance individual employability

and heighten levels of entrepreneurship These were common across all the projects

examined for this study Three are more related to innovation primarily through

collaborative working on company projects as part of the educational experience or

through the co-production of new courses and course material these were visible in

each of the cases studied but to varying degrees Finally the remaining three

4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 7

The emergence of global lsquogrand challengesrsquo (eg climate change ageing

terrorism etc) which cannot be solved by government or business alone

but requires a multi-disciplinary and collaborative approach

Increased marketisation of higher education leading to greater competition

between universities and an emphasis on the lsquostudent experiencersquo and

student outcomes (including entrepreneurship)

The global economic crisis which has ushered in an era of austerity in public

finances in many countries

The recognition by many firms that working with Universities provides an

advantage for their future recruitment pipelines and ancillary profile

benefits

This has resulted in a range of policies designed to encourage universities to build

stronger links with business suggesting that businesses should be (more) involved in

designing curricula (including undergraduate postgraduate as well as CPD and other

training) and that universities should work more closely with industry partners to

promote entrepreneurship mobility (between business and academia) and lifelong

learning

In practice many universities and businesses have recognized the value of

cooperation particularly in terms of how this can strengthen student recruitment for

universities improve the educational experience for students and provide a supply of

more experienced labour for companies Some companies and universities also

recognize the wider benefits that this cooperation can bring in the form of higher

levels of innovation the upskilling and retention of existing staff and the higher profile

that successful collaboration activities can bring

A number of reports have usefully set out different lsquotypesrsquo of cooperation between

businesses and universities including student and academic mobility programmes

curriculum development and delivery lifelong learning and entrepreneurship4

However most cooperation activities examined for this work incorporated more than

one of these cooperation types This suggests that it is the purpose of cooperation

that is more important rather than the type of cooperation undertaken

Each cooperation partner (business university student) has their own reason to

engage in the cooperation activity but a number of common themes can be identified

These are illustrated in Figure ES1 Three of these relate to strengthening levels of

human capital either of students or of existing workers within companies and

universities to promote a stronger labour supply enhance individual employability

and heighten levels of entrepreneurship These were common across all the projects

examined for this study Three are more related to innovation primarily through

collaborative working on company projects as part of the educational experience or

through the co-production of new courses and course material these were visible in

each of the cases studied but to varying degrees Finally the remaining three

4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 8

reasons relate to developing longer-term contacts and the profile of the industry

companies and universities involved for future potential benefits

Figure ES1 Purposes underlying cooperation activities

Benefits of cooperation

The potential benefits of collaboration activities in the field of education can take many

different forms (Figure ES2) Very often it is the qualitative benefits that cooperation

activities bring (such as the deepening of skill sets) that are valued by participants

rather than quantitative benefits (such as increasing the numbers of graduates)

although this does also feature

Figure ES2 Four forms of outcomes

Some benefits are more immediate and occur directly following the end of the

successful completion of a course of studies (such as the number of graduates) or

during it (such as through engagement of staff and students on joint activities) whilst

others may be longer-term (such as new business starts arising out of stronger

entrepreneurial attitudes) Some benefits are direct (such as the creation of a new

Secure labour supply enhance employability

Knowledge exchangeinnov

ation Build contacts

Entrepreneurial skillsattitudes

Pedagogic development

Raise profilebrand

Staff retentionupskill

ing

Institutional modernisation

Attract students

Quantitative vs

qualitative

Immediate vs longer

term

Tangible vs Intangible

Direct vs Indirect

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 9

course) whilst others are more indirect (such as the value of contacts gained for

potential future research projects) Equally some benefits are very tangible (such as

when a student develops a particular skill) but others are much less so (such as where

student knowledge spills over to workers in a company who work alongside them on a

project) Examples of all these potential benefits were visible throughout the cases

explored for this study

In practice our study illustrates a focus on immediate highly-visible returns by 10

university-business cooperation projects but also emphasises the significance of less

tangible and more indirect returns to participants with different participants placing

differential values on particular outcomes For example in the case of the North East

England sub-sea collaboration businesses were keen to fill an emerging skills gap

upskill staff and develop contacts with the university and other businesses For the

academics it was a means to generate additional student numbers to engage with

the business sector and to explore further research avenues

For students the reason they wished to participate in these collaborative offers in the

field of education was primarily related to the fact they felt it would provide them with

greater skills than a more conventional course that they were more likely to find

subsequent employment or that it would provide them with enhanced entrepreneurial

opportunities

For universities and academics there was a similar emphasis on the enhanced skills

acquired by students the greater employability of these students and the stronger

entrepreneurial outcomes anticipated as highlighted in the EUEN project These

outcomes were expected to feed through into an increase in either the quality or the

quantity of student applications with improvements in student perceptions of course

quality also anticipated Additional outcomes which were cited in some instances

included the opportunity provided for increasing contacts in the business world the

updating of skills through exposure to contemporary businesses practices and the

building of trust-based relationships for future collaborative research

For businesses the primary outcome reported was in terms of an enhanced pipeline of

suitably skilled potential employees together with improvements to the wider labour

force There were also particular outcomes specific to individual cooperation

arrangements such as in the case of AppCampus where one measure of success is the

number of applications developed Businesses also reported that the cooperation

arrangements provided benefits in the form of raising their profile but also helped to

build trust and wider contact networks with academics for future collaborative

research Firms also reported that involvement in live projects did provide the

opportunity for product and process innovations adding economic value to the

company and that the bringing together of students and existing employees served to

raise the knowledge levels of employees as well as students

For other parties such as regional authorities the anticipated (and realised) outcomes

largely revolved around economic benefits such as an enhanced labour supply

increased levels of graduate employment higher levels of new business-starts or

(rarely) higher levels of innovation

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 10

Assessing the benefits of cooperation

Currently there is very little guidance on approaches to assessing the benefits of

university-business cooperation activities in the field of education The existing

literature is strongly inclined to cooperation in the field of research and innovation

What examples can be found tend to stress measures of inputs and student outcomes

The wider literature can however provide valuable insights This incudes highlighting

the value of less tangible benefits such as contact between students company staff

and university staff often termed as productive interactions

Monitoring and assessment has a number of potential benefits it provides a means to

assess whether a project or programme is performing as expected it enables a

project leader to identify what works and what does not it provides a useful

mechanism to feedback lessons to others and if undertaken in advance it can

provide a means for selecting between different potential options In the evidence

available to this project the main approaches to monitoring and assessment have

focused on project performance often with an emphasis on measures of inputs and

outputs However there is also evidence of the use of results-based formats often of

a qualitative nature

The evidence available from the cases explored in this work suggests that monitoring

activities are undertaken at two levels in the field of university-business collaboration

in the field of education

Firstly through the use of standard student-orientated metrics

Secondly through informal metrics based around joint-perceptions of

success

The cases also suggest that there are a range of outcomes of university-business

collaboration activities in the field of education that are not currently being captured

by existing monitoring and measurement arrangements such as the potential

upskilling of existing staff through their involvement in the educational experience or

the raising of the profile of participating institutions However in no case was this felt

to weaken the cooperation arrangements underway Indeed to the contrary the

suggestion was that widening the formal measurement and monitoring process to

include more indicators would incur an additional burden for limited gains It is

essential that assessment methods are not only fit for purpose but are also seen to be

so

Whilst there are many methods that can be used to assess the outcomes of a

particular cooperation activity (whether that is a project or a programme) the

balanced scorecard approach seems particularly relevant The central idea of the

balanced scorecard is that several metrics can be used to evaluate the process in

question A causal map for educational university-industry collaboration initiatives is

given in Figure ES3 Four areas of measurement form the essential components of a

scorecard for university-industry collaboration The people involved the resources

available the activities undertaken and the results achieved

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 11

Figure ES3 Scorecard general structure

Although each participant may have their own scorecard for the cooperation (either

held explicitly or implicitly) a combined scorecard can be drawn up for each co-

operation project in the form of a logical framework This allows assumptions to be

made explicit expected relationships to be identified and tests whether the

cooperation planned can meet its intended purpose(s) A stylized logic chain is set out

in Figure ES4

Figure ES4 Stylised causal map for university-industry collaboration

Alongside the scorecard approach an assessment methodology may also wish to set

out the framework for monitoring and evaluation Where evaluation may relate to the

assessment of project applications or the success of a completed project Here a

simple framework such as the success map set out in Figure ES5 may be

advantageous

People

bullThe qualification fit and motivation of the people taking part in the cooperation

Benefits

bullThe beneficial outputs that are generated as a result of the collaboration

Resources

bullThe amount and quality of the resources dedicated to the collaboration

Activities

bullThe quality and appropriateness of processes and organizational set-up

People

bullInstructors students managers

Resources

bullFunding equipment technology

Activities

bullOrganizational processes and structures

Benefits

bullEducation skills employment

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 12

Figure ES5 Success map for cooperation in the field of education

In developing such a success map due regard should be given to four purposes of

university-business cooperation in the field of education broadly stated

Those that relate to teaching and learning aimed at strengthening the

labour supply and employment

Those that relate to stimulating entrepreneurship through teaching and

learning

Those that relate to knowledge exchange aimed at promoting

productprocess innovation or new pedagogy

Those that relate to raising the profile of an organization

In seeking to assess an actual project the framework can ensure that appropriate

indicators are selected and measurement approaches adopted It is beyond the scope

of this report to suggest a standard set of indicators as this will vary by project

purpose However a range of potential indicators have been identified relating to

each of the above purposes and which might be used to monitor inputs activities

outputs and outcomes Examples of potential indicators are illustrated in Figure ES5

In selecting indicators a number of principles should be observed including those of

parsimony (to seek to limit the data burden) and proportionality (data collected should

not incur costs disproportionate to the benefits received) The collection of data does

incur costs and for an assessment process to operate successfully there must be

agreement of all parties as to the role of particular indicators and the value of their

measurement

Interaction opportunities

Access to resources

Motivated educators

High-quality students

Relevant course delivery

Relevant course design

New ideas developed

Skilled amp trained labour

New pedagogies

Student applicant numbersquality

Non-academic organisations involved

Time inputs

Finance inputs

Newimproved courses

Number of graduates

Student attainment

Entrepreneurial attitudes

New products processes developed

Joint objective setting

Internshipsplacements

Co-delivery and assessment

Interaction intensity

Graduate employment

New business starts

Employer satisfaction

Organisational ranking in lsquoleaguersquo tables

Successful alliance

Human capital

Entrepreneurship

New ideas

Profile raising

Input Activity Output Outcome

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 13

In the selection of indicators full consideration should also be given to the role of

qualitative measures of success and the potential means of capturing these This

might be through survey techniques but can also be captured by structured review

meetings with inputs from the various parties The value of self-assessment

techniques should also be considered Quantitative measures should not be over-

looked and where these can be part of wider measurement processes should be

welcomed

Implications for policy and practice

This study has proposed a flexible approach to measuring the impact of university-

business cooperation activities using a balanced scorecard approach allied with a

success-map for cooperation activities This recognizes the multiple purposes that

underlie cooperation activities and acknowledges that a one-size-fits-all approach is

unlikely to prove fit for purpose What are the benefits and implications of the

approach proposed to policy-makers and to the collaboration partners themselves

For policy-makers the tool can be used in a variety of manners

In the first instance it forms a framework for the ex-ante development of

the proposed programme By identifying the resources that are available to

the programme or might be available and the desired objectives

programme officials can identify the types of activity that might lead to the

desired results Through a process of iteration the scorecard can be refined

further This can then provide a baseline for a call for applications

The scorecard approach might then be used for the assessment or

appraisal of applications Assessors can check the applications against the

programme scorecard to identify the extent to which each applicant

contributes to the desired approach and objectives In a dynamic approach

the programme scorecard can be updated through the inclusion of novel

and innovative approaches proposed by applicants which were not foreseen

during the development of the programme A blank scorecard template

might also be completed by assessors as a means of summarising the

information contained in applications in a common format The approach

could highlight missed opportunities for collaborative activities or

resourcepeople inputs which can form the basis of negotiation with

programme applicants

Following a successful application a project scorecard can then form the

basis for the review of project progress This can examine the activities

undertaken resources expended and people involved against what was

initially planned together with the benefits realized compared to those that

were expected The review meeting can then explore the reasons for any

differences examining cases where inputs are greater than anticipated

perhaps owing to the cooperation going better than planned or where

results or inputs are less than originally proposed and whether remedial

action needs to be taken

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

2014 14

The scorecard approach can also deliver significant benefits to the collaboration

partners themselves

As a starting point it can form the basis for robust project design The

scorecard can be used as a mechanism for each partner to set out what

they wish to achieve from the cooperation activity and what they are able

to contribute Each partner can then identify what they expect of the other

partners Using the scorecard approach can also assist in identifying where

potential opportunities might have been overlooked in the original project

design and where small changes might reap large rewards This can then

form the basis of the collaboration agreement between parties

Project partners can then use their own stakeholder scorecard as a means

for internal review and monitoring of the collaboration activity This

provides a useful mechanism for identifying whether desired outcomes are

being achieved and the extent to which anticipated inputs and activities are

occurring As a dynamic process the scorecard can be updated as

activities are added or amended or other inputs change

The partnership as a whole can also use the scorecard as the basis for

ongoing project reviews This can consider whether the original

assumptions remain valid and whether some activities are more appropriate

than others in achieving the desired objectives Crucial to the review

process is to ensure that all partners are realizing the benefits that they

sought initially and if not how this might be rectified The scorecard

approach provides a means of structuring this review process

In order to promote the use of the scorecard approach the Commission may wish to

provide a guide to its use in the context of university-business cooperation activities

This would offer the opportunity to develop worked examples tailored to different

cooperation purposes Such a guide could also incorporate advice on practical

techniques for measuring the outcomes of cooperation activities In the first instance

this could form part of a toolkit available to assist those parties interested in applying

for Knowledge Alliance projects

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

HOW TO OBTAIN EU PUBLICATIONS

Free publications

bull one copy

via EU Bookshop (httpbookshopeuropaeu)

bull more than one copy or postersmaps

from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)

from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)

by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or

calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()

() The information given is free as are most calls (though some operators phone boxes

or hotels may charge you)

Priced publications

bull via EU Bookshop (httpbookshopeuropaeu)

Priced subscriptions

bull via one of the sales agents of the Publications Office of the European Union

(httppublicationseuropaeuothersagentsindex_enhtm)

NC-0

1-1

4-2

99-E

N-N

doi10

doi10