ATLANT- KIS Final Evaluation Report

By:

Final Evaluation ReportMay, 2012

ATLANT KIS Project Evaluation

ContentsSECTION ONE1.- Introduction

1.1.- Structure of the Evaluation Report

1.2.- Evaluation objectives

1.3.- Evaluation constraints

2.- Contextualization

2.1.- Transnational Territorial Co-operation Programme - Atlantic Area: Interreg IV B

Seminario Final - 27 de Marzo de 2012

Final Report – May 2012Página 2 de 134

2.1.- Transnational Territorial Co-operation Programme - Atlantic Area: Interreg IV B

2.2.- ATLANT KIS Project: definition; scope; objectives and actions;

target public and organisation and management structure.

3.- Methodological approach

3.1.- Types of Evaluation

3.2.- Evaluation criteria and key questions

3.3.- Gathering information tools

3.4.- Timeline and field research


Contents (II)

SECTION TWO4.- Evaluation o the ATLANT KIS Project’s Implementation

4.1.- Project's envisaged activities physical execution analysis

4.2.- Project’s management and co-ordination processes analysis



4.2.- Project’s management and co-ordination processes analysis

4.3.- Achievement analysis of the below criteria

o 4.3.1.- Consistency and relevance

o 4.3.2.- Effectiveness

5.- Evaluation on the results achieved by ATLANT KIS project

5.1.- Achievement analysis of the below criteria

o 5.1.1.- Impact

o 5.1.2.- Satisfaction /Expectations

o 4.3.3.- Efficiency

o 4.3.4.- Comunication and Visibility

o 5.1.3.- Sustainability


Contents (III)

5.2.- The specific case of the region of Navarra

SECTION THREE6.- Conclusions related to the criteria and issues raised in the evaluation.



6.- Conclusions related to the criteria and issues raised in the evaluation.

7.- Answers to the Evaluation key questions

8.- Recommendations

9.- Appendix


Section one



Section one


1. Introduction




1. Introduction1.1.- Structure of the Evaluation Report



Report


With the purpose of structuring the available information in a selective manner, this report is divided

into three sections, detailed below:

� Section one: To bring into context, the Co-operation Programme - Atlantic Area: Interreg IV is

previously analysed, more specifically, its geographical scope, description and development phases

and the areas of planning and management. Then, the ATLANT KIS project is presented. Apart from



and the areas of planning and management. Then, the ATLANT KIS project is presented. Apart from

that, this section explains the methodological approach used for the project evaluation as well as the

job’s constrains.

�Section two: This section contains, on the one hand, an analysis of the processes and actions

carried out to implement the ATLANT KIS project -establishing the grade of physical execution of the

activities, and a management and coordination process analysis- and a study on the fulfilment criteria

through the execution phase of the project. On the second hand, this section offers an assessment of

the results reached and effects caused by the project in accordance with the objectives and

expectations set.


� Section three: This section compiles the main conclusions reached after analysing each

evaluation criteria (in both implementation and results areas), which make it possible to answer

the two key questions posed in the evaluation.

Finally, this part includes a series of recommendations to be considered by the involved



partners, that may give cause for reflection and serve as a guide for future projects.


1. Introduction1.2.- Evaluation objectives



1.2.- Evaluation objectives


�Overall Objectives:To evaluate the implementation of the activities and actions carried out from

June 2009 to May 2012 on the ATLANT KIS project and give an overview of the



June 2009 to May 2012 on the ATLANT KIS project and give an overview of the

main results achieved and the effects brought about for both internal and

external beneficiaries.


�Specific Objectives:

• Monitor achievement of goals and objectives

• Describe the life cycle of the project throughout the reporting period

• Reflect on management ’s execution performance



• Reflect on management ’s execution performance

• Provide useful information orientated towards decision-making for

designing and executing of new projects and/or future actions.


1. Introduction1.3.- Evaluation processconstrains



constrains


� The non-existence of a previous evaluation -Conceptual Evaluation or Design Evaluation- limits the

possibility to study the actual consistency and pertinence of the ATLANT KIS project, that is, its internal logic.

This type of study is intended to analyse the excellence of the diagnosis executed prior to the initiation of the

intervention, verify the existence of clearly defined and measurable objectives, analyse the connexion with the

problems observed and finally, to examine the logic of the type of intervention designed, from both programme’s

internal perspective and in comparison to other policies’ and programmes’ (benchmarking).



internal perspective and in comparison to other policies’ and programmes’ (benchmarking).

� The evaluation process was assisted by the participation and cooperation of KIS suppliers from all the

regions involved, with the exception of Galicia. A list containing KIS suppliers having participated in any

action promoted by the partners in their territories was requested with the purpose of knowing their

opinion, expectations, motivation and grade of satisfaction on their participation on the project and on its

resultant tools and products. In the cases in which the list was not provided, the suppliers’ details were obtained

among those suppliers registered in the KIS platform. Nevertheless, in the case of Galicia, no supplier from the

region was registered in the platform. Therefore, they were unable to cooperate with the evaluation.


� Systematicity and information gathering: No unique data. As a result of the analysis carried out on

information gathering for the evaluation, the evaluators consider that, despite the project coordinator’s efforts

(Iniciativas Innovadoras) to elaborate a homogeneous procedure of data collection for all entities, this does not to

cover all aspects of the global monitoring of the intervention (actions and participants), as it was noticed

that neither all organisations collected the same information, nor the same fields were systematised; the manner



that neither all organisations collected the same information, nor the same fields were systematised; the manner

in which the information was processed and the level of dissemination, also varied from one to another.

That is, the mechanisms started do not guarantee, either homogenously or systematically, an appropriate

monitoring of the physical execution and the impact of the project.

�Evaluation process tight schedule >> This prevented the evaluation team from gathering all associated

partners’ opinions, with the exception of those located in the region of Navarre.


2. Contextualization




1. Introduction2.1.- Transnational Territorial Co-

operation Programme - Atlantic

Area: Interreg



Area: Interreg


Mean goal >> the attainment of significant and tangible advances in transnational cooperation with aim of furthering the

cohesive, sustainable and balanced territorial development of the Atlantic Area (with the exception of the regions of the

Azores and Madeira and the Canary Islands, the Atlantic regions included in the territory are Spain, France, Ireland,

Portugal and United Kingdom) and its maritime heritage.

The programme is financed through ERDF for the period 2007-2013.



The programme is financed through ERDF for the period 2007-2013.

The "Atlantic Area 2007-2013" programme is the fourth transnational cooperation programme in the area and seeks to:

� enhance the Atlantic maritime heritage;

� valorise the maritime resources of the Atlantic area;

� contribute to the emergence of new economic activity clusters;

� promote accessibility and logistic conditions;

� contribute to the balanced and sustainable development of the Atlantic area.

Differences in comparison to previous programmes >> It focuses on concrete achievements, the exchange of

experience as well as the transfer of know how and cross fertilisation between projects that address similar issues.

Whereas, the previous programmes were limited to studies or exchange of information.


The Operational Programme "Atlantic Area 2007-2013" is structured along the following priorities:

PRIORITY OBJECTIVES

1. Promote transnational

entrepreneurial and

innovation networks

• the development of knowledge transfers between companies and research centres;

• the enhancement of competitiveness and innovation capacities in the maritime economy of the Atlantic

area;

• the stimulation of economic conversion and diversification by promoting regional potential.

2. Protect, secure and

enhance the marine and

coastal environment

• the improvement of maritime safety;

• the sustainable management and protection of the resources of marine spaces;

• the exploitation of the renewable energy potential of the marine and coastal environment of the Atlantic



coastal environment

sustainably

• the exploitation of the renewable energy potential of the marine and coastal environment of the Atlantic

area;

• the protection and promotion of natural spaces, water resources and coastal zones.

3. Improve accessibility

and internal links

• the promotion of the interoperability and continuity of existing transport networks as well as sea, road, rail

and air intermodality;

• the promotion of short sea shipping (SSS) and cooperation between ports.

4. Promote transnational

synergies in sustainable

urban and regional

development

• the pooling of resources and skills in the field of sustainable urban and rural development in the Atlantic

area;

• the increase of the influence of cities and regions and their attractiveness trough networking;

• the conservation and promotion of the Atlantic cultural heritage.

5. Technical assistance • Financial support is available for administration, monitoring, evaluation and control of the programme.


1. Introduction2.2.- ATLANT KIS Project




ATLANTIC-KIS aims to provide an answer to the first priority of the Operational Programme (OP): : Promote

transnational entrepreneurial and innovation networks; contributing thus to the “Cohesive development of the

Knowledge Economy” challenge. Specifically the project falls under Objective 1.1. leading to the

“Development of knowledge transfers between companies and research centres”.



Overall goal>> To enhance the Knowledge and Technology Transfer and innovation processes on SMEs through

the promotion and co-operation of Knowledge Intensive Services (KIS) at the Atlantic Area.

Long term goal >> To contribute to the development of Clusters of KIS at the Atlantic Area, that might help identify

the area as an excellent one in the supply of KIS.

Length of the project>> From 1 June 2009 to May 31 of 2012.


Specific Objectives of the project:

1. To design a methodology for the mapping of regional and transregional demand and supply of KIS, transferable toother regions.

2. To develop and disseminate 7 Audits on KIS demand and supply from involved regions.



3. To design a methodology for the identification and transfer of Best Practices at the regional policy level, enabling the promotion and dynamisation of KIS.

4. To identify, exchange and disseminate to other EU regions Best Practices resulting from successful regional experiences on the field of KIS promotion and dynamisation.

5. To transfer the Best Practices identified to the regional policies of involved regions, and to experiment newmodels and approaches based upon the latter, aimed at the promotion and dynamisation of KIS.

6. To create stable communication links among KIS of involved regions, to foster their networking and cooperationby the development of a collaborative platform, open to all KIS from the Atlantic Area.


Geographical Scope >> The project involves 8 partners from 5 different countries (7 European

regions) constituted as a Consortium:

COUNTRY INSTITUTION REGION

SPAIN

Dirección General de Empresa e Innovación (Gobierno de Navarra)

Navarra



SPAINConsellería de Innovación e Industria (Xunta de Galicia)

Galicia

FRANCE Bretagne Développement Innovation Bretaña

PORTUGAL Agência de Innovaçao

UNITED KINGDOM Devon and Cornwall Business Council Devon y Cornualles

IRELAND

South and Eastern Regional Assembly Southern and Eastern

WestbicBorder, Midland and WesternBorders, Midlands and West Regional

Assembly


Work Plan (matrix of the project)

Activities Partner leader

of the activity

Actions Follow-up indicators

1. Preparation Activities

1. Development of the project idea, taking into consideration the Programme.

2. Identification and contact of relevant partners .

3. Design and agreement of the proposal, following the Applicant's Handbook.

- 1 Project Proposal submitted.- Consortium of 8 partners agreed.- 8 Co-financing declarations agreed.



Dirección General de Empresa e Innovación (Gobierno de Navarra)

following the Applicant's Handbook.

2. Management and Co-ordination

1. Design of management and co-ordination system and tools

2. Project Co-ordination3. Quality Management

1 Partnership agreement signed.1 Steering Committee1 Technical Committee constituted.1 Project Implementation Guide.1 Intranet.6 Consortium Meetings.4 Progress Reports and 1 Final Report.1 final external Evaluation Report.

3. Audit on KIS demand and supply on involved regions

1. Definition of Methodology for Audit2. Development of Audit3. Dissemination of Audit

1 Methodology agreed on the definition of KIS, the scope of the study, and the outreach tools.7 regional Mappings on KIS demand and supply.More than 500 KIS contacted.

1 Audit Report on 4 languages.

7 Dissemination Workshops: 300 part.



Work Plan (matrix of the Project) (II)

Activities Partner leader of the

activity


4.

Benchmarking

Study on Best

Practices for

Bretagne

Développement

Innovation

1. Good Practices Methodology

definition

2. Identification and analysis of Good

Practices

1 Methodology Guide for identification

of Good Practices and Transfer check.

60 Good Practices identified.

1 Exchange Workshop: 30 part.



ATLANT KIS Project EvaluationPractices for

KIS promotion

Practices

3. Exchange of Best Practices and

development of new models

4. Selection of Good Practices to

transfer

5. Dissemination of Best Practices and

Lessons Learnt on experimentation

1 Exchange Workshop: 30 part.

1 Guide on 4 languages.

7 Action Plans on GPs transfer.

7 Dissemination Workshops: 300 part.

5. Transfer of

Best Practises

and new

models on the

promotion of

KIS

Consellería de

Innovación e Industria

(Xunta de Galicia)

1. Transfer framework definition

2. Transfer and new models

implementation

7 Transfer Implementation Plans and

Evaluation Reports.

Good Practices effectively transferred:

20.

Stakeholders involved: 500.

7 Seminars: 300 part.

Nº of new clusters and networks : 3.

Nº of new links at transnational level: 20.


Work Plan (matrix of the Project) (III)

Activities Partner

leader of the

activity


6. Atlantic

Area KIS

Platform

South and

Eastern

Regional

1. Platform services definition and

development.

2. Platform launch and maintenance

1 Virtual Platform in 4 languages.

Visits to Platform: 10,000.

Nº of registered users: 500.



Platform Regional

Assembly

2. Platform launch and maintenance

3. Transnational KIS Market Place event

4. Platform Sustainability study

Nº of registered users: 500.

Nº of enquiries received: 100.

1 KIS Market Place: 150 part.

Nº of networks: 25.

Sustainability Scheme and Action Plan: 1

7. Promotion,

Dissemination

and

Exploitation

Activities

Agência de

Innovaçao

1. Communication Strategy and Plan

2. Development of communication tools

3. Implementation and monitoring of

Communication Plan

4. Final Dissemination Seminar

1 project website : 500 visits/month.

4 E-Newsletter: 1,000 recipients.

1 Project Brochure: 1,200 copies.

1 Final Seminar: 150 part.

Articles published: 20.

Presentation at other EU fora: 12.

10 Press Conferences: 100 media reached.


Target audience

DIRECT BENEFICIARIES INDIRECT BENEFICIARIES



� Partners involved in the project

�KIS suppliers

� Associated partners involved in the project

�Knowledge and technology sources

(universities, technological centers, higher

education bodies…)

� KIS

demanders

From public sector

(PA)

From private sectors

(SMEs, Chambers of

Commerce, business

associations)


Bodies for management and coordination of the Project

Management>> Lead partner

of the project = DG

Enterprise, Department of

Innovation (Government

Navarre)

Project coordinator>>

external technical assistance

(Iniciativas Innovadoras, S.L.)



Navarre)

STEERING GROUP (SG) TECHNICAL COMMITTEE (TC)

� Composed by 1 representative from each partner .

� Tasks: to give strategic advice and guarantee the

adequacy to the general objectives.

� 6 meetings in conjunction with the TC. Managing

ERDF, Environmental and Equal Opportunities Authorities

from host regions were invited to the SG.

� Composed by all partners.

Tasks: to agree on the contents, work plan and

monitoring of the programmed activities; validation of

outputs; assurance of the involvement of all partners in

shared tasks, accomplishment of the objectives and goals

and conflict resolution among partners.

� 6 meetings.


3. Methodological approach




1. Introduction3.1.- Methodological Approach of the evaluation: types of evaluation



the evaluation: types of evaluation


2 different

Analysis Approach

PROCESS

(Tasks and Actions taken to

achieve results)=

EVALUATION OF

IMPLEMENTATION



2 different

analysis

approachesRESULTS

(outputs and outcomes) = EVALUATION OF

RESULTS


Methodological Approach of the evaluation

Types of Evaluation

• Evaluator-based:>> external evaluation.

• Content-based:>> evaluation of implementation and outcome evaluation.



• Based on the time perspective:>> intermediate evaluation as it is conducted previous to the

end date of the project: 31 May 2012 and ex post evaluation.

• Purpose-based:>> formative evaluation –it is intended to be a tool which enables the

evaluator to gather useful feedback to improve future actions (recommendations)- and

summative evaluation -gives an overview of the tasks carried out while starting-up and

implementing the project by the various agents involved, with a view to render a final and

conclusive judgment on the results –products- achieved-.


Some clarifications on the methodology approached

�The coincidence in time of submission of the project’s intermediate-evaluation report and the

project’s scheduled finish date (May 2012), makes it truly difficult to asses the impact and



effects reached, as a longer period of time would be necessary to determine if the changes and

effects anticipated to result from the implementation of the intervention have made satisfactory

progress.


1. Introduction3.2.- Key Questions that this

EVALUATION aims to answer



EVALUATION aims to answer


Concerning evaluation of implementation

Has the project been developed in accordance with the management, execution and

accessibility terms foreseen?

Concerning evaluation of results



Concerning evaluation of results

Has the project produced a desired effect/changes on the project’s beneficiaries?

What is the immediate impact?

The strategy-formulation tool used to carry out the evaluation is the EVALUATION

MATRIX. The evaluation criteria have been based on the following key issues:


CONSISTENCY: A measure which verifies appropriate connection between the

project’s objectives and the strategic tools established.

RELEVANCE: The extent to which the project's objectives and actions are pertinent

in relation to the framework in which they are developed.



in relation to the framework in which they are developed.

EFFECTIVENESS: The extent to which the activities and indicators are fulfilled when

analysing management and internal coordination processes and with no regard to

inputs.

EFFICIENCY: A measure of how economically resources/inputs (time) are converted

to results.


TRANSFERABILITY/VISIBILITY: The extent to which the project has been visible

in those areas involved. Transferability assesses performance of communication

actions for dissemination in order to determine if they have succeeded in

transferring objectives, activities, good practices and results.



transferring objectives, activities, good practices and results.

IMPACT: Term to describe the effects of the project. Impacts are the changes

produced, which can be either foreseen, as the achievement of overall

objectives, or unforeseen.


SATISFACTION: The acceptance level of all partners involved as well as both

direct and indirect beneficiaries with regard to the project. It addresses various

issues: drawing up and implementation processes and outcomes achieved.



SUSTAINABILITY: The extent to which the project's positive impacts can be

expected to last after its termination

Additionally, the evaluation includes an assessment on management and

monitoring procedures in the Atlankis Project.


1. Introduction3.3.-Gathering Information Tools



3.3.-


Objective >> Encouraging participation among partners and beneficiaries of

the implemented actions, in order to gather information on their

participation, experiences, assessment and level of satisfaction upon outcome

being achieved.



Partners’ questionnaireAssociated partners’

questionnaire

KIS suppliers’

questionnaire


1. Introduction3.4.-Timeline(evaluation phases

& work shedule)



& work shedule)


• Phase I : Analysis of Project Information Requirements

Primary Sources Analysis Secondary Sources Analysis

• Project intranet •Council Regulation (EC) No 1083/2006 of 11 July 2006

•Atlant KIS Website •Transnational Cooperation Operational Programme

"Atlantic Area 2007-2013"



"Atlantic Area 2007-2013"

• KIS platform (www.kis4smes.com) • Spain’s National Strategic Reference Framework 2007-

2013

• Project candidacy form • A variety of studies and reports on KIS:

•COTEC 2011 report

• Technology Knowledge Intensive Based Services

(TKIBS). A competitiveness tool for businesses and

regions

• “El uso de las PYMEs de servicios intensivos en

conocimiento. Factores relacionados e implicaciones

de política “.

• Reports to JTS (Joint Technical Secretariat)

• Balanced Scorecard

•Meeting Minutes

• Outputs: Regional Diagnosis of Supply and Demand; BPs

Identification Report; Several guides and PPT among others.


• Phase II: Matrix evaluation elaboration phase

• Identification of both criteria and key questions in accordance with the envisaged time to

perform the evaluation.-meeting with Iniciativas Innovadoras (lead partner-technical

assistance) 22nd February 2012-.

• Phase III: Planning of applicable tools for data collecting and field researching

• Quantitative techniques



• Quantitative techniques

Type Sample Participation

KIS suppliers’ questionnaire 120 32

Partners’ questionnaire 8 8

Associated partners’ questionnaire 2 2


• Phase IV: Preparing Outputs

• Exploitation and analysis of the information gathered through the surveys.

• Drawing up and presentation of the evaluation outputs:

• A report on preliminary conclusions.



• A performance indicator report (performance-based and outcome-based).

• A final evaluation report.

• A database including compiled information from the questionnaires.


TIMELINE OF THE EVALUATIONPROCESS

2012

FEBRUARY MARCH APRIL MAY

ACTIONS TAKEN (By week) W4 W5 W1 W2 W3 W4 W1 W2 W3 W4 W1 W2 W3 W4

PHASE I. Designing and validation

Workshop (Discussion on the different approaches

and criteria that will lead the methodology and

more specifically both the evaluation matrix and

the validation of the evaluation work plan).

Sources analysis


Sources analysis

Tool designing for data collection

PHASE II. Execution: field research

Surveying process

Phone and mailing reminder

PHASE III. Evaluation ending

Findings analysis

Preliminary conclusions’ presentation 22nd

March

Preliminary conclusions ‘ validation by partners27th

March

Drawing up and rendering of evaluation outputs

Final Evaluation Report ‘s presentation2nd

May

Draft validation by lead partner

Drawing up and presentation of the final evaluation

report

28th May


Section two

Seminario Final - 27 de Marzo de 2012Página 46 de 134

Section two


4. Evaluation of the ATLANT KIS Project’s

Implementation


Implementation


4.1. Project's envisaged activities

physical execution analysis


physical execution analysis

Note: The data shown in this document were analysed

and assessed in April 2012, thus the outcome obtained



and assessed in April 2012, thus the outcome obtained

can be considered as partial and not definitive, as

project completion is programmed for May 2012, when

all planned activities are expected to be completed.


ACTIVITIES ACTIONS HIGH AVERAGE LOW

1. Preparation Activities

1. Development of the project idea X 2. Partnership constitution X 3. Design and agreement of the proposal X

2. Management and Co-ordination

1. Design of management and co-ordination system and tools X 2. Project Co-ordination X 3. Quality Management X

Degree of fulfillment of the project work plan (I)


3. Quality Management X 4. Fulfillment of contractual obligations with Managing Authority X

3. Audit on KIS demand and

supply on involved regions

1. Definición de la metodología del diagnóstico X 2. Development of Audit X 3. Dissemination of Audit X

4. Benchmarking Study on Best

Practices for KIS promotion

1. Good Practices Methodology definition X 2. Identification and analysis of Good Practices X 3. Exchange of Best Practices and development of new models X 4. Selection of Good Practices to transfer: X 5. Dissemination of Best Practices and Lessons Learnt on experimentation X


Degree of fulfillment of the project work plan (II)

ACTIVITIES ACTIONS HIGH AVERAGE LOW

5. Transfer of Best Practises and new models on the

promotion of KIS

1. Transfer framework definition X

2. Transfer and new models implementation X

6. Atlantic Area KIS Platform

1. Platform services definition and development X

2. Platform launch and maintenance: X


6. Atlantic Area KIS Platform X

3. Transnational KIS Market Place event X

4. Platform Sustainability study

X

7. Promotion, Dissemination and Exploitation Activities

1. Communication Strategy and Plan X

2. Development of communication tools X

3. Implementation and monitoring of Communication Plan

X

4. Final Dissemination Seminar X


� Most of the partners consider the ATLANT KIS project to be effective. 87%

of them consider the general objective achieved and give an overall score of 7.6 to

� About the objectives achievement…


of them consider the general objective achieved and give an overall score of 7.6 to

the degree of achievement of the specific objectives.

This statement does not coincide with the perspective of the KIS suppliers>> (only

41% of KIS suppliers consider the general objective achieved).


40%

60%

80%

100% 72%

24%

GRADE OF EXECUTION OF THE ATLANT KIS ACTIVITIES (May 2009 -April 2012)

HIGH

MEDIUM

LOW


0%

20%

40%

HIGH MEDIUM LOW

4%LOW

�This appreciation about the effectiveness is reinforced by the high level of physical execution

of the project, given that 81% of the monitoring markers have been fulfilled.

Regarding the degree of execution of the actions, 72 % have been carried out at a high level

of execution, 24% at a medium level and 4 % at a low level.


4.2.- Management and monitoring

process analysis


process analysis


Based on the obtained data, it is possible to state that the Lead Partner has performed a strategic and decisive rol

during the project life cycle.

The partners assess at an excellent level of fulfilment, the tasks relating to the Project Coordination and the

Secretary tasks.


The remaining assessed tasks (project planning, implementation and monitoring) obtain a good level of

fulfilment.

As for the aspects that could be improved, some partners claimed that the Lead partner should have had more

control and resolution over those partners who didn’t fulfil their tasks and deadlines, taking into account that, in

some activities, the organizations had to work in a parallel way, which has caused some delays, non-fulfilments

and scarce outcomes in some actions.

In any case, the great majority of partners answered affirmatively to this question: Would you use again, on a

second stage of the project, the Lead partner services?


It can be said that, as part of the coordination tasks, appropriate tools have been created

(intranet, templates, forms…) for the correct technical and administrative project management. All of

them have been positively assessed by partners. The use of these tools has been increased all

along the project. More specifically, the use of the Intranet as a means of storing files and


exchanging information. This tool was rather underused at the beginning of the project.

Seven Consortium meetings have been adequately held. Partnership is satisfied with both the

number of meetings and the communication mechanisms used among partners (an overall score of 8

out of 10). Each Consortium meeting has been led by a host partner covering the meeting costs .


Date Place Entidad anfitriona

July 2009 Pamplona (Spain) Dirección General de Empresa e Innovación (Gobierno de Navarra)

March 2010 Rennes (France) Bretagne Innovation

October 2010 Bruselas (Belgium) Dirección General de Empresa e Innovación (Gobierno de Navarra) y Fundación Galicia Europa

February

2011

Santiago de Compostela (Spain) Consellería de Innovación e Industria (Xunta de Galicia)

May2011 Cork (Ireland) South and Eastern Regional Assembly (SERA)

October 2011 Exeter (UK) Devon and Cornwall Business Council (DCBC)

March 2012 Oporto (Portugal) Agência de Innovaçao (ADI)


All efforts have been made to make coincide the great majority of meetings (Rennes, Brussels, Santiago, Cork

and Exeter) with the most important workshops (brainstorming and discussion for GP selection; a regional

transfer programme presentation …), and/or with project dissemination events (KIS market place held in Cork

and Atlant-KIS Final Conference in Oporto). All of them have enabled partners to deal with the various issues

raised while becoming the most important coordination and communication tools among partners and, at

the same time, a good opportunity for partners to come together to network and exchange experience with

projects beneficiaries (KIS suppliers, associated partners, and innovation centres among others).

March 2012 Oporto (Portugal) Agência de Innovaçao (ADI)


On the other hand, partners said that business meetings had served them to reach the necessary

agreements for effective progress making.

INTERNAL PROJECT CONSTRAINTS …


INTERNAL PROJECT CONSTRAINTS …

After analysing the answers given by partners to the questionnaire, we conclude that, overall, there

are not significant internal constrains affecting project implementation. If anything, the more

highlighted issues seem to be the fact that the ATLANT KIS is “too ambitious” (2.7 out of 5) and

“partners’ lack of experience in projects of this nature.” (2.6 out of 5).


4.3.- Achievement analysis of the

below criteria


below criteria


4.3.1.- CONSISTENCY and

RELEVANCE analysis


RELEVANCE analysis


Overall, the partners consider the project’s objectives to be clearly defined but perhaps, too

ambitious.

The planned measures in order to achieve the outcomes and objectives are adequate. However, a

greater dedication and effort to some of them has caused unbalanced outcomes.


The schedule has distributed the tasks in a coherent way, although some partners consider the

number of activities to be excessive with regard to the scheduled execution periods.

On the other hand, all partners agreed that the envisaged actions and activities meet the specific

needs of their regions; however, the Lead partner highlights that due to the fact that the contents of

the programme were defined prior to the regional diagnosis, there was little room for adjustment in

accordance with the regional needs expressed in the aforementioned diagnosis.


50%37%

13%

PARTNERS ASSESMENT ON THE GRADE

OF PERTINENCE OF THE ATLANT KIS

PROJECT

MUCHO BASTANTE SUFICIENTE POCO NADA

Following what it was mentioned before and as set

out in the chart, 87% of the partners assessed

the ATLANT KIS project as “pertinent”, in the

sense that it is considered to be valid and useful

for the future.


The factors which encouraged partners to participate in the project were:

� Learn from other regions how to identify and develop pilot projects, through other EU regions GPs

knowledge transfer and exchange, focused on the development and promotion of KIS to encourage

SMEs innovation and growth.

�Gain knowledge on other innovation systems from the regions involved.

� Create regional and transregional networks.


� Compare the various regions’ situation with those having a similar economic and sectorial

structure.

� Enhance communication among different agents: SMEs and KIS.

� Own region knowledge sharing.

� KIS sector analysis and knowledge.


� KIS sector analysis and knowledge.

These expectations are in line with the overall goal and the specific objectives of the project.

However, no aspect related to the overall long-run goal is mentioned: “the development of Clusters of

KIS in the Atlantic Area”, it is for this reason, that the goal set for this project may be considered

overly ambitious.


As to the KIS suppliers’ answers …

The project objectives are clearly understood and identified by all beneficiaries. Once the answers to the

following questions have been analysed: “Overall, could you indicate what you have heard or you know about the

ATLANT KIS Project? What are the project’s objectives?, the below conclusions have been rendered:

• Out of the total KIS suppliers surveyed, 34% answered in line with the overall goal: enhance the Knowledge

and Technology Transfer processes on SMEs through the promotion and co-operation of KIS.


• 30% focused their answers on the specific objectives: They stated that the project aims to: create a virtual

platform for KIS supply and demand matching, disseminate GPs in the KIS area, boost co-operation among

SMEs to develop strategic interests for products and services; help companies’ growth through innovation and

raise awareness of KIS demand and companies’ resources .

• 16% identified the ATLANT KIS project as a European project joining 5 countries of the Atlantic Area.

• Only one out of the all respondents highlighted the overall long-run goal : “development of Clusters of KIS in

the Atlantic Area”.


4.3.2.- EFFECTIVENESS analysis



�Out of all partners, 62% reported that the ATLANT

KIS project is efficient, with 50% gauging that

objectives had been “highly achieved” and the

remaining 12% claiming that the objectives had been

“achieved”.38%

PARTNERS ASSESMENT ON THE GRADE OF

EFFICIENCY OF THE ATLANTKIS PROJECT


�A Cluster, made up of universities and firms from the

medical technology industry in the region of BMW

Assembly, has been created as a result of the

ATLANT KIS project,

�The KIS market place celebrated in Cork along with

the development of the virtual platform

(kis4smes), have given rise to the creation of a new

KIS Cluster at the transnational level.

50%

12%

38%

VERY MUCH MUCH SOME LITTLE NOTHING


� Partners assesment on project’s achievement of overall goal and specificobjectives

1

Partners thinking the ATLANT KIS

overall goal is achieved

YES

ASSESMENT ON THE ATLANT KIS

ACHIEVEMENT OF THE SPECIFIC

OBJECTIVES

HALF EVALUATION (1 TO 10 SCALE)


� Out of all partners, only one reported that

“it is too soon to gauge whether the overall

goal has been achieved...”

7

YES

NO

To develop and disseminate 7 Audits

on KIS demand and supply among

participants involved .

To develop a KIS platform aimed at

matching the supply and demand of

KIS across the Atlantic Area.

8,5

7,8

6,6


� KIS suppliers assessment on project’s achievement of the overall goals

Chart showing percentage of KIS suppliers thinking the ATLANT KIS overall

goal is achieved


41%

12%

47%

YES

NO

NOT KNOWS/ NOT

ANSWERS


� Most of the executant partners consider the ATLANT KIS project to be effective. 87% of them

consider the general objective achieved and give an average score of 7.6 to the degree of

� About the objectives achievement…


achievement of the specific objectives.

This statement does not coincide with the perspective of the KIS suppliers>> (only 41% of KIS

suppliers consider the general objective achieved).


� Added-value of the ATLANT KIS project for…Partners reported that the programme has brought them and KIS demanders, as direct beneficiaries of theproject, the added-value described below:

PARTNERS KIS DEMANDERS

•Gain Knowledge in local KIS supply and demand • KIS platform (kis4smes)


•Gain Knowledge in local KIS supply and demand • KIS platform (kis4smes)

• Transregional networking and experience exchange.

•Gain knowledge in the various types of KIS services

offered in their regions and the benefits of using them.

•Enhance relationship between Policymakers on the

innovation field and Public Agencies.

•Research Vouchers Scheme

•Raise awareness of their KIS specific needs

•Pilot experiences/GPs testing.


� Added-value of the ATLANT KIS project for… The ATLANT KIS added-value for KIS suppliers varies depending on the group surveyed:

KIS SUPPLIERS (main beneficiaries of the project)

According to Partners

•Services promotion through the KIS platform

• KIS market place•Gain knowledge in the most demanded KIS services


• KIS market place

• Pilot tools for sector development.

•Gain knowledge in the most demanded KIS services

• Possibility to be in contact with other suppliers and find new

partners.

According to KIS Suppliers (18% of those surveyed stated that, the ATLANT KIS project has brought NOTHING into their

company so far)

•Raise awareness of the importance of KIS and the

possibilities they offer.

•Advertisement through the KIS platform

• Gain knowledge in their company’s KIS needs

• Possibility to gain new clients and increase revenue

• Network with the various agents and/or potential partners for

global projects development.

• Access to key service providers

• Gain knowledge in the variety and volume of KIS services

available in their region.

• Financial support for SMEs growth


4.3.3.- EFFICIENCY analysis



�Out of all partners, 63% gauged the project as “efficient”.

� Inputs to carry out the various actions were considered to be

“average”. However, partners struggled to justify expenditure.

Valuation of the grade of

efficiency of the project

ATLANT KIS according to

the partners

VERY MUCH MUCH

SOME LITTLE

NOTHING

Grade of the partners' agreement regarding the

following aspects of the efficiency of ATLANT KIS


37%

63%

NOTHING

There have been sufficient resources available

to perform the actions in terms of economy


to perform the actions in terms of material


to perform the actions in terms of human team

My organisation had no problems to

economically justify the expenditure involved

2

2

3

3

6

5

4

3

1

1

1

1

following aspects of the efficiency of ATLANT KIS

VERY SATISFIED SATISFIED AVERAGE DISATISFIED VERY DISATISFIED


� Timeline and deadlines

� It is noted that initial delays forced some partners to ask for up-to-six-month extensions.

� The communication activities were the most delayed ones due to -according to ADI, the partner responsible

for the initiation of the activity- internal cost-cutting measures, which impeded personnel allocation. It is for

this reason that at the last Consortium Meeting held in Oporto on 28th March 2012 it was agreed to outsource

an associated partner (SEBIC) to accomplish this task. SEBIC will also try to follow indicators and fulfil the


an associated partner (SEBIC) to accomplish this task. SEBIC will also try to follow indicators and fulfil the

objectives set for the last stage of the project.

� Principal constraints>> reporting to the STC and Managing Authority was slower than expected

caused by the online certification platform and the European Authorities’ late response.

� In some cases, partners’ performance has been unbalanced. This, has posed problems and delays in some

particular actions that were programmed to start sequentially.


�Project financial management

� The lack of flexibility of the computer application to validate expenditure led to continuous

budget modifications adjusting the expenses to the corresponding budget headings established

in the agreed budget. Consequently, three budget modifications were requested –the last one


has been introduced along the current month, April 2012-. The first to modifications have mainly

enabled performance of activity nº.5, (Transfer of Best Practices).

�STC flexibility to approve changes is positively assessed. However, it should be remarked that

changes were made at a slow pace.

�Difficulties were encountered for project’s financial management co-ordination, they were

caused by the different First Level Controls timeframe set up in each region.


4.3.4.- COMMUNICATION

AND VISIBILITY analysis


AND VISIBILITY analysis


Analysing the answers given by the partners of the ATLANT KIS project…

The activities related to project communication and dissemination carried out

within the various regions have been truly unbalanced, no only in terms of the

number of de actions put into place, but also in terms of coverage and reaching

the target public.


Whereas five partners believe that the actions carried out in their regions were

“sufficient and clear” , three of them think that they were “insufficient” (BMW

Assembly, SERA and ADI).

The great majority of partners consider the grade of coverage of the project to

be “acceptable”(with the exception of one of the partners involved who rated it

“high”), however, they have pointed out that this is due to the goods

produced, rather than the actual promotion and dissemination of the project.


Among all communication tools used in the project, the partners involved have

experienced various visibility levels:

Low rated tools: “Digital newsletter”, “Press conferences” and “Project’s website”


Average rated tools: “Articles in press”, “Project presentation in other EU forums “ and

“Project’s brochure”

Tools having more impact and visibility: “Seminars ”, “Workshops organised in the various

regions participating in the project” and “KIS market place held in Cork”


Conclusions to evaluation of the organisation responsible for project’s communication and

dissemination .

Overall, in reference to the tools developed along the project, they achieved an “acceptable”

level of satisfaction (with the exception of one of the partners who thinks they are

insufficient).


insufficient).

It should be said that, detailed monitoring of the regional and transregional activities carried

out was missing, as well as detailed information for the rest of the partners.

Partners believe that external co-operation from professional organisations is needed. They

also suggest that an organisation working on online promotion and dissemination (e.g.

twitter, facebook) be considered for future interventions, this could be done throughout

consortium, clusters...


Assessing answers from KIS suppliers involved in the ATLANTKIS project…

� More than 50% of the KIS suppliers surveyed stated

that they had been able to join the ATLANT KIS project

because of the dissemination job done and contacts

made by the partners.5%

2%

2%

2%

Half of communication through

which KIS suppliers met ATLANT KIS

projectDirect contact from an

institution

Workshop/regional forum

Project’s website


� However, as set out in the chart on the left, the action

of gaining beneficiaries through the various

communication tools developed has been limited. Only

6% of the respondents were aware of the project

existence through brochures, digital newsletters, articles

in press...

33%

18%16%

11%

11%

Project’s website

Attendance at the KIS

Conference held in Cork

Project’s platform

(www.kis4smes.com)

Project’s brochure

Project’s e-newsletter

Article in press

Other


Assessing answers from KIS suppliers involved in the ATLANTKIS project…

� It should be remarked the important dissemination job done

by partners for KIS platform promotion, this is proved by the

fact that out of all KIS suppliers surveyed, 75% know of the3%

Grade of knowledge of the KIS

platform (www.kis4smes.com)

YES


fact that out of all KIS suppliers surveyed, 75% know of the

platform existence.

75%

22%

3%NO

NOT

KNOWS /

NOT

ANSWERS


5. Evaluation on the results achieved by

ATLANT KIS project


ATLANT KIS project


5.1.- Achievement analysis of the

below criteria


below criteria


5.1.1.- IMPACT analysis



ECONOMIC IMPACT

SOCIAL IMPACT

POLITIC IMPACT

ENVIRONMENTAL IMPACT

37,5%

12,5%

37,5%

12,5%

12,5%

37,5%

25,0%

12,5%

50,0%

25,0%

25,0%

12,5%

12,5%

37,5%

12,5%

25,0%

12,5%

Assesment on the ATLANT KIS impact according to partners

VERY MUCH MUCH SOME LITTLE NOTHING NOT KNOWS/NOT ANSWERS


� According to the partnership, the ATLANT KIS project has made a “high” political impact, while

the social impact is believed to be rather “moderate” (37.5% of the partners rated the impact

“high” and 25% “average”).

�The environmental impact of the project is, overall, the lowest rated, as 62.5% of the partners rated

it “very poor”.

�As shown on the chart above, there are different points of view as to the economic impact of the

project is concerned (half of the partners stated that the project had “very high” or “high” economic

impact, while the other half rated it “average”).


�ATLANT KIS EFFECTS

As the project is not yet finished, it is technically too early to evaluate the ATLANT KIS effects

in terms of impact.

Therefore, the ATLANT KIS benefits must be assessed, in terms of efficiency, as outputs derived

from project participation (gain KIS knowledge and its possibilities, make contacts and exchange


from project participation (gain KIS knowledge and its possibilities, make contacts and exchange

experiences with other SMEs and entities; a chance for promotion in the industry).

However, there are some possible effects that, according to the stakeholders surveyed

(partners, KIS suppliers and associated partners) start to show:


�ATLANT KIS effects for…

PARTNERS

•50 % of the partners said that after the ATLANT KIS project, new opportunities of exchanging ideas havearisen among them:

• Galicia >> Cooperation among clusters while sharing synergy and taking as reference other projects, inwhich Galicia, Bretagne and Ireland have been involved, e.g. AT Clusters.

• Attempts to hire teaching staff and experts from Exeter for BP exchange.• Agreement with Porto Business School to run the course “Innovation Managers”.


• Agreement with Porto Business School to run the course “Innovation Managers”.• BMW Assembly >> Co-operation with DCBC

•75 % of the partners stated that the ATLANT KIS project has made an impact on the creation of publicpolicies and programmes in their regions:

• DCBC >> Redefine their innovation strategy to support the growth of potential KIS companies and otherindustries under the experience of some pilot projects already implemented and transferred.•Navarra >> 4th Technology Plan in Navarra• Galicia >> The project have enabled this region to update the Innovation Managers Network of GaliciaProgramme set up in 2010 (Rede Xiga). A Dirección Xeral (The General Secretariat )is currently going

through a transition period and, consequently, La Axencia de Innovación de Galicia (The Innovation GalicianAgency) will be shortly implemented. One of its first missions will be the implementation of the “Registro deAgentes del Sistema Gallego de Innovación.” (“Agents of the Galician Innovation System Registry”).• ADI >> They expect to adopt some of the identified BPs in any future programmes.


KIS SUPPLIERS

• The ATLANT KIS project has had no effect on the principal beneficiaries of the project in terms of

business growth so far:

• None KIS supplier has received any service request or query via the KIS platform.

�ATLANT KIS effects for…


• None agreement has been signed between KIS demanders and suppliers after their

registration in the platform.

• There is only one supplier who stated that they had been able to contact other supplier as a result

of their participation in the programme.

• Some of the suppliers surveyed agreed that, as a consequence of their participation in the ATLANT

KIS programme:

• Awareness of the importance of these services has increased.

• Their services have been widely promoted.


5.1.2.- SATISFACTION

analysis


analysis


The partners involved are satisfied with the overall performance of the project. Yet, there are important

issues, among the various aspects evaluated, that should be pointed out.

Of the total number of items assessed, the greatest level of satisfaction was found at performance of co-


Of the total number of items assessed, the greatest level of satisfaction was found at performance of co-

ordination, organisation and management tasks, with an overall score of 8.2 out of 10.

Partners are specially satisfied with the distribution of tasks and duties arranged to achieve project’s

success, more specifically with the procedures and methods used for information gathering.


With another high score (7.8), partners expressed their level of satisfaction in reference to

“Outcome and achievements reached after implementation of the actions and activities

scheduled”, and “level of co-operation and experience exchange reached ” (7.3).


“Communication and dissemination strategies” (5.6) is the lowest rated item.

It should be said that there is no consensus among partners in regard to the communication and

dissemination strategies defined for the project as 3 out of 8 of the partners involved rated it

“poor” (scores ranking between 1 and 3 out of 10), 2 of them “good” and the remaining partners

“very good ” or “excellent”.


The final beneficiaries are highly satisfied with the activities promoted throughout the project.

The overall score obtained concerning their satisfaction level is 7.2 with the items being rated -from

highest to lowest- as follows:

-Contactable institution in charge of the project in my region


-Contactable institution in charge of the project in my region

-The objectives and goals pursued by the project meet my business sector’s needs

-Innovative nature of the activities

-Length and timetable of the workshops and seminars run in your region

-Accessibility to the information, material and products processed during the project

-Opportunity to exchange experiences with other companies from my region


5.1.3.- SUSTAINABILITY

analysis


analysis


The great majority of partners (five out of eight) thinks that the Atlankis project is sustainable and has a

“high level “ of continuity.

ATLANT KIS has produced outputs lasting beyond project life such as handbooks and

methodologies, however, the Sustainability Study aimed in activity nº6, whereby continuity of the virtual

KIS platform after project’s completion is to be raised, has not yet been conducted by partners.

As regard to transferability, 50% of the partners believe that the ATLANT KIS project is “highly”


As regard to transferability, 50% of the partners believe that the ATLANT KIS project is “highly”

transferable to other regions.

50%

25%12,5%

12,5%

PARTNERS’ ASSESMENT ON THE GRADE OF

TRANSFERABILITY OF THE ATLANT KIS

VERY MUCH MUCH SOME LITTLE NOTHING


5.2.- The specific case of the region of

Navarra


Navarra

The DG Enterprise and Department of Innovation from Navarra Government was mainly assisted by two entities

acting as associated partners for the development of the ATLANT KIS project in the region: The European Business

and Innovation Centre of Navarra (CEIN) and the Navarra Association of Consulting (ANEC).

� Previous experience: While, CEIN had participated in other programmes assigned to Interreg

(EURIS, Organza, ICT Value…) before the ATLANT KIS project, ANEC had no previous experience.



(EURIS, Organza, ICT Value…) before the ATLANT KIS project, ANEC had no previous experience.

�Project involvement (specific actions): ANEC has been involved in the development, implementation and

dissemination of activity nº 6: KIS transregional co-operation platform (www.kis4smes.com), by advising and

acting as a link between other key entities and beneficiaries, as well as drawing up reports/studies and arranging

events and workshops. More precisely, they have participated in the platform’s launching and maintenance activities

related to act 2 by performing the following actions : They presented the use and operation of the platform at the KIS

Regional Forum held on 8th February 2012 in Navarra Factori (CEIN), while encouraging the attendees to join it.

Promotion and dynamisation actions: ANEC has provided access to the platform through a link on ANEC’s

website as well as on the main business and innovation portals: business portal “Navactiva”; Navarra’s

innovation portal “Navarrainnova”; “Navarra.es”; business press “Nueva Gestión”.

ANEC has participated in the Transregional Market place as well as in the platform sustainability study.

They believe that the objectives related to these actions are accomplished as far as the platform facilitates

communication among SMEs while helping internationalisation and online work and gives KIS suppliers visibility.



communication among SMEs while helping internationalisation and online work and gives KIS suppliers visibility.

Constraint >> attract KIS demanders to the platform

In addition to this, ANEC has supported the activity nº 3 Audit on KIS demand and supply on involved

regions, by delivering two presentations in two different fora:

• 18th May 2010. KIS Sector Navarra Discussion Forum: “Regional Diagnosis of Supply and Demand in the KIS

sector in Navarra”, at CEIN’s premises

•16th February 2011: KIS Sector Discussion Forum: “Transregional Diagnosis of the KIS Sector”, at CEIN’s

premises.


� CEIN: CEIN has participated in activities nº 4 and nº 5, identification and implementation of GPs. More

precisely, they have contributed to the GPs Guide and the implementation and transfer of the three GPs which

were identified by the region of Navarra. The actual implementation status of these GPs is as follows:

1. Business Accelerators (transferred by DCBC): It consists of a course to foster business growth and mentoring of a

limited number or companies (6 companies). Run by the Instituto de empresa, this course offered high quality

contents, speakers and mentoring network . The course comprised two phases:


contents, speakers and mentoring network . The course comprised two phases:

•A training programme: A five-day training programme was held (from 7th to 16th November) from 9 .00 to 18.30.

Number of participants: 12 companies.

• Mentoring: Out of the 12 participants, 6 were involved in this activity. They were selected following companies’

greater growth potential criteria. By means of an individualised meeting, each company and its mentor defined the

mentoring process objectives, the working procedure and the milestones follow-up. This activity is aimed at: review

the business plan; enhance marketing and commercial plans; open new distribution channels and guide companies

throughout their internationalisation process. Mentoring was carried out from December 2011 to March 2012.


2. Research Vouchers Scheme: They aim to promote cooperation between SMEs and knowledge centres to foster

innovation and companies’ development throughout technology transfer. Firms can apply a basic research voucher for

research service assistance to technological experts. The vouchers can cover advice and technical research. (They

provide up to €5,000). Voucher recipients assist companies with processes and procure product analysis to boost

productivity by providing technical knowledge. Firms are obliged to put proposals into practice within three months

from the study completion. A total of 5 firms participated in this scheme. This BP finishes in May 2012.


from the study completion. A total of 5 firms participated in this scheme. This BP finishes in May 2012.

3. IDEAGEN: A structured brainstorming session allows participants to generate entrepreneurial ideas and also network

within the larger group situation. They are led by innovation experts. Firms, technology centres, universities and

entrepreneurs are brought together in a dynamic and interactive online forum. A session is planned for the 24th April

2012 and will focused on new software technologies. Experts from the Public University of Navarra will lead the activity

and, it is expected that entrepreneurs working on new business ideas and willing to set their own businesses attend

the session.


�Opinion of the associated partners from navarra about the added-valueof the ATLANT KIS project for…

ASSOCIATED PARTNERS

•Greater visibility as an entity representing the sector

KIS SUPPLIERS (main beneficiaries of the

project)

• Closer links between SMEs, Technology centres and

Universities.


• Gain knowledge on european projects

• Network with other entities and regions

• Broader knowledge of KIS supply and demand in the

region.

• Broader knowledge of KIS research centres and teams

•Learn from the implemented BP

Identity ideas through three creative challenges:

• What can be done to jointly create an EXPERT

NETWORK of use to companies?

• What can be done to learn more about TCs and

Universities’ know-how and thereby help business

innovation?

• What role can our company play to CONNECT ideas

and entrepreneurs?

• A growing awareness of the importance of the KIS

sector


� Assesment on their participation on the ATLANT KIS project

Both entities CEIN and ANEC rated the overall work carried out by the entity they have cooperated

with with a 7.3 on a scale of 10, being the more valued aspect the “level of involvement of the partner

in the project” (8.5), followed by the “level of communication with the partner” and the “coordinating


activities carried out by the partner” (both items were rated with 8).

The activity planning in which they have been involved is highly rated (7.2 out of 10), being the

more valued aspects: the “activity planning”; “activity designing: development and contents”; “the

adequacy of inputs to goal achievements” and the “time scale and envisaged planning adaptation” (all

items were rated with 8 out of 10).


� Regarding the overall assesment of associated partner on the ATLANT KIS project…

Usefulness of guides and other material produced throughout the project.

Direct contact with the institution in charge of the project in my region.

Opportunity to exchange experiences with other with other knowledge centres …

7

8,5

8

7


The associated partners from Navarra rate the ATLANT KIS project with anoverall score of 7.3 out of 10.

0 2 4 6 8 10

The objectives and goals pursued by the project meet my KIS business sector’s …

Innovative nature of the activities.

Planning and arrangement of the activities in your company has been involved.

Length and throughout timetable of the workshops and seminars run in your region

Accessibility to the information, material and products processed he project.

Project advertisement

6,5

6,5

8

8

7,5

6,5

7


� ATLANT KIS Strengths and areas of improvement according to the associatedpartners from Navarra...

STRENGTHS AREAS OF IMPROVEMENT

• The project has raised awareness of a sector which

was unknown for the great majority of agents and

enterprises in Navarra.

• To enhance innovation and creativity for workshops and seminars.

• It has enhanced estate promotion of the KIS • Delays in some activities.


• It has enhanced estate promotion of the KIS

companies, this sector did not receive public support up

to now.

• Delays in some activities.

• The project is well structured and planned. • Low level of attraction of KIS demanders.

• The dissemination of KIS has made it possible to bring

them to the public and acknowledge that they are

services related to innovation and knowledge.

• To boost dissemination and promotion of the project. The

communication process all along the project has been slow . The

web page has not been developed until the project was well under

way.

• The synergy created among participants (entities and

companies).

• There should have been more meetings for KIS companies fostering

networking and joint projects working, among others, not only in

transatlantic areas but also in the region of Navarra.


� Regarding the impact of the ATLANT KIS in Navarra…

The DG Enterprise and Department of Innovation of Navarra Government, states that the

ATLANT KIS project has made an impact in public policies within the Navarra area, more


ATLANT KIS project has made an impact in public policies within the Navarra area, more

specifically in the development of the 4th Technology Plan of Navarra.


Section Three


Section Three


6.- Conclusions related to the criteria and

issues raised in the evaluation.


issues raised in the evaluation.


PROJECT MANAGEMENT and

COORDINATION


COORDINATION


The assessment team considers the coordinating organization to have made

an important effort in order to develop a homogenized procedure for all the

organizations of information collection (monitoring system).


Nevertheless, this should be revised for future interventions since, despite

fulfilling the minimum information requirements, it does not allow to cover all

the aspects relating to a comprehensive monitoring of the intervention

(actions and participants), as well as not incorporating the gender

perspective.


With regard to the indicators, which are understood as project advance andprogress measures, it can be confirmed that their level of suitability is rather one of a

general type (physical and financial execution).

That is to say, quantitative indicators of fulfilment have been envisaged but no

qualitative indicator has been designed. It can also be said that two different toolshave been observed to asses the degree of fulfilment of indicators for information


have been observed to asses the degree of fulfilment of indicators for informationgathering: the Balanced Scored Card and the indicator suite of the project’sapplication form.

This has given rise to misunderstandings related to the project’s performance and

results, therefore, the measurement and knowledge of the project effects in the variedterritories have been limited.

On the other hand, it does no exist a global criterium which allows partners to

understand the indicators set, this has led to important differences in informationgathering.


EFFECTIVENESS



The assessment team states that when all the data has been collected is the moment in

which the results of the project start to be visible.

Up to the date of handing in this report, the degree of advance and progress regarding


Up to the date of handing in this report, the degree of advance and progress regarding

the physical execution of the measures is “adequate”. In general, the degree of

achievement is acceptable.

However, if we analyse and compare each of the activities in the project, significant

differences in advance and progress have become evident, being even clearer in some of

the regions.


EFFICIENCY


EFFICIENCY


After analysing the aspects in which this criterium is specified, the assessment

team, considers the AtlantKis project to be efficient and well-measured with regard to the

fact that the available resources have been enough to carry out the scheduled activities.

It is not possible to make the same value judgment about the pace of project development


It is not possible to make the same value judgment about the pace of project development

since some delays in the initial stage caused some of the expected products not to be

finished at the time of the final seminar (especially, those related to the activity no 7:

communication and dissemination).

The ineffectiveness of the administrative systems of account submission and accountability

is confirmed as the main reason for the mismatches which have been detected in the

project and the unbalanced performance of the partners.


CONSISTENCY and

RELEVANCE


RELEVANCE


The assessment team confirms that the good coordination of project objectives, scheduled

actions and expected outcome is adequate, to a large extent, for the specific context and needs

of the involved regions, not only from the partners’ perspective but also from the KIS suppliers’.


However, in a higher level of planning there are certain doubts whether the Atlankis

performance strategy will enable the accomplishment of the overall goal in the long run:

“contribute to the development of Clusters of KIS in the Atlantic Area, which will help to ensure

that the area is recognised for its ecellence in the field of KIS”. It seems that, the way this goal is

stated, the key question fitting into this criterium can not be answered: Which visible or

measurable outcome will show that the project’s goal is accomplished?


SATISFACTION and


SATISFACTION and

EXPECTATIONS


The evaluation team states that, the partners along with the beneficiaries and the

associated partners assessed a “high level” of satisfaction as to management,

performance and results achieved is concerned.


It became clear that the goods produced by the project have met their expectations,

although some of them (especially the KIS platform) will require that a strategy be

developed to ensure their future continuance.


COMMUNICATION AND

VISIBILITY


VISIBILITY


From the evaluation team side, we can confirm that there is a lack of an actual

Communication Plan being more effective and suited to the regional and inter-regional

framework and in which partners specify: objectives, expected results, indicators,

activities, reach and/or media foreseen to make the project visible.


activities, reach and/or media foreseen to make the project visible.

Various communication tools have been delineated as it was anticipated. Nevertheless,

we think that they could have been re-defined to ensure accomplishment of the effects

foreseen. Our view is that criteria should have been clearly stated in advance. That is,

“what?” and “for what purpose?”, as well as “use” and “effects and impacts foreseen”, in

both coverage and visibility areas .


IMPACT



ATLANKIS PROJECT RESULTS AND EFFECTSPARTNERS

50 % of the partners involved stated that new opportunities to share experiences have arisen

as a result of the ATLANT KIS project.

75 % of the partners involved stated that the ATLANT KIS project made an impact on the

development of new schemes and policies in their region.

KIS SUPPLIERS


KIS SUPPLIERS

To date, the ATLANT KIS project has not caused any effect on the major beneficiaries of the

project in terms of business development.

It can be observed that , in regard to the impact and effects attributed to the project, they

seem to be more noticeable within the framework of the development of new insights, tools

and methodologies , as well as within experience exchange among partners.


The impact and effects derived from the implementation of the ATLANKIS project would mainly

remain among the institutions (project partners), being less noticeable for the beneficiaries in

each of their territories.

Yet, most long-term effects/impacts, could only be verified several years after the end of the

project. It will be necessary for 2/3 years to have passed to be able to confirm if the success

factors, which are associated with the actions carried out along the project, have evolved as


factors, which are associated with the actions carried out along the project, have evolved as

successfully as it was expected.

It is for this reason that, the benefits attributed to the ATLANTKIS project should be

considered, in terms of efficiency, as the resultant outputs (gaining greater KIS knowledge;

networking and experience exchange opportunities among SMEs and entities; advertising ... ).

At this stage of the project, it is not possible to come to a solid and reliable conclusion with

regard to either the outcome reached in the long term or the sustainability of the results.


SUSTAINABILITY



Having received feedback from partners, beneficiaries and associated

partners, based on the answers obtained from surveys, the evaluation team

can state that:

The lack of inputs and funding represents the weakest point of the Atlankis


The lack of inputs and funding represents the weakest point of the Atlankis

project.

New cooperation formulas as well as new tools and methodologies which

may contribute to innovative solutions beyond the life of the project

(mainly guides and the KIS platform), along with knowledge transferability

to different frameworks and a key sector within the Atlantic area, are the

project’s biggest success factors.


7. Answers to the key questions



Has the project been developed in accordance with the

management, execution and accessibility terms foreseen?

Answer: The objectives and goals set have been largely achieved, however there have been different

Concerning evaluation of implementation


Answer: The objectives and goals set have been largely achieved, however there have been different

levels of involvement and performance among partners. In addition to this, different responses concerning

the grade of execution of the envisaged activities have been observed.

On the other hand, variations in the budget and schedule have forced partners to excessive dedication to

paperwork and setting-up time.

This fact has significantly limited the top performance of certain activities, from the physical and financial

execution perspective.


Concerning outcome evaluation

Has the project or programme produced a desired

effect/changes on the project’s beneficiaries? What is the

immediate impact?


Answer: The impact and effects that the implementation of the Atlant Kis Project has

produced on stakeholders are more noticeable among institutions (partners involved), the

effects have been less noticeable, in the short term, among KIS suppliers (as direct beneficiaries)

in each of their territories. As for KIS demanders, we do not have any information about their

opinion on the level of influence that the project has exercised over them.

CONCLUSIONS OVERVIEW LEVEL OF ACHIEVEMENT OF THE EVALUATION CRITERIA

ANALYSIS

APPROACH

CRITERIA (KEY ISSUES) ACHIEVEMENT

/SCOPE

CONSISTENCY �

RELEVANCE �

SYMBOL MEANING

�Satisfactory level

of achievement

and scope of the

evaluation criteria

�There are some

aspects that



Evaluation of

Implementation

RELEVANCE �

EFFECTIVENESS �

EFFICIENCY �

TRANSFERABILITY/

VISIBILITY

Outcome

Evaluation

IMPACT ��

SATISFACTION �

SUSTAINABILITY �

�aspects that

could be

improved

This criterion

requires

immediate

attention, as

there are various

threats that may

put the

achievement of

the foreseen

objectives, results

and/or effects at

risk


8. Recommendations


8. Recommendations


In general terms:

To carry out a design assessment of any future action in order to guarantee that the objectives

and strategy are COHERENT AND APPROPRIATE in the context(s) in which they are developed.

Likewise, this is going to allow to make the appropriate modifications and reorientations, thus the

achievement of the expected outcomes and their contribution to the objectives and specific

characteristics of the Community Initiative INTERREG IV Atlantic Space will be of a greater scope

and adjustment.


and adjustment.

Cooperation and experience learning among different territories with similar geographical

characteristics are possible and generate good results.

With reference to the information, tracking and assessment systems, to improve observation

tools (results and impact indicators) and data collection techniques (quantitative and

qualitative) in order to get them to be more suitable and appropriate to answer the key

questions, taking into account both our own information availability and others’.

It will be necessary to design them from the beginning of the program and as broken down as

possible, taking into account the principle of equal opportunities.


In specific terms:

To compile a TERMS GLOSSARY at the beginning of the project, with the aim of making the

main and most common concepts which are used in the project easier to understand and

favoring the usage of a common language among partners of different nationalities.

In the project draft process, to spend more time on a detailed definition of objectives,

activities (including feasibility and sustainability) and budget, with the aim of avoiding


activities (including feasibility and sustainability) and budget, with the aim of avoiding

subsequent modification and rescheduling requests which slow down the start-up of the

project.

To improve the administrative processes of management and certification in order to

ease and speed up the report and justification tasks.

Put more effort into the project communication and spreading actions in order to ensure

a wider visibility of the project, from a coverage and impact criterium. (the success in the

design of powerful working tools such as the KIS platform is blurred because of its scarce use

and impact in terms of visibility).


For similar projects to ATLANT KIS, in which sequential working activities are

proposed, it is necessary that mechanisms be established so as to ensure that

deadlines are strictly met and a backlog is not produced for the rest of the activities . In

addition to this, it is necessary that, when monitoring and managing, a certain timeline

flexibility be foreseen for the tasks to be carried out in order to have other alternatives

available as well as to be able to move forward in the event that the project has not yield

any results in its first stages.


any results in its first stages.

Consortium meetings should represent the main partnership meeting point, however, this

should not be the only means of communication and cooperation among partners. The

added-value of co-operational projects should be displayed, by all or some partners, in a

greater number of communication spaces, in which specific issues, related to project

execution, be tackled in view to obtain a shared and global orientation. Special attention

should be focused on time gap between meetings in order to pursue cooperative dynamics.


Once the objective of performing a “Sustainability study” on the ATLANT KIS project is

accomplished- this has determined the preservation of the KIS platform for at least a year after

project’s completion- it is recommended that both the sustainability concept and scope of

the project should be defined. In addition to this, a programme should be drawn up


the project should be defined. In addition to this, a programme should be drawn up

establishing objectives and key challenges and serving as a base for project’s sustainability, it

should also include partners’ commitments.

In addition to this, it will be necessary to establish some action lines and measures, and

implement a management and feasibility model.


9. Appendix


9. Appendix

ATLANT- KIS Final Evaluation Report

Documents

Transcript of ATLANT- KIS Final Evaluation Report