FP7 Energy Mid-Term Evaluation report

FP7 Energy Mid-Term Evaluation Multiple framework services contract with re-opened competition for impact assessments and evaluations TREN/A2/143-2007

Final Report Client: DG ENERGY

ECORYS Nederland BV Robert Williams Koen Rademaekers Sil Boeve Matthew Smith

Rotterdam, July 2010

Final Report - Mid Term Evaluation of NNE FP7 Projects Supported by DG ENERGY

FP7 Energy Mid-Term Evaluation

Table of contents

Executive Summary 0

1 Introduction 4 1.1 Purpose and Status of this Report 4 1.1 Overview of method 4

2 Relevance / Utility / Acceptability 7 2.1.1 Definition 7 2.1.2 Specific Questions and Summary Answers 7

2.2 Intervention Logic / Objective Tree 10 2.2.1 Fundamental Objective of the Programme 11 2.2.2 Specific Objectives 11

2.3 Consultation Views 13 2.3.1 Evolution of calls in FP7 13 2.3.2 Fewer Projects with Less Partners 13 2.3.3 Nature of Projects Requested and Supported 15 2.3.4 Picking Winners and Research vs. Demonstration 16 2.3.5 Balance Between Renewable Energy and Energy Efficiency 17 2.3.6 Quality of Supported Research 18 2.3.7 Increased support for International (non EU) partners 19

3 Efficiency 20 3.1.1 Definition 20 3.1.2 Specific Questions 20

3.2 Participant Survey 22 3.3 Consultation Views 23

3.3.1 Efficiency improvements that have occurred 23 3.3.2 Areas for improvement 24 3.3.3 Project management and external experts 25 3.3.4 Project Applications and Evaluation Procedures 25 3.3.5 Co-ordination with IEE 27

4 Effectiveness 28 4.1.1 Definition 28 4.1.2 Specific Questions 28

4.2 Indicators 29 4.3 Measuring progress 30

4.3.1 Commercial vs. Research Impacts and Benefits 32

4.3.2 Indicators of Project Level Impact – Carbon Savings 33 4.3.3 Programme Level - Strategic Added Value 34

5 Consistency 36 5.1.1 Definition 36 5.1.2 Specific Questions 36

5.2 Project Survey Findings 37 5.3 Project Coordinator and Participant Interviews 37

6 Allocative / Distributional 39 6.1.1 Definition 39 6.1.2 Specific Questions 39

6.2 Programme Data Analysis 41 6.2.1 Total Allocations 41 6.2.2 Allocation by Activity 43 6.2.3 Specific Activity Area 45 6.2.4 Geographic Spread 47 6.2.5 Network Analysis 49 6.2.6 SME Participation 49 6.2.7 New Member State Participation 51 6.2.8 Gender Balance 51

6.3 New vs. Repeat Applicants 52

7 Sustainability 53 7.1.1 Definition 53 7.1.2 Specific Questions 53

7.2 Strategic Energy Technologies (SET) Plan 54 7.2.1 Consultation Views on the SET plan 55 7.2.2 Officer Views of the SET plan 57 7.2.3 Framework Programme Development 58 7.2.4 Longevity of Impacts and Improving Dissemination 59

8 Conclusions and Recommendations 62 8.1 Programme Relevance / Strategic Fit 62 8.2 Information to Applicants 62 8.3 Programme Management 63 8.4 Project and Programme Impact 64 8.5 Programme Future Direction 65

Annexes 66

1 Context 67 1.1.1 High Level Policy Drivers 67 1.1.2 Energy research under FP7 69 1.1.3 Division between DG Energy and DG RTD 70

1.2 Lessons from previous evaluations 71 1.3 Programme History and Evolution 76



2 Strategic Energy Technology (SET) Plan 78 2.1 Overview 78 2.2 Wind 80 2.3 Solar (PV) 82 2.4 Solar (CSP) 82 2.5 Electricity Grids 84 2.6 Carbon Capture Storage (CCS) 85

3 Evolution of calls 90

4 Participant Survey 95 4.1.1 Project Information 95 4.1.2 Relevance and Effectiveness 97 4.1.3 Efficiency 104 4.1.4 Results and Outputs 108 4.1.5 Project Impacts 112 4.1.6 Other comments 119

4.2 Past Beneficiary Survey 119

5 List of Consultees 123

6 Stakeholder Interviews 125 6.1 Evolution of calls in FP7 125

6.1.1 Fewer Projects with Less Partners 125 6.1.2 Large vs. small projects 126 6.1.3 Mix of technologies in approved projects and calls 127 6.1.4 Mix of Technologies and SET Plan Alignment 127

6.2 Quality of Supported Research 130 6.3 Increased support for International (non EU) partners 131 6.4 Measuring progress 131

7 Project Participant Interviews 134 7.1 Relevance / Utility 134

7.1.1 Global Competitiveness 134 7.1.2 Measuring the Impact 135 7.1.3 International Co-Operation 135

7.2 Efficiency 137 7.2.1 The Application Process 137 7.2.2 Project Size 139

7.3 Effectiveness 141 7.3.1 FP7 Objectives 141 7.3.2 R&D benefits Vs. Commercial benefits 143

7.4 Sustainability 144 7.4.1 Future focus for FP7 144 7.4.2 Strategic Future of FP 146 7.4.3 Longevity of Impacts 147 7.4.4 Improving Dissemination 147 7.4.5 SET Plan 149

8 Network Analysis of FP6 and FP7 150 8.1 Energy FP6 151 8.2 Energy FP7 156



Glossary

CCS Carbon Capture and Storage CHP Combined Heat and Power CSP Concentrated Solar Power CT Carbon Trust (UK) DG Directorate General DG ENER Directorate General Energy DG ENTR Directorate General Enterprise and Industry DG ENV Directorate General Environment DG INFSO(C) Directorate General Information Society and Media DG RTD Directorate General Research and Technological Development DG TREN Directorate General Transport & Energy ECAS FP7 Energy project IT system EE Energy Efficiency EERA European Energy Research Alliance EII European Industrial Initiative EIT European Institute of Innovation and Technology EU ETS European Union Emission Trading Scheme EU-12 Bulgaria, Cyprus, Czech Republic, Estonia, Hungary, Latvia, Lithuania,

Malta, Poland, Romania, Slovakia and Slovenia. EU-15 Austria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland,

Italy, Luxembourg, the Netherlands, Portugal, Spain, Sweden and the UK.

EU-27 All 27 members of the European Union (see EU-15+EU-12) FP (5, 6, 7, 8) Framework Programme GHG Greenhouse Gases ICT Information and Communication Technologies IEE Intelligent Energy Europe IPR Intellectual Property Rights JRC Joint Research Centre JTI Joint Technology Initiative KIC Knowledge and Innovation Community KPI(s) Key Performance Indicator(s) KTP Knowledge Transfer Partnership (UK) MDG Millennium Development Goal MS(s) Member State(s) NEF FP7 Energy project IT system NMS(s) New Member State(s) – see EU-12 R+D Research and Development RES Renewable Energy Sources SET Strategic Energy Technology plan SETIS Strategic Energy Technology plan Information System SME(s) Small Medium Enterprise(s) ToR Terms of Reference WP Work Programme

Final Report - FP7 Energy Mid-Term Evaluation 0

Executive Summary

This report presents the mid term evaluation of the non nuclear energy research and demonstration projects supported by DG Energy 1 under the 7 th Framework Programme (FP7).The approach followed has been:

• Programme data an alysis: A review of the available data on applications and approved projects.

• Literature review : To illustrate the policy context and to help update / reconstruct the objectives of the programme.

• A participant survey : Covering all project partners involved in the target group of FP7 projects plus another survey of recipients of FP6 funding who have not yet applied for FP7 funds.

• Consultations: A series of structured interviews with high level stakeholders, DG officers and project coordinators and partners f rom 10 current projects.

The report presents our findings and conclusions against each of the main evaluation questions and then presents and describes a series of recommendations structured via the main stage s of the programme life cycle. The evaluation questions, our findings and recommendations are summarised below. Relevance / Utility / Acceptability “To what extent are the objectives of a public intervention (project, programme or policy) appropriate regarding the needs perceived and the problem t he intervention is meant to solve?” “The extent to which stakeholders accept the policies in general and the particular instruments proposed or employed.”

• Programme understanding among applicants is ok but could be better. • Projects see a weaker link to competitiveness than to enhancing research . • Past applicants are still interested in the FP and do not feel excluded. • The SET roadmaps need to be explicitly linked to FP, this will become easier over

time. • RES appears to get more funding than EE, though other EU programmes improve the

ratio. An important reason is the lack of successful EE market mechanisms . • There is a need to retain funds for smaller and more research focussed projects – DG

RTD do this, but the ENER link to their activities should be retained.

1 To aid simplicity this report refers to DG ENERGY throughout although prior to 2010 it was actually DG Transport and Energy

(TREN).

1 Final Report - FP7 Energy Mid-Term Evaluation

• There is a need to i mprove the clarity on the definition on ‘demonstration’ in the application material.

• The participants regard themselves as among the best in the world. This is hard to prove with the only real proof becoming apparent ex-post. Finding this proof requires increased evaluation.

• The current amount of i nternational (non EU) co-operation is acceptable offering a balance between access to expertise and market vs. leakage and non EU benefits.

• Applicants should be required to estimate the CO2 saving potential from their bids , though guidance is needed, it does not need to be precise and it should not have an increased role in ranking bids .

• The strategic fit of the programme has improved since FP6 and during FP7 – this reflects the larger and fewer projects and SET plan alignment.

• Policy influence is positive but view on importance and impact is mixed. Efficiency “How economically have the various inputs been converted into outputs and results? Were the (expected) effects obtained at a reasonabl e cost?”

• SET alignment has brought benefits of reduced officer workload. • Time from approval to contract still too long, though the admin burden is ‘as

expected’ for the majority. • Previous FP experience doesn’t appear to reduce the time taken to administer

projects, though it does lead to a slight increase in application time. • The retention and recycling of unspent budget at the end of the programme should be

investigated. • Application and evaluation procedures – suggestions:

o Formalise pre submission meeting procedures and applicant interviews . o Off site (remote) evaluations preferred over on site. o A more inclusive end of evaluation meeting.

• Programme management - suggestions: o Project management KPIs . o Increased ease of access to external expertise.

• IT systems, some issues, but improving and outsourcing makes it hard to address. Effectiveness “What effects (impacts) have been obtained by the intervention and, in particular have these effects contributed to the achievement of the objectives of the intervention? ”

• CO2 savings are suggested as the best indicators despite some participant resistance. • Patents and citations are not the ideal indicators for a demonstration focussed

programme, though they are still of interest . • The SET plan KPIs will be of great assist ance in measuring project success. • Commercial participants are more interested in and optimistic on commercial

benefits, than research focused participants. This may seem obvious but it explains a number of tensions within projects and with regard to defin ing acceptable measures of success.


Consistency “The extent to which positive / negative spill over into other social, economic or environmental policy areas are evolving?”

• Multiple socio- economic benefits have been identified, as spin off benefits from the project core, these include::

o Employment, environment, economic, health, networking and skills. • On the issue of energy prices, the project objectives are to improve cost effectiveness

and efficiency, which should; reduce costs to the consumer but energy prices and subsidies are complex a nd set at MS level.

Allocative / Distributional “The extent to which disproportionate negative / positive distributional effects of policies are evolving.”

• Programme data analysis shows: o 89% of the programme budget to date has been spent in the first 4 calls. o Major drop in applicant numbers from 231 in the first call to 45 in the fourth. o Major reduction in areas open for bidding. o Increase in chance of success, from 11% to 33% or 1 in 3. o The average total project value has increased from 6.1 MEUR to 17.6 MEUR

(8.9 MEUR on average over the 4 calls) . o The industrial match funding has increased over time.

• SME participation o 19-20% which is above average (13%) and target (15%) . o Have much to offer but no desire for quotas .

• NMS o Can add value, but quotas generally resisted .

• Clustering and networking o Old MSs dominate - 95% of funding goes to EU 15 . o Research organisations are still highly relevant to the programme but its

change in focus has lead to an increase in industrial partic ipation from 47% to 64% in terms of EU contribution granted .

Sustainability “Will the effects achieved last in the medium or long term?”

• SET plan alignment is vital to the future nature of the DG ENER FP. o Misunderstanding of RTD as well as demo scope . o KPIs need to be referenced . o SET can learn from the experience of FP. o Debate over where to sit the FP in the SET.

• Dissemination o Academic routes adequate for their purposes . o Some support for networking and joint promotion (EU facilitated) . o Clarification of pos t project funding would be helpful .


Recommendations The recommendations have been structured in line with the programme life cycle.

• Information to applicants o Clearly state the programme objectives to applicants. o Clarify that SET includes research as wel l as demonstration. o Clearer description of ‘demonstration’ .

• Programme relevance / strategic fit

o Maintain communication with DG RTD on positioning / focus of calls etc. o Clearly reference the SET plan in FP ENER calls . o Clearly articulate energy efficiency a s a top objective and analyse the whole

of the FP for energy efficiency related projects.

• Programme Management o Trial remote evaluations of applications, with an on site briefing. o Clarify the position on the availability of DG officers for pre submission

meetings with applicants . o Request applicant interviews for the largest projects. o Trial a longer and more inclusive, end of evaluation decision meeting. o Develop and circulate project management KPI checklist. o Clarify position on availability of expert advise rs to DG officers.

• Project and Programme Impact

o Add a post project obligation and payment to report on project impact post completion.

o Request an estimate of CO 2 potential savings in all applications. o Trial collective presentation of FP projects at a high profile trade show /

conference. o Clarify position, and circumstances, for additional funds at project end.

• Programme Future

o Accepting that the SET plan is key to the future of the FP Energy projects, a debate needs to be had on how, and if, the FP should retain its identity.

o The SET plan operation should learn for the experience and knowledge of the FP.

o DG ENER need to continue to make their voice heard in the evolution and development of the wide range of EU activity starting up in relation to low carbon research, development and demonstration, in order to try to m aximise the synergies with other EU spend .


1 Introduction

1.1 Purpose and Status of this Report

This is the final report for the Mid-Term Evaluation of the Seventh Framework Programme (FP7) Energy p rojects supported by the Directorate General for Energy (DG ENER). Our intention is to meet the requirements for the final report as defined in the Terms of Reference (TOR) - which are: "A final report that will take into account the commission's comments and requests. The final report, containing all relevant information shall also provide a written summary aimed at a non -specialised audience. As all impact studies shall be available to the public, no form of confidential data shall be contained in the final report (they shall be included in a separate Annex)". To this end, the report has an executive summary designed to be understandable by non specialists and is structured with an introduction followed by the findings presented against each of the primary evaluation questions. Each of the primary evaluation questions is defined and then our reasoned and justified answers to the specific questions of relevance from the Terms of Reference are given. The main findings from each of the research methods used (literature review, data analysis, participant survey and consultations) are then presented. The report concludes with a s ection which summarises our key findings and recommendations structured against the programme life cycle. The annexes to the report include the detailed results of the participant survey, a detailed review of relevant policy and previous evaluations, a re view of programme evolution and the SET plan, the detailed findings from the stakeholder and participant surveys and a list of consultees.

1.1 Overview of method

Figure 1.1 outlines the methodological approach we have taken to this study.

Executive summary intended for non specialists, followed by a summary of answers to each evaluation question then recommendations.


Figure 1.1 Methodology

Our desk research has involved a review of the previous evaluations of relevance to this work as well as the programme documentation and the fas t evolving policy analysis in this field from information sourced by ourselves and signposted during consultations. Our consultations have involved a number of groups and methods. In consultation with the Commission we contacted a number of high profile experts in the field of low carbon research and development and asked them to act as an expert panel for our project. The purpose of this panel was to help us ensure that our work is in line with th e most current thoughts in the field. To this end the initial consultation with the panel members covered the intervention logic for the programme, the survey questions we proposed, and the structure and nature of the calls to date, and in the future. The next stage of interaction with the expert panel concerned the draft conclusions and recommendations of the draft final report. We carried out structured interviews with a broad cross section of Commission officers from DG Energy and DG Research. These vi ews were very valuable in both testing the opinions of

Literature review of policy and previous evaluations.

Consultations with experts, DG officers and participants.


others, testing the survey results and in collecting ideas and suggestions for improving the programme. We also interviewed a number of high level stakeholders . These individuals were identified and selected with the assistance of the Commission as being able to give an excellent insight into the thinking and decisions behind the programme we are evaluating as well as giving high level opinions on the current thinking in the sectors of most relevance, i.e. renewable energy and energy efficiency. We also carried out individual interviews with over 15 current project participants, from an agreed sample of 10 current projects, who are by definition important stakeholders in relation to the energy aspects of the FP. We have reviewed and analysed the data on the projects funded to date to seek insight in to the distribution of funding by nature of applicant and subject matter. In addition to this analysis we have reviewed the strength and nature of the networking between organisations and countries that the programme has enabled. An online survey of 2 major groups of Framework Programme participants was carried out:

1. Current participants (project co -ordinators and partners) in FP7 Energy projects – this covered the 49 projects approved and funded under the 2007 and 2008 calls.

2. Past participants in FP6 but not FP7 – principal contractors or project co -ordinators of FP6 Energy projects that as far as we could determine were not receiving for FP7 Energy funding.

Table 2.2 Shows the response rates that were achieved. The survey for current FP7 participants achieved a remarkably high response rate of 46%, with over 220 res ponses. A good response rate of 20% was also achieved among group 2. Response rates of this size will give a highly representative indication of the views of FP7 energy participants and high levels of confidence can be applied to the results of the current participant survey. Table 2.2 Survey response rates

Group Contacts emailed

Inactive / Opted out

emails

Total actual survey

audience Responses

received As % of

total audience

1. Current FP7 Energy participants 510 33 477 221 46%

2. FP6 (but not FP7) participants 251 49 202 40 20%

The survey questions and detailed replies are included as an annex to this report.

Review and analysis of programme data.

On-line survey of current and previous participants.

High response rate on participant survey.


2 Relevance / Utility / Acceptability

2.1.1 Definition

Relevance and Utility “To what extent are the obje ctives of a public intervention (project, programme or policy) appropriate regarding the needs perceived and the problem the intervention is meant to solve?”

Acceptability “The extent to which stakeholders accept the policies in general and the particular instruments proposed or employed .”

2.1.2 Specific Questions and Summary Answers

The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opini ons that we have collected. What is the perception of the supported activities by external actors (consortium members and coordinators, especially for the industry sector)? What could be the approach to better match their expectations? This issue was much discussed with all of those consulted. The participant survey showed that they perceived a good, and improved, fit between the programme objectives and their own and soc iety's wider energy needs. An interesting finding of the participant survey was the improvement that respondents saw in terms of the strategic fit of the programme since previous FPs and also during FP7. This is a vindication of the change in programme targeting and the alignment with the SET plan. Past participants in the DG Energy part of the FP6, who had not yet applied for FP7 funds were surveyed with regard to why they have not reapplied, in order to see if the programme was somehow missing their needs. It appears that there was no strong perception of this and that these past applicants would reapply if a suitable project became available. The most popular reason given by FP6 participant s who have not received FP7 funds for not yet applying is that a suitable project for them to participate in has not yet come along, another reason quoted was the lack of a suitable call for their area of research. This indicates that some researchers are being excluded, however the reduction in the breadth of the calls was generally regarded as being the correct decision for othe r reasons – mainly efficiency.

Participants perceive a good match between the programme and society’s energy needs.

Past participants still see the programme as relevant to them.


There was a clear consensus that the shift to larger projects was welcomed, particularly by the industry sector. From a policy and replication perspective the large projects should have the larger impact in comparison to smaller (but more numerous) projects that they are expected to. The industrial participants found the larger budgets and increased chance of success in terms of being awarded funding, to be positive. The perception of the programme, and its influence on the activities of the energy industry was raised by one strategic consultee. Their opinion was that the FP has had very positive role in persuading large energy companies to invest more in low carbon r esearch, by both funding research into areas that they were unwilling to pursue or by encouraging them to pursue them with funding support. The example of the research funding put toward smart networks was mentioned as a good example of this , as this is an area that electricity utilities had not pursued to the extent which some felt they should , but FP funding has helped others pursue the area and the utilities have now increased their own efforts. The consultee felt that without FP it is quite possible tha t this increase would not have happened, though it is very hard to know the counterfactual. The case could be made that the expertise funded by the FP in this area would not exist otherwise , so we would be behind in terms of a starting point for the SET plan. This was felt to be a reasonable assumption given the very low funding that went to clean energy R+D in MS level energy research. The policy influence of the FP was also mentioned as an important factor in pushing the issue up the agenda of MSs, and h elping build the case for the policy targets in this area (the 20/20/20 package) which has been developed over the last 3 -4 years. However a number of the project participants were less convinced of the importance of the FP in influencing the agendas of companies and MS. We feel that the strategic influence of the programme is positive, in terms of promoting low carbon energy research and technologies, but relatively minor in comparison to wider market and policy signals. To what extent do the FP7 activiti es mentioned in the programme "Cooperation" adequately match the political objectives in the field of energy? A key way in which we attempted to answer this question was by reconstructing the objective tree and testing this with stakeholders. This objective tree attempted to c ombine the objectives in the field of energy with those of the FP7 programme more widely. This pr ocess illustrated that such a combination was possible but that there were some differences of opinion on where the priorities lay. The majority felt that the objective of reducing greenhouse gas emissions (and those objectives which link most closely to t his, such as reduced demand, improved efficiency and increased use of renewable energy) was the most important, while others felt that objectives relating to supporting and enhancing European competitiveness and research abilities should be at least as imp ortant. In the opinion of the evaluators the fact that an objective tree created from a combination of energy and FP7 objectives has a very large number of complementary and mutually beneficial objectives (such as the positive contribution of energy effici ency to EU competitiveness) illustrates the strong match between FP7 and energy policy objectives. There remains a need to keep applicants aware of the breadth of objectives that the programme has and to help them understand the ways in which their project s could and should contribute to these.

Industrial participants and stakeholders support the move to larger projects.

The programme has a positive influence on the profile of low carbon research and technologies across the EU.

An objective tree combining FP7 and energy policy objectives was constructed and found generally acceptable with many complementarities.


The participant survey indicates that they have a reasonable, though not very strong, understanding of the programme level objective s. While a very strong understanding would be ideal, it could be argued that it is not vital in attracting good and relevant projects, though it does indicate that perhaps more effort should be made to clearly articulate the objectives to applicants. The b eneficiaries do regard their projects as having a clear link with the high level objectives of the FP7, particularly reducing greenhouse gas emissions and improving energy security. The objective that least projects feel a strong match with is the competit iveness of the energy industry. It is also interesting to note the low risk rating that most project partners assign their own projects. As is discussed under allocation later in this report there are more renewable energy projects supported by ENER than energy efficiency projects. With regard to the bala nce between renewable energy and energy efficiency focused projects. Most of those consulted agreed that the balance was in favour of renewable energy, though it was pointed out that there are other parts of the FP which support energy efficiency related p rojects (e.g. resource efficiency, ICT projects) and joint calls related to buildings, which go some way to address this balance. The most significant reason given for more renewable energy than energy efficiency demonstration projects coming forward and b eing supported relates to the presence of much better market conditions, mechanisms and incentives to support renewable energy in comparison to the relative failure for such market incentives to materialise for energy efficiency. With regard to the demonstration focus of the calls the case was made that DG ENER should look to retain a portion of funds to s upport smaller, more research focused projects. The clear counter argument to this is that the DG RTD supported elements of the programme will support projects of this nature. However it is clearly important for DG ENER to maintain an input to the focusing of DG RTD calls. It was suggested that the call for proposals needs to make it very clear to applicants what is meant by a demonstration project in order to avoid applicants attempting to present research focused projects as demonstration. Individual sectoral strategies or sectoral road maps are aimed at further improving the implementation of FP7 and at ensuring a better achievement of the expected deliverables. Should they be developed? The key issue for this question relates to the existence of the Industrial Initiative plans and their associated road maps and Key Performance In dicators (KPIs) that have been recently developed under the Strategic Energy Technology (SET) Plan. For most relevant energy technologies these plans have a very strong relationship with the objectives of the FP, particularly relating to demonstration and replication. For those technologies there was a clear, and in the opinion of the evaluator logical, consensus that the DG ENER FP work programme should make explicit reference to the SET plan and the Industrial Initiative roadmaps. Some areas, particularly energy efficiency, are so far less well covered by the SET plan Industrial Initiatives but this should improve over time. The detail of the plans for more research focussed areas is also less clear, though this too should become clearer as the SET plan develops

Applicant understanding of programme objectives is acceptable but could be better.

The programme appears to support more RES than EE projects. Across the whole FP the balance may be closer but market conditions are more favourable in terms of large scale replication of RES technologies.

DG ENER focus on demonstration projects while DG RTD focus on further from market research. This split and coverage need to be made clear to applicants.

The SET industrial initiative road maps are a clear and sensible way to meet the needs for an FP7 energy roadmap, and as such should be referenced in the programme documents.


What would be the added value of international cooperation if implemented in the supported activities? This question was interpreted as an interest in investigati ng an easing of the restrictions / increasing the funding available for non EU partners. There was support for such an increase / easing in some technologies where either the non EU country involved was exceptionally strong (e.g. Brazil in biofuels) or the technology was so far from market that a global approach to solving the problems was seen as beneficial (e.g. wave power). There are other parts of the FP, and MS level programmes available which work on this basis. However with regards to the DG ENER par t of the FP the existing situation was felt to be a good compromise between the benefits of accessing non EU expertise and demonstrating in these markets against the risks of loosing intellectual property from the EU and supporting non EU industries. Does FP7 play an adequate role in positioning Europe on the global map of science and technology for energy? Virtually all of the project participants consu lted ranked themselves and their projects as among the world leaders in their fields. A number of the strategic consultees had a slightly lower opinion of the quality of many of the participants. The most quoted international competitor was the USA who wer e characterised as adopting a more focussed and intense approach than the FP with large sums being directed to specific areas, which gives them a good chance of getting ahead in these areas. The link with MS programmes and industrially funded research is an important one here as both of these sources of funding are larger than the FP. The SET plan, with its intention of leveraging private and MS funds, is a potentially vital element here. There was little progress made in identifying measures which would help prove this. It was generally agreed that although useful and relevant, patents and citation indexes, were too open to manipulation and not ideally suited to the demonstration, replication and commercialised energy saving goals of this part of the FP. All of the indicators suggested and discussed, including many of the SET plan KPIs would require an increased ex -post evaluation of projects. To this end it was suggested that some form of additional, formalised post project reporting should be required. The authors of this report suggest that some form of relatively low cost payment could be offered to project coordinators as an incentive t o provide short update reports at fixed periods after project completion.

2.2 Intervention Logic / Objective Tree

Our first step in considering the questions of relevance was to formalise what the programme objectives are by drafting an updated objective t ree based on programme documentation and our own knowledge, which we then tested via consultations with strategic stakeholders, DG officers, project participants and our expert panel. The m ost significant update to the objectives is the creation of the SET plan and the decision to align the FP7 projects supported by DG Energy with it.

The current approach to funding non EU partners is accepted as a good compromise.

The project participants consider their status as among the world leaders, but this is hard to prove.

An effective way of testing impact and ranking projects is to formalise (and fund) post projects reporting of impacts.


2.2.1 Fundamental Objective of the Programme

One consultee stated that in terms of the fundamental pu rpose and targeting of FP funds it should be made clear that ideally the funds are intended to supplement Member State (MS) level research and not provide a substitute or means of filling in the gaps. Each MS should focus on those areas which it is best pl aced to exploit, for example where a country has a lot of a particular resource, e.g. wave energy in the UK or Portugal, the MS should lead the funding in this area, possibly with FP funding to supplement it. However, given that low carbon research has no t been heavily supported in all member states, though this has rapidly improved, the FP funds have provided two fundamental support functions: 1. Additional funds and policy justificat ion / validation for those MSs that are already

active and pursuing low carbon energy research and implementation. 2. Funds to support and retain capacity in those MSs where the research and policy

activity to support low carbon technologies has been less in tensive. Most of those consulted agreed that for this programme, at a g lobal level the most important objective is reducing greenhouse gas emissions. The other high level energy related objectives (security of supply and industrial competitiveness) were thought to be important, and to be relevant, but were more 'side effects' from the reduced fossil energy demand that improved energy efficiency and increased uptake of renewable energy. However some thought that promoting the industrial capacity and innovative abilities of Europe was the primary objective. An important high level objective which one consultee thought should be made more apparent is that of the social aspect. This includes a number of issues or relevance, from protecting the global population from the impacts of climate change, to protecting the population from high energy costs. The energy cost issue is a complex one as this can be interpreted in two ways in this area; e.g. more renewable energy can have a negative impact on energy prices, particularly in the short term, as energy prices go up to fund the subsidies provided to renewables. On the other hand much research is focussed on reducing the cost of renewable energy which will result in lower cost to the consumers than would b e the case if the research had not been funded.

2.2.2 Specific Objectives

Energy efficiency has been explicitly mentioned at a high level in the objective tree. The justification for this is that energy efficiency is the most cost effective way of reducing GHG emissions, and if 'closeness to market' is an important criteria for the programme it should arguably be higher rated than renewable energy support. Although beyond the scope of an objective tree the relevant point was raised that there are other EU funds with similar objectives, for example schemes such as the EU ETS. A number of consultees raised the point that an object ive regarding the need to achieve system changes, or enabling them would be a useful addition. For example smart grids (which is a multi dimensional issue) and Carbon Capture and Storage (CCS) – which requires the optimisation of multiple technologies and where the optimum choice is not yet clear.

Some feel FP funds should supplement MS funds and avoid MS specific strengths.

FP funds can also help justify MS activity and help fill gaps for others.

CO2 reductions are generally accepted as the primary objective of the programme though industrial competitiveness and security of supply are high profile, and generally complementary.

Social objectives are present but were not felt worthy of more intense targeting.

Energy efficiency was identified as an important and specific objective.

There is multiple cross over with other EU funded programmes.


2.3 Consultation Views

2.3.1 Evolution of calls in FP7

Our consultations have revealed a number of interesting points regarding the way in which the calls under FP7 have evolved. The first call was described as being drafted with an FP6 mentality which led to a relatively high number of projects with a high number of participants being supported. There is no formal, specific evaluation of the projects supported by DG TREN under FP6 but it is clear from the opinions gathered that there were a number of problems. These problems can be summarised as follows:

• The high number of projects and the high number of partners within these projects resulted in a high level of administrative requirements for TREN, for example more

partners increased the chances of partner changes, which is an admin heavy task. • The high number of partners implied a low budget per partner which can increase the

likelihood of the project receiving little attention.

One external consultee raised the point that the decision (to reduce the number of areas open and go for fewer, larger projects) could be justified solely for administrative efficiency reasons. These related to the very poor perception among some beneficiaries and applicants that developed regarding FP6. These could be summarised as stemming from lots of small projects, with lots of partners, leading to diluted efforts which acted as a disincentive to many applicants. The creation of this disincentive needed to be addressed and the approach adopted has achieved this. When asked their opinion on whether the FP7 programme had become better able to help achieve its objectives in comparison to FP6 and during its lifetime many of the on line survey respondents replied that they did not know, but a very clear majority of those that did express an opinion thought it had improved since FP6 with a smaller majority thinking it has improved during its lifetime. Previous participants in DG ENER FP Energy funded projects also felt that the programme remained relevant to them. The fact that no appropriate project had arisen was found to be the most important factor in not applying to the FP7 Energy programme from the survey of previous FP Energy applicants. Other lesser factors were also identified including; no FP7 funding in their research area, a focus on national (public) funding sources, insufficient funding and issues around transnational partners and the required consortium approach. There was no significant support for the view that FP Energy was no longer relevant or that previous experience of FP acted against future participation in the programme. Overall the views of the programme were positive from past participants that have not yet applied to FP7, the majority feeling it likely that they would apply for FP7 funding in future if their research area was called and a suitable project for their organisation arose.

2.3.2 Fewer Projects with Less Partners

The switch to fewer projects, with fewer partners, which began in the second call and has been retained since, raised a number of points.

The poor perception of FP6 was an important factor in the decision to realign FP7.

Project participants felt that FP7 was a better strategic fit than FP6.

FP6 participants who have not yet applied for FP7 funds still consider the programme relevant to them.


A benefit which was mentioned by both Commission staff and external consultees was the reduction in the administrative workload of commission staff. This brings a number of positives, including:

• The project officers have a more reasonable workload, so can do a better job. • The officers are able to focus more closely on the project, so learn about it, and from

it, more, for example each project should now get at least one visit from the project officer.

• Learning from the projects is useful as the project officers are involved in policy development so this direct and practical involvement with research is of clear benefit.

The ideal situation, in terms of project workload on an officer, was described as a portfolio of some large and high profile projects plus some smaller ones. The generally increased scale of projects (average project value up from €6m to €15m) means that the projects should be of sufficient scale to achieve high impact and visibility on the research landscape. Larger projects are also more likely to attract attention within the Commission which helps give profile to the policy goals and activity of DG Energy and can even help in terms of speeding up Commission administration. Industrial participation in the DG ENER part of FP7 is high, currently reported as over 70% which is much higher than the rates achieved under previous FPs. The high level of industrial participation has led to large sums of private funds being attracted to match the commission funds. This is believed to build in a positive cycle in that a large cash contribution from a private firm will increase and improve the management and input from the company, which makes for better projects with less management input required from the Commission. The major reduction in the number of topics open per call was also described as having the following benefits. As the total budget remains the same, the amount per open topic has increased. As the number of potential, credible, participants in a particular field is limited the implication, and reality, of the situation has been larger budgets per project and larger budgets per consortium member. This increase in budget is perceived as being attractive to potential applicants. This issue is illustrated by the reduction in the number of bids and the subsequent increase in the chances of a successful application. One consultee, who is employed by a large company that has participated in FP7 projects, agreed that fewer and smaller projects resulting in a higher success rate (for applicants) does make the FP more attractive to large companies, as they see a better chance of winning and the effort they expend is more likely to result in funding. The improved chances of success when bidding for funds, narrowing from 1 in 10 previously, to 1 in 5, or even lower, is more in line with the chances of success companies typically work with when bidding for commercial contracts. This motivates better bids, particularly from commercial organisations. Improved motivation also relates in part to the significant time input required in preparing a bid of this nature, for example the need to identify demonstration sites usually requires the agreement and buy in of a third party such as a local authority.

Fewer projects with less partners has helped enable efficiency improvements.

Larger projects are accepted as higher profile than smaller projects.

Industrial participation has increased.

The reduction in the number of areas open per call has helped increase the budgets per project and reduce competition.

The increased chance of success have made the FP7 more attractive to industy.


With regard to the question of a general approach of many small vs. fewer and larger projects there were a number of interesting points raised. While it was agreed that in general projects with lots of partners are more difficult to manage, there are some cases and target markets where this approach is beneficial. For example in some building related projects the ideal consortium would have a representative from each part of the long (in comparison to renewable energy) supply / value chain. Although this implies a high number of partners many will only have a small, but important, involvement However the change to fewer and larger projects is recognised as having a number of down sides. These include:

• A perception of a focus on large companies, though in terms of SME participation rates this does not appear to be true.

• Some past participants are unhappy that their opportunity to participate has reduced. This is most often the case for smaller research institutes, which are more concentrated in certain member states.

• There is a perception of decreased participation from new member states, partly due to potential applicants there lacking the administrative capacity and experience to submit bids.

• With regard to projects which are not focussed on demonstration, the overall approach of less and bigger projects is less obviously beneficial. In these more research based areas narrower calls will lose potential ideas and approaches. Bids of this nature are often easier to put together than large demonstration type projects, for example because there is no need to agree large sites, so applicants are more likely to be willing to accept a lower chance of winning.

• It is recognised that, although improved, the flexibility in the financial arrangements could still be better.

2.3.3 Nature of Projects Requested and Supported

Those consulted raised the following points regarding the mix of technologies covered in the calls and approved projects. The fact that each call description is presented as a combined document between DG Energy and Research was reported as intentional as it gives a picture of a coordinated and larger approach than separating the focus areas between the two DGs. In terms of participants perception of the cost and risk profile of their projects around 80% of respondents to the survey felt their project was of medium or high technical risk. This indicates a quite high risk profile for a programme which is seeking projects relatively close to market and one where 46% of the recipients consider that their project is of a demonstration nature. It is possible that the beneficiaries were describing risks other than the risk of the technology not being commercialised. In terms of participant understanding of the aims and objectives of the programme, 76% of the respondents felt they understood them very, or moderately well. A similarly high percentage of respondents felt their project was a good or very good match with each of the high level objectives, with contribution to energy competitiveness felt to be slightly less strongly supported than reducing carbon emissions and improving security of supply. The energy specific objectives of the programme were also felt to be well matched, with

Bigger projects have led to the exclusion of some participants and is not ideal for all subject areas.

Bigger projects makes it hard for small research organisations to participate. This relates to the size and focus on demonstration as opposed to research.

The calls are intentionally co-presented by DG ENER and RTD.

Beneficiaries consider their projects to be riskier than would be expected for a demonstration focussed programme

Participants understand the programme objectives reasonably well but not fully.


developing technologies which could achieve short term market penetration slightly less strongly supported than the other energy objectives.

2.3.4 Picking Winners and Research vs. Demonstration

The question was raised of balance between the objectives, with the assumption that a portfolio approach, where projects covering all potential areas of research are requested, will allow comparison between the technologies and multiple approaches to the same technology. This is a first step to 'picking winners' both within a technology area and between technology areas. Such a portfolio approach also allows cross sectoral issues to be picked up. This leads on to questions around the fundamental targeting of the programme, which revealed a number of alternative approaches and opinions, some of which are energy specific and some of which are more generic, though still relevant. The consultee from the UK’s Carbon Trust (CT) provided an interesting comparison between their approach and their perception of how the FP works. He described a high level split in the approach, such that a programme could either focus on known areas of strength or improving areas where the EU performance is currently weak, but improvements are necessary / thought to be beneficial. The CT approach was described as being to offer a range of schemes to fit the variety of R+D needs. This approach was felt to differ from that of the FP which is perceived as starting from a final objective and working in a top down manner to specify the research it wishes to support. One consultee described the current situation in terms of technology choice in low carbon research as still being at the ‘let all flowers bloom’ stage, i.e. the problem is so large and important, and the potential solutions so diverse that it is unwise to try and focus on particular technologies to the exclusion of others. Another reason not to remove any technologies and to continue to support all is that the policy environment in terms of member state and industry commitment to low carbon is still fragile. It is also the case that the technologies which eventually emerge as the most important may not be those which appear best placed at the moment, or those which are technically / theoretically the best, as this is how technology development works. With regard to the focus of the projects, one consultee stated that the focus on near to market research was understandable, as industry is motivated by the prospect of profits and there is a need to spur carbon savings as soon as possible. This needs to be balanced with a recognition that innovation does not finish when a product comes to market. Ideally there should be some funds available to support all stages of the research cycle, including product development. This point is of relevance to reducing energy use in the buildings sector, as here research needs to focus on the cost reduction of relatively mature technologies - i.e. more like product development but still research. Industry needs to have confidence that they can look for support to help them plan investments. Some of the DG officers felt that ideally the programme should look to retain some balance, within each project and across projects, between demonstration for commercial benefits and scientific benefits, with the projects selected for the scientific benefits typically being more cutting edge and riskier. This balance needs to start in the calls for proposals. On a similar note one officer felt that it would be good to retain some of the budget to support smaller projects. Another officer agreed that supporting some smaller projects reflects the industry approach as they will carry out smaller scale testing prior to large demonstration.

The programme continues to balance picking winners against supporting multiple possible approaches.

.. and between addressing weaknesses or supporting strengths.

There is a clear need to support research as well as demonstration.


In order to help applicants clearly understand what ENER mean by ‘demonstration’ projects it was suggested that the definition used in the call for proposals needs to be very clear. This will avoid applicants submitting and presenting research projects as demonstration (not to say that research projects shouldn't be funded). The attempts in previous calls to define ‘demonstration’ could be improved. From the DG RTD perspective it was felt that DG ENER now managed the demonstration projects and DG RTD the research focused projects. It is important that research is continued to be supported as it drives innovation. With regard to putting both research and demonstration projects under DG Energy, this approach was raised by a limited number of consultees but was not widely supported by the majority, including those from DG RTD. One officer felt that the clear focus on demonstration had helped clarify the division between DG Research and DG ENER – this split (in terms of distance from market) had not always been obvious to external observers.

2.3.5 Balance Between Renewable Energy and Energy Efficiency

With regard to the split between RES and EE one DG officer made the comment that if this split is done by project numbers it can be misleading as the large CONCERTO projects might have been 3 to 4 separate projects in the past but the focus on larger projects has led to them being combined. Analysing by funding can also be misleading as some of the efficiency projects will have an impact on a lot of buildings despite being relatively low budget in comparison to RES projects. The building specific calls have helped to keep the balance – which is better than it appears. In general terms, heat related projects are less supported than electricity related projects, if the programme wishes to reflect the use of energy, this is an under representation. This also relates to the fact that heat is not a traded and transported commodity like electricity. Improving the efficiency of its use will require approaches that are attractive to consumers, which has been something of a market failure to date. Energy efficiency is more cross sectoral than RES in terms of the technologies being developed. Much of the research / improvements available relate to controls and ICT and these projects would typically not come under DG ENER. Energy use in buildings is complex and many of the FP projects / calls are pursued in conjunction with other DGs, (2 calls of this nature were mentioned). One officer raised the logical point that any relative (to renewables) lack of energy efficiency demonstration projects, fundamentally relates to the lack of a winning commercial business model for energy efficiency. The technology is mostly well known and proven, but the economic models and incentives are still to be properly devised. Many of those consulted felt that the current approach of supporting large, demonstration focussed projects presents an easier opportunity for renewable energy focussed projects than for energy efficiency focussed projects. This was believed to partly relate to the nature of the projects, for example a large scale installation of a new wind turbine is a simpler prospect (one site and one technology) than installing multiple small scale efficient appliances. The Concerto approach (considering multiple low carbon solutions on a spatial basis) was mentioned as a way in which this issue had been addressed in the past.

Applicants need to be clearly reminded what demonstration as opposed to research means.

The DG RTD / DG ENER split is clear to the DGS. Putting the research focussed activities under the control of DG ENER was not widely supported.

It appears that RES projects receive more funds than EE but this simple split misses the whole story.

Energy efficiency brings quicker financial returns and CO2 savings than RES so some say it (EE) should be the priority.

Other parts of the FP fund EE related projects.

The market conditions / subsidies are better developed for RES than for EE, making demonstration projects more feasible in RES.


With regard to funding projects focussed on industrial energy efficiency, another problem with these is that there is a perception that industry should be pursuing these opportunities anyway, particularly as energy costs are so high, leading to a significant risk of a lack of additionality. Despite this concern some projects are funded, about 20% of the project portfolio. It was suggested that the ideal solution would be a cluster of projects to help get scale and momentum.

2.3.6 Quality of Supported Research

Views on the quality of the research supported were sought from Commission Officers and project participants. The global ranking of energy R+D in Europe was reported as an issue which is always considered when assessing applications. The point was also raised that as there are now fewer but larger) projects it becomes even more important, because the numbers of projects are lower, so more is more pressure on each individual project to deliver something of global significance.) With regard to biofuel research and demonstration, the opinion was raised by a DG officer that FP support has helped put the EU at the front of global research. However, it is difficult to compare EU research with the position in the US, as their approach is different. The US approach involves more funding and more concentrated efforts on a technology once it has been selected, e.g. ten projects funded at one time in biofuels in order to increase the chances of success. This gives more intense coverage in some areas than the EU, but less in others. Most of the coordinators interviewed felt that FP7 put Europe at or near the top of the world rankings in terms of energy research, and that it was important, indeed critical, for EU competitiveness that that was the case, though one project co-ordinator felt Europe lagged behind global competitors somewhat. He did comment, though, that FP7 was helpful in somewhat redressing the imbalance. That said, few felt they had a sufficient grasp of the overall balance of energy R&D funding globally on which to make comparisons, though in their particular topics and fields of expertise, most felt the FP7 projects were cutting edge. A number commented that on renewables development and deployment, Europe was world leading, and particular applications such as laser treatment of photovoltaics was global state of the art. Another consultee also felt that the EU was currently leading in CCS but the US were now dedicating large resources to this area and could overtake the EU. The comparison to the US was made by another consultee who felt that the FP7 has smaller budgets than the US Department of Energy research programmes, both for research and demonstration focussed projects. One interviewee pointed out that there was a global increase in the level of interest in, and resources directed towards renewable energy sources, which he characterised as a global shift away from IT R&D towards renewable energy. This opinion was endorsed by another consultee, a large commercial partner, who commented that the absolute importance of EU or Member State funding of research had reduced somewhat in recent years, as companies have a sufficiently strong commercial driver to undertake research in house or with partners but without public funding, the benefits to firms being greater control over the research, the ability to progress more swiftly (not having to wait for a decision on funding award and subsequent contract negotiation), less

Demonstrating simple energy efficiency projects risks becoming industrial subsidy.

Global ranking is considered in evaluations.

Many DG officers and participants regard the work in their fields as world leading. Others are less convinced but without the FP they agree the situation would be worse.

The US approach to funding energy R+D tends to be more funds and more focussed.

Low carbon R+D investment is increasing globally due to market and policy pressures.


bureaucracy and, perhaps most importantly, the ability to keep highly commercially sensitive work secret. The downside of purely commercial work is that results don't get shared. The implication is that the terms for commercial partners to participate need to be made sufficiently attractive such that strategically important work could be done within the context of an EU public/private project.

2.3.7 Increased support for International (non EU) partners

The question of whether the quality of projects would be improved if it was made easier for non EU applicants to participate in projects drew a variety of comments. It was pointed out that there are already a number of other parts of the FP, and other EU funding sources specifically targeted on cooperation with partners from outside the EU. For example a joint RTD/ENER topic, under the Specific International Cooperation Action, focused on EU cooperation with India. There are also overseas aid programmes which can support energy infrastructure development and it should be remembered that the FP has a very different purpose to the aid budget. On international partners, the benefits of additional inclusion were felt to vary by sector. Where there are clearly global leaders who are not EU based, (e.g. biofuels in Brazil) there is a good case, but where the EU leads (e.g. wind), or where the project will result in developments outside of Europe the case is much less strong. For new sectors (e.g. wave) all developers have the same initial goal (i.e. a working large scale prototype) so it is possibly easier and more sensible to collaborate. Two interviewees in favour of international co-operation argued that "artificial geographic limits" do not reflect the reality of teams and production capacity being placed anywhere globally, even if the parent company is European. One argued that, given the global nature of the GHG problem, it was important to have global co-operation on solutions. Other points raised in favour of increased cooperation with non EU partners included the expertise in particular technologies that exists outside the EU, the extra scale that more international partners could bring, which is of particular relevance for major issues such as CCS and the benefits of being able to site a demonstration project outside the EU where the largest potential market for that technology is outside of the EU. Where non EU cooperation is supported it should be done on an equal basis (in terms of funding support) and care needs to be exercised with regard to leakage of technology and expertise, especially to countries who are actually at the cutting edge themselves, e.g. current coal generation plant installations in China are more efficient than in the USA and emit less CO2 per kWh produced. Contractual complexity also increases with the inclusion of non EU partners, which is another reason for limiting it. In conclusion the most commonly held opinion was that international partners can improve the quality of projects, but also bring risks and downsides so the current flexibility is a fair compromise.

Other sources of funding, including other parts of the FP, are focussed on growing international cooperation with non EU partners.

More cooperation with non EU partners is of benefit where the world lead is outside the EU, or the market is immature.

.. or the market is outside the EU.

There are risks of knowledge and commercial benefit leaking outside the EU.


3 Efficiency

3.1.1 Definition

How economically have the various inputs been converted into outputs and results? Were the (expected) effects obtained at a reasonable cost?

3.1.2 Specific Questions

The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opinions that we have collected. Are the activities carried out efficiently, clearly, appropriately and cost effectively?

The participant survey indicates that the majority of participants do not consider the administrative effort required of them to be excessive / more than they expected. However it should be pointed out that a number of participants do still consider the administrative efforts required to apply and participate as being high. The aspect which drew the most consistently negative comment was the length of time taken to move from project approval to contracting. This is an issue across the FP and an area where improvements would be appreciated by both participants, stakeholders and DG ENER staff. Analysis of the survey returns to identify if previous experience of the FP Energy programme reduces the time required for applications and administration found that this has little or no effect on time spent. In fact, contrary to expectation, the only slight effect detected was that those with past experience tended to spend more time on applications than those without. For SMEs there was variation from the mean on efficiency, on average they spent less time on applications but more time on project administration than non-SMEs It is clear from the DG Officer, stakeholder and participant views that the move to fewer and larger projects has brought an improvement in the efficiency of programme management. The idea of using an external agency to administer the ENER FP projects was not strongly supported and a reasonable case was made that the added value that is gained from the DG officers getting a first hand insight into research and development in their energy policy fields outweighs any efficiency gains which an agency might bring. Although external expert support is available to assist DG ENER officer in managing projects and reviewing the interim and final reports the availability of this expertise appears to be on a somewhat ad-hoc basis. The programme management could be improved if this situation was clarified.

Participants find the admin burden high but as expected.

Time form approval to contract is too long.

Previous FP experience increases the time spent on applications. SMEs spend less time on applications but more on operation.

Fewer and larger projects has boosted efficiency.

External expert assistance is available but the arrangements are unclear.


One consultee felt that the fragmented way in which the FP is managed causes problems in terms of management optimisation, as different parts use different approaches which makes it hard to compare, and harder to analyse and group the expenditure. Are the human and financial resources devoted to the supported activities appropriate to reach the objectives stated? The majority of those asked this direct question felt that the resources were appropriate, though everyone would appreciate more funding. Given the increases in budget that the FP has already had, the economic problems currently facing the EU and the SET plan approach (looking to attract more private, MS and other funds for low carbon energy research and demonstration) it does not appear appropriate to suggest seeking major additional funds at this point in the FP7. With regard to the overall budget of the FP in this area, a technical point which was raised was the benefits that would derive from a mechanism which enabled funds which were not spent, as a result of project failure / early termination) to be retained within the programme, rather than lost back into the general Commission funds. Have the IT systems been sufficient to allow the efficient and effective management of the programme? The quality and ease of use of the IT systems involved in applying for and administering FP funds was somewhat higher regarded by the project participants than was expected. Although some participants noted problems with the IT system (e.g. the searchability of CORDIS could be improved, problems with the participant identification code system) most felt that the systems had improved over time. There is some frustration evident among DG ENER staff regarding the IT systems, largely due to the slow rate of change and the difficulties of improving systems which are operated by external contractors. Has the application process been efficient in terms of timing and content of calls? The successful applicants had no major concerns with the application process. The DG ENER officers consulted made a number of helpful suggestions regarding the details of the application and evaluation process. The ideas which appear strongest and best supported, and therefore worthy of at least a trial, are:

• Pre-submission advice- Clarification of the procedures for accessing Commission staff prior to the application. Although already covered in briefing days the procedures should be clarified.

• Applicant interviews- Short interviews for the applicants behind large projects. With the exact individuals – i.e. which partners the coordinators chooses to bring along, being left to their discretion.

• Remote evaluations - An approach to evaluation where the experts are briefed in Brussels, then evaluate the projects at their own premises. The on site procedure involves the external experts attending an ‘evaluation week’ where the meetings and panel reviews occur in Brussels with all the evaluators present, over a period of 5 days. In remote evaluations the external evaluators take the bid material home to review following a Brussels based briefing. The majority of the officers consulted felt that on balance remote evaluations were better than on-site, with the ability they

FP management is fragmented across the DGs

Funding is generally regarded as enough, more could be used but the economic downturn and the focus on attracting private and MS match runs against this.

Unspent funds at the programme end should be retained in this field.

IT has some problems but not major, Outsourcing slows the speed of response.

Positive suggestions for improving the administration are: - Clarify DG officer access - Interviews for large projects - Remote evaluations -More inclusive end of evaluation meetings.

.- Project management KPIs - Clarify access to external experts.


offer to attract better experts (due to increased flexibility) outweighing the concerns over document security, we agree with this conclusion.

• A more extensive end of evaluation meeting – involving the external experts and extended discussions to arrive at the agreed approved project list. If the resources and budgets are available to support this then it appears a sound idea that is worth trying.

• A set of project management KPIs – to enable easier and more fomalised ongoing assessment of project progress. A short summary sheet would be useful for project officers, combining procedural and output / quality indicators.

• Increased use of external advisors to assess interim and final reports – to bring additional technical expertise and scrutiny to the reviews of project progress. It appears that this advice is available to project officers on request, but the procedures do not appear to be fully consistent. Therefore, on the assumption that the resources are available to provide more support, the procedures should be clarified to enable more consistent, and presumably greater, uptake of external advice.

Those where the balance of pros and cons are much closer are:

• Two stage application process. The positive sides of this are that bidders can submit an outline proposal without dedicating the extensive efforts required to submit a full bid, which could attract more bids, particularly from those for whom the time required is less available (e.g. SMEs). However on the negative side the process adds an extra step of administration and, partly due to the need to review the likely high level of projects quickly, the step runs the risk of missing potentially strong bids which have not fully developed. These negatives in combination with the relatively small number of projects that are capable of being supported (due to their high cost) and the attraction of the high success rate for industrial applicants, lead us to the conclusion that this approach is not currently worth pursuing.

• Increased involvement of project officers in financial monitoring – to give the officers a deeper understanding of the projects and to increase scrutiny as to the appropriateness as well as accuracy of expenditure. This idea was not as well supported as the other suggestions and we agree that on the assumption the financial aspects of the project are properly reviewed at application stage and any budget change requests are reviewed by project officers this should not be pursued.

• A page limit on applications and reports – to focus the efforts of applicants and beneficiaries and avoid the inclusion of superfluous detail. Given the practical / procedural and, potentially even legal difficulties of enforcing this we suggest a continuation and re stressing of the current approach where applicants and projects are reminded of the preference for concise applications and reports on the assumption that it is possible to present good ideas, and good work, concisely and overly long and unfocussed work will be at a disadvantage.

3.2 Participant Survey

The findings of the survey indicate that participants in the FP7 Energy programme typically spend between 20-45 person days each in preparing their application and meeting the management and administration requirements of the programme. For many the burden is

Less well supported suggestions were: - 2 stage applications. - increased financial monitoring by officers. - page limits.


lower but for a small minority of participants (around 10%) the time required is over 100 person days. Further analysis of the survey returns, to identify if previous experience of the FP Energy programme reduced the time required for applications and administration, found that this has little or no effect on time spent. In fact, contrary to expectation, the only slight effect detected was that those with past experience tended to spend more time on applications than those without. For SMEs there was variation from the mean on efficiency, on average they spent less time on applications but more time on project administration than non-SMEs. Overall the programme participants were broadly satisfied with the various elements of programme implementation and particularly with the scientific and technological areas covered, the strategic orientation of the programme and project payment arrangements. There was one clear area where satisfaction was much lower and this was the time from project approval to contracting. For those with prior experience with the Energy Framework Programmes a majority believed that their experience of FP7 is better than in the past and only a very small minority believe it is worse. The view on efficiency from the survey responses is mixed, significant time commitments are still required by applicants and participants, but it can be said with some confidence that administrative and management processes have improved since previous FPs. One area for potential improvement is in reducing the time taken from project approval to contracting.

3.3 Consultation Views

3.3.1 Efficiency improvements that have occurred

The DG officers consulted, suggested that the shift in focus to larger, fewer projects was partly driven by a need to reduce ENER officer workload and it was generally accepted that it has resulted in them being able to monitor projects more closely. DG RTD project officers typically manage between 10 and 20 projects, which is regarded as a high workload. The shift in the focus of the programme – from smaller projects to larger demonstration type projects has required a change in the attitude / focus of effort by the officers as the projects require a different approach in terms of coordination and management (as a result of their larger scale). The change is also regarded as rewarding as the projects increasingly involve the largest, highest profile actors in the energy sector (for instance Vattenfall, Volkswagen, EON, ENEL, Siemens etc.). This involvement is also genuine, e.g. often as a coordinator, rather than in the past when in some projects the large companies were involved but their involvement was perhaps less crucial to the project scope. This makes for projects with an opportunity to give bigger impacts on the sector and on society as a whole. Efficiency in terms of handling and length of procedures is improving. E.g. the online submission and negotiation tool is better than the previous paper / manual approach. The reduction in the number of projects per project officer has been a factor in the reduction of the amount of time taken to process amendments, an issue that has been a long standing complaint from FP participants.

Previous FP experience does not significantly reduce the admin efforts made.

Participants are broadly satisfied with the admin apart from the length of time to contract.

Admin arrangements have improved FPs.

Fewer, larger projects has improved programme efficiency.

Larger projects improve participant efforts / attention.

Improved since previous FPs.


A number of project participants suggested indicators that could be used to measure the efficiency of the programme and projects, these included:

• time to contract; • results of R&D obtained vs. costs, meaning the results are those of the various

projects funded in a framework programme compared with the costs of R&D to obtain those results;

• Number of projects funded; and comparison of the above results with other similar programme (MS or European).

3.3.2 Areas for improvement

Time to contract is still accepted as being too long by the DG officers interviewed. It was also recognised that the complexity of the documents and forms required makes it time consuming for coordinators and partners to take part and it may particularly discourage SMEs. Reducing this complexity is an issue around balancing flexibility and control. There is though the intention to further simplify the procedures for taking part in EU-funded research projects – this was announced end of April 2010 by the research Commissioner following a consultation for ideas in this respect2. The overall aim is to make participation transparent, less bureaucratic and attractive to the best researchers and innovative companies in Europe and beyond.

IT systems were generally agreed to still show room for improvement though the point was raised that it is difficult for DG ENER to effectively intervene in this issue when the majority of the IT services are outsourced outside the DG. Three or four coordinator respondents felt that IT systems were adequate. One person rated them as "seven out of ten". Others had issues. One commented that… "the IT system is complicated and there are different applications, different contact points, passwords and so on. In addition, the IT platform is considered to be not very user friendly. Integration of submissions and a single contact point would be helpful." Four people had used the NEF system – one said it didn't work at all; the other two recognised it was still being piloted, but felt that once the bugs had been ironed out, it should work satisfactorily. Another found the system ok but commented on the large amount of unnecessary information on the screen which could be addressed by hiding functions off the screen until needed. The ECAS user guide was also felt to be overly complex Comments on Cordis were generally slightly negative, with it being described as "non user friendly", "chaotic" and "difficult to extract information from". Potential improvements to Cordis identified include automatically sending information on calls by email to those who have registered details, classification of projects by theme and information on what has been funded (at the moment this list of projects has to be obtained from the national representative and then used as a basis for online searches to obtain further details).

2 http://europa.eu/rapid/pressReleasesAction.do?reference=SPEECH/10/194)

Time to contract is too long. There are FP wide efforts to address this.

IT could be improved but outsourcing makes the responses slower.

Participants are generally supportive of the IT with some suggestions as to how to improve user friendliness.


3.3.3 Project management and external experts

The suggestion was made that for projects with some initial research required prior to demonstration project finance could be structured in a way to allow a 'break point', where the project could either continue with full funds, or be stopped, depending on the success or otherwise, of the initial research. The suggestion was made that technical officers would gain a more rounded and better informed view of the project if they had some involvement with the financial monitoring and control of the project. At present this is done by the financial cell who are understood to be more concerned with the accuracy and correctness of the figures, e.g. expenditure vs. budget, rather than if they are realistic and appropriate. The project officers would need some training in order to do this role. One officer could see no benefits to be gained from more officer involvement in the financial management and reporting of projects, particularly as projects now get more officer time as they have less projects to oversee. A page limit on the proposals and reports was suggested as a way of improving the efficiency of the evaluation process, as often much of the detail, particularly for large demonstration projects, is superfluous. A two step application and evaluation procedure was also suggested. The first stage would be a short (e.g. 10 pages maximum) summary submission, which could be reviewed relatively quickly and, if of sufficient quality, a full application could be submitted and reviewed. This approach is used in DG RTD (with both stages being evaluated remotely). It was suggested that the management / monitoring of projects would be made easier if there were an agreed set of measures (Key Performance Indicators) of project progress. Such a list was reported as existing in the past (for FP5) though this was regarded as too long. Suggestions for a list of KPIs were that it should contain a mixture of administrative type measures – e.g. have the WPs been achieved to schedule and more commercial / scientific measures, e.g. progress towards a marketable product. This would enable progress to be reviewed and the presence and quality of any contingency plans to be tested. Some concern was raised about the ability of DG RTD staff to properly manage some of the complex projects they fund, as the staff do not stay in their roles very long (too much staff rotation). The increased use of external experts to help evaluation etc. was mentioned by one officer as a way to help ease the administrative burden of this on DG ENER officers, such external experts were suggested as also being useful to help project officers review interim and final reports as they have high levels of technology knowledge. The use of external experts to assist in ongoing programme management was reported as also being used by DG RTD on a discretionary basis.

3.3.4 Project Applications and Evaluation Procedures

The method of evaluating proposals was discussed, with the two procedures used by various parts of the FP – remote and on site, both seen as having pros and cons. The on site procedure involves the external experts attending an ‘evaluation week’ in Brussels, where the meetings and panel reviews occur over a period of 5 days with all the evaluators present. In remote

There were a number of suggestions on how to improve programme management….

Programme managers should get more involved with financial monitoring.

Page limits on bids and reports.

Two stage applications.

Project management KPI checklist.

Concerns about high staff rotation in DG RTD.

More use of / access to external experts would be appreciated by some DG officers.


evaluations the external evaluators take the bid material home to review following a Brussels based briefing. The positive aspects of remote evaluations described by officers included the following:

• Improved flexibility and efficiency for evaluators in terms of how they manage their time - which should help attract better experts.

• Less chance of a team decision emerging, as although this is not the intention it is very hard to avoid when the evaluators are all in the same room – this could also be viewed as a negative in that the grouped approach helps to moderate / check opinions.

In order for remote evaluations to be effective it was noted that online sharing of evaluations would be helpful. Some joint FP calls, e.g. with DG INFSO, were described as having been evaluated by this method. It was also pointed out that the experts need to be more experienced and to understand the aim of the programme better, as there is less chance to help / guide them. There is also more of a need to manage the reviewers – as they are remote their timely progress needs to be reviewed and managed. There is also a risk of confidential documents getting lost in transit to the reviewers. Concerns about the confidentiality of the application material meant that some officers thought ‘on site’ evaluations were preferred over the remote approach. DG RTD use both remote and on-site evaluations but reported that there were no criteria for selecting which parts of the FP use which approach. The majority of the officers consulted felt on balance that remote evaluations were better than on-site. With regard to evaluating the large projects now being submitted it was suggested that a meeting / interview / hearing with each consortium should be requested / offered. This would enable ENER to clarify any points that were not clear in the bid. The point was also made that the position on meetings prior to project submission should be formalised and clarified, as currently it could be interpreted as giving some bids an unfair advantage (to both explain their own projects and seek clarification) via their own contacts. The approach of a formal pre-screening submission and meeting, as used by DG RTD, was suggested as a way of doing this. For the large demonstration projects it should be made clear that there is a preference for the coordinator to be an industrial partner – as they should be best placed to commercialise the results. It was suggested that the ranking of projects at the end of the evaluation process would benefit from a more in-depth process, for example an extended discussion panel including external experts and two technical officers as moderators during the evaluation of large projects. The project participants generally felt that the application and evaluation process was reasonable, if time consuming, with a number of projects employing external assistance. As with the DG Officers there were a number of suggestions as to how this could be improved. There was some support for a 2 stage application process, including from those who had used this in other parts of the FP. There was some concern over the lack of availability of project officers prior to submission, though the fact that this was not a consistently reported issue suggests that the officer comments about a need for clarification of the procedures is genuine.

Remote evaluations offer more flexibility to external staff and can help independence.

Other DGs use this approach.

But there are security risks.

Consortium interviews for the largest projects would be helpful.

A more inclusive end of evaluation meeting would improve the decision making.

Participants generally happy with the application process.


3.3.5 Co-ordination with IEE

One officer felt that there is potential to better coordinate the FP activities of ENER with those pursued under Intelligent Energy Europe3 (IEE). IEE is moving towards calls that have a technology focus which suggests an opportunity to coordinate with the FP (which also has technology focused calls) the softer issue focus of IEE should complement the FP. However, another officer felt that coordination between the two programmes was already in place and working well enough on a sectoral basis, e.g. by particular renewable technology.

3 Intelligent Energy Europe is part of the Competitiveness and Innovation Programme. Its strategic direction is set by DG ENER but it

is administered by an agency staffed by a combination of DG officers and external staff. The programme focuses on the non technical barriers to the take up of energy efficiency and renewable energy, e.g. information failures, skills issues, transposition of directives etc.

IEE coordination needs to be maintained.


4 Effectiveness

4.1.1 Definition

What effects (impacts) have been obtained by the intervention and, in particular have these effects contributed to the achievement of the objectives of the intervention?


The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opinions that we have collected. This area presents some difficulties for a mid term evaluation as it is looking for impacts from projects which have only been up and running for a maximum of 2 years and therefore have not reached a point where impacts are yet apparent. Therefore although we have sought to find evidence of impact we have also considered the issues of how success could and should be measured. What is the impact of the supported activities on the large-scale deployment of energy technologies in the European Union? What is the evidence of structural change as a result of these activities? How can this impact be improved? Our survey and interviews with project participants illustrate that, unsurprisingly, research focussed partners are more hopeful of, and are motivated by research / scientific benefits and outcomes, while participating companies are more hopeful of, and motivated by commercial benefits. Commercial impacts were felt by the participant survey respondents to be the most likely at EU level, with less expectation of local, regional, national or global impact, this is in-line with the level of the FP7 programme, though not necessarily industrial objectives. The project participants were also asked how important the programme was with regard to influencing the policies of the EU, member states and the energy industry, which would be a strong indicator of a move towards structural change. The general view was that its influence is positive but relatively minor. The positive aspects mentioned included generic issues such as raising the profile of R&D. With regard to energy companies, the opinion was that they are now increasing the R&D efforts in this area, but this is largely due to them seeing the commercial necessity and benefits of doing so. Identifying and listing the concrete results of the research funding and assessing positive and negative outcomes. When asked about the expected outcomes from their projects 29% of the participants predicted that patents would be produced. This is higher than the 13% of patents that were produced from FP5 projects. This could represent an over optimism on behalf of the projects.

Few impacts to date from new projects so measurement has been the focus.

Prospects for commercial deployment are ranked more highly by company project partners.

Policy influence (on MS and EU) is positive but relatively minor.

29% of projects expect patents to result from their work.


Special attention should be given to assessing the impact on competitiveness of the European economy and on security of supply, as well as the potential for reduction of CO2 emissions in the energy supply chain. An interesting finding from the participant survey related to the questions posed asking if the beneficiaries could provide an estimate of the CO2 savings their project should enable if successful. The vast majority said they could not do this and neither could they estimate the scale of replication of their project. This reply appears to contradict the high number of respondents who felt that their projects would reduce CO2 in another survey question. We did suspect that the questions was not understood by many respondents however our interviews with project participants suggest that this way of thinking about project impact is not one that participants are happy with. In the consultants view this could relate to the participants wanting to calculate the savings to too high of a degree of accuracy. Where this question in asked in the UK Carbon Trust application forms the intention is to gain an insight into the applicant’s optimism regarding their innovation and their knowledge of the market into which it would be entering, i.e. its approximate scale and the key competitors. The majority of the strategic consultees felt that given the importance of CO2 savings, and the clear link which should exist between demonstrating low carbon technologies and large scale replication, project applicants should be very clearly instructed to estimate the scale and nature of the CO2 savings that their projects could enable. The project participants were concerned with the difficulty of making such assessments and a number were concerned that this approach would favour larger scale nearer market technologies. In order to allay these concerns any additional requirement for CO2 saving estimates should include simple standardised methods and the question should receive no more emphasis in the evaluation procedure than it already does. The other indicators which were most suggested and discussed were patents and citations. The consensus, from both stakeholders and participants, was that although these measures were useful and of some relevance to demonstrating innovation and high quality research they should not be the subject of additional efforts to seek out and measure. The main reasons for this are the ability to manipulate both indicators, the lack of a definite link between either and large scale replication / take up of low carbon technologies and the fact that good projects may well seek to avoid both in order to protect their intellectual property.

4.2 Indicators

Given that the FP7 projects supported by DG ENER have only been running for a maximum of 2 years it is not yet possible to investigate their actual outputs and results. Therefore much of our efforts to answer these questions have centred on the consideration on how project and programme success could and should be defined and measured. The standard approach in evaluations is to translate the programme objectives (from the objective tree) into specific indicators that can be used to assess and measure the success of the programme and the individual projects within it. As with the objective tree we investigated this issue by drafting an initial list of indicators and then discussing this with our expert panel and strategic stakeholders. The findings of this consultation and the resultant list of potential indicators are presented below.

70% of participants were unable to estimate the CO2 benefits of their projects (if successful).

The CO2 question in the application would show the market understanding of the applicant as well as the energy benefits.

Stakeholders support the call for applicants to estimate the CO2 benefits of their projects.

Participants think the CO2 question is very hard to answer.

Patents and citations are interesting indicators but should not receive extra scrutiny.

The objective tree translates into indicators of success.


Table 3.1 Objectives and associated indicators

Programme Level

Objective Group Indicators

Policy Influence Opinions of MS and EU level policy makers Alignment of MS policies with EU policies

Management efficiency and simplification

Processes have been modified in the light of previous evaluations Projects start on time and receive payments on time Survey feedback on satisfaction with processes Reduction in time from agreement to contracting

Measures achieve objectives efficiently

Grants awarded to leading companies, researchers and institutions More SMEs participating than in previous FPs Participation of New Member States Benchmarking with other parts of the FP and international comparators on:

Activities promoted match the objectives set

Work programmes match across to the "essential issues" identified by the EU The activities funded match objectives Project portfolio supports wider policy goals (economic, societal) Projects supported are balanced across the energy sub-themes

Adequate level of funding

Level of applications is neither too high nor too low Funding compares favourably to other RTD themes Funding has increased commensurate with policy priorities Balance of funding between sub-themes has changed in response to emerging priorities.

EU leadership in energy RTD

Consortia are led by and/or include leading global researchers and research organisations Benchmarking with international comparators

- total patents - total actual and potential CO2 savings - total actual and potential RES installations - citations - Industry match funding

Project Level

Objective Group Indicators

Contribution to the energy priorities

RES uptake – actual and potential if research succeeds Carbon savings - actual and potential if research succeeds Larger scale of projects - on the assumption that this addresses fragmentation Exploitation and commercialisation of results – patents (or patentable IPR)

High quality Citations Patents Opinion of external experts

Increasing cross-sectoral impacts of the research

Analysis of projects shows strong cross-sectoral focus Project consortia include industrial players and other end-users

4.3 Measuring progress

The DG officers that were consulted raised a number of interesting points concerning the measurement of project, and programme achievement.


The following measures of success were suggested, with the proviso that they can only be measured some time after the project funding period has passed.

• The speed and scale of technology deployment. • The establishment of European companies as world leaders in their field (as has

happened with wind energy). • Achieving / contributing to the policy objectives, i.e. replication and the CO2 savings

each replication achieves. • Improving the cost effectiveness of the technology, which requires clarity on the cost

reductions achieved, even if only a forecast. • Progress towards market for the technologies supported, this is more complex to

measure and requires an in depth analysis by field. Fundamentally each project should be evaluated on whether it achieved its goals. For example, for a project demonstrating a new plant – did it get the plant up and running? Did it achieve its targets (e.g. % efficiency)? Has it reduced the production cost? Has it led to replicate plants in the EU? However even if some (or all) of these goals have not been achieved the project can still produce useful results, although this would require an in-depth and project specific review. It was generally recognised that these issues are not very well picked up in the project monitoring process and that the applicants can be somewhat weak on specifying these measures of success in their applications. There was some concern that the follow up of projects was lacking and that the focus of attention is much more on launching and running projects and programmes rather than looking for their longer term / ultimate impact. It was suggested that metrics such as an index on related growth or 'build and run' related indexes, would help address this and illustrate programme sustainability. It was suggested that ex-post monitoring of impacts, could be a contractual obligation on the projects but this would imply long contractual periods which would be difficult to enforce once payment (for the bulk of the project) has been given. This is a strong point and one which is picked up in more detail in the sustainability section of the report. The use of patents and citations (resulting from FP funded projects) although interesting and positive, was not felt to be a very strong or suitable indicator of success for the projects funded. There were a number of logical reasons given for this opinion, as follows:

• Spurious innovations can be patented if this is seen as an end in itself. • Citations can also be manipulated. • For the demonstration projects which are now the focus of FP ENER, it would be

expected that much of the innovation being demonstrated is already patented. • Sometimes the innovation in a project is the scale, which is not really a patenting

issue. • Some innovations achieved by programmes cannot be effectively or efficiently

patented. • Increasingly ideas are not patent protected anyway, due to the difficulty of

international patent protection. There were very mixed views among the coordinators on the best metrics to use to measure the impact of the programme. For every participant who suggested that one or other of the proposed metrics (citation index, patents, overall R&D spend, and R&D spend per capita

Impact and success are only apparent some years after funding.

Impact is project specific.

The programme does not do enough to measure impact.

Patents and citations are relevant but open to manipulation and miss many beenfits.


GHG emissions) is a good indicator, three or four other participants cited drawbacks with each one: Picking up the theme of the importance of results, 6 interviewees said that the true impact of the programme can only be measured by the downstream impacts, be that sales volumes, number of production plants constructed, increased turnover of participating companies, number of new start up/spin off companies created, or GWh of energy saved or GHG emission reduction. The DG ENER officers confirmed that at the evaluation stage, they and the evaluators judge how effective a project could be, by looking at what it says it will produce and how this could ultimately address the issues of energy efficiency and CO2 savings. The proposals also have to say how the results are going to be used. The 'impact' assessment criterion rates projects on the potential impact through the development, dissemination and use of their results. This is judged by looking at a combination of the quality and effectiveness of the S/T methodology, the work plan and the appropriateness of measures they propose for disseminating and exploiting their results, including how IPR issues are addressed. In respect of effectively delivering outputs that contribute to programme objectives, the survey responses showed that a high proportion (>70%) of respondents expect to produce a demonstrator, prototype or pilot as a result of their project, but that only 30% expect an exploitable product (patent) to be among the project outputs. There are a variety of reasons why this may be the case, which may be worth exploring further. It will be interesting ex-post, to compare these expectations of results with the actual results reported, as if 30% of the products are able to lodge a patent this will be a significant improvement from the actual results of FP5 where only 13% did this4.

4.3.1 Commercial vs. Research Impacts and Benefits

Reviewing the project participant interviews it is apparent that partners with a research background were more optimistic of achieving, and are arguably more motivated by, the scientific advances that they expected their project to make. As expected the converse of this was that commercial partners are more optimistic of, and motivated by, the potential sales and income that a successful demonstration project can help generate for them. From the survey it also appeared that the programme may potentially be less effective in delivering against the commercial objectives of the EU, than it is against the scientific. Responses from the survey showed that respondents are more positive about achieving significant scientific or technical impact rather than commercial. There was some variation between partners and co-ordinators, with partners more positive in improving their competitive positions and increasing profitability. A similar variation was found between private companies and non-companies, with private firms more confident of significant positive impacts against commercial objectives. SME status appeared to have little or no effect on the respondent’s expectations for commercial gain. Commercial impacts were felt by respondents to be the most likely at EU level, with less expectation of local, regional, national or global impact, this is in-line with the level of the FP7 programme, though not necessarily industrial objectives. 4 Ex Post of NNE Projects Funded by DG TREN under FP5. Ecotec Research and Consulting. 2006

Participants agree that impact is post funding.

Project bid evaluation already considers CO2 savings.

Participants think their projects match programme objectives and 30% expect to patent something as a result.

Research focussed project participants expects research benefits, companies expect commercial benefits.


4.3.2 Indicators of Project Level Impact – Carbon Savings

A number of consultees pointed out that comparable national level energy research schemes across the EU had more tightly defined output indicators for the projects they support. For example the Carbon Trust in the UK requests estimates of potential carbon saving and market replication from the very first application. These estimates are used to judge the theoretical carbon benefits of the proposal – both in terms of the savings of an individual installation and the replication scale. For example a very small improvement to every car has a massive potential, but a major improvement to a very rare industrial process may still be viable but has a much smaller potential. The question on replication potential also provides a clear insight into the applicant’s knowledge of the market they expect their innovation to enter into. If this knowledge is lacking it is a good indicator of an applicant not well suited to commercialising a product. However, the Carbon Trust consultee stated that they could see that some other FP objectives would be more difficult to measure, for example, if quality of research is measured by citations in peer reviewed journals, this will favour a certain type of project and not one which necessarily saves the most carbon. With regard to asking projects to estimate the carbon savings their project should result in if successful, there was general acceptance of this idea, though with a number of qualifications. It was pointed out that research can be complex and the exploitation of the results varies widely and usually occurs most intensely after a project has finished (in terms of FP funding). So if the Commission asked for information on expected carbon savings post exploitation they would need to be careful that they were not swayed by over-ambitious claims - that they would not check up on anyway as they would occur after the FP funding period. The majority of those questioned thought that the CO2 savings question is important as it quantifies the impacts and clearly links to the priority target for ENER – 20/20/20. Even though it is not necessarily an easy question to answer, applicants should be obliged to attempt to do so, as long as the method to do this is clearly explained, a rational view taken on any claims which appear extravagant, and it was not used as the only criteria for judging applications. On a more fundamental level one consultee pointed out that the ultimate carbon savings that a project achieves are very difficult to isolate as they are driven by such a complex, and member state specific set of drivers. There are many clear examples of this, such as the high concentration of concentrated solar power (CSP) plants in Spain, where there are significant subsidies available to support the technology, compared to Italy where the solar conditions are very similar but no such subsidy exists and therefore no CSP plants have been installed. The guidance documents on completing the application form ask about CO2 savings (although there is no explicit question in the form) and this is an important factor in assessing bids. Replication potential is also raised as relevant but it is recognised that this a more complex issue than CO2 as it requires a commercial understanding of EU market. Some officers also thought that estimating CO2 savings was also too difficult for applicants to estimate. These markets vary a lot by technology, e.g. retrofit vs. new and by MS and even region. For example with biomass the replication potential depends on resource availability – this varies widely and at a very local level – and is therefore very hard to know for the whole of the EU. Estimation of replication also requires some prediction of the future, which is complex, though this could be improved, for example a view on future fossil fuel prices is key to estimating replication potential for the next 10-15 years.

Potential CO2 savings are requested in applications to a UK programme.

These show market awareness of the applicant, which is key to commercialising a demonstration project.

Stakeholders agree with the benefits of asking for CO2 savings. But don’t over rely on the accuracy.

Its hard to see in advance the long term large CO2 benefits from come from some projects.

Application guidance asks about CO2 but there is no specific question on the form.

Complex MS market variations make the CO2 savings estimate difficult.


It was also recognised that the ease and speed with which a low carbon technology becomes commercial is strongly influenced by factors other than the technology, i.e. financial incentives. This reflects the fact that at the present time most technologies targeted by the programme are still not competitive with conventional (fossil) solutions, though some are getting close (e.g. wind). The length of time that such financial incentives will be required is very closely linked to the fossil fuel price. However the support provided by the FP will improve the technologies and make them more competitive with fossil fuels. A number of the officers described it as a balance to strike between supporting the ideas which are very close to commercial and those which are higher risk. If projects are too close to market the support becomes subsidising what the companies should do themselves, if too risky, the implementation may be difficult to achieve and too few of the projects succeed.

4.3.3 Programme Level - Strategic Added Value

One consultee raised the point that an extremely valuable programme level benefit was the positive strategic and policy influence. She voiced the opinion that even relatively recently in 2006/7 when FP7 started, the implied logic (of the overriding importance of low carbon energy) was not generally accepted by the energy industry or at MS level. Some MSs had adopted this policy position and were actively pursuing it while others (probably the majority) were not pursuing it with as much interest. This position was evidenced by the very low percentage of energy related R+D expenditure that was targeted on clean / low carbon technologies, 3-5% of the total. This figure has rapidly increased over a very short period of time to the point where it now the main focus of research expenditure. The consultee felt that FP7 was an important stimulus in increasing this percentage, even though in comparative terms the funding amounts involved in the FP are very low. The FP also had an important influence in helping move clean energy up the political agenda, its objectives are now the generally accepted approach, with all MSs sharing the same objectives. This reflects the rapid policy development in this area as climate change has been widely accepted and firm policies and goals have been put in place to start to address it. Therefore in considering the impact of FP7 it is important to look at both direct investment in FP7 projects and the influence it has had on the policies and spending of MSs and the energy industry. The project participants were asked how important the programme was with regard to influencing the policies of the EU, member states and the energy industry. The general view was that its influence is positive but relatively minor. The positive aspects mentioned included generic issues such as raising the profile of R&D. With regard to energy companies, as has been mentioned previously, they are now increasing the R&D efforts in this area, but this is largely due to them seeing the commercial necessity and benefits of doing so rather than any influence from the FP. The point was also raised that the presentation of the results was key to their ultimate influence. Where results are presented as guidance there will often be a wide variance in how MSs interpret this. One coordinator felt that “the policy influencing role is critical and that the Commission is absolutely driven by it”. He could see the policy role and targets (2020) as very important and in some ways as a 'weight on the shoulders' of commission people. This could be a downside in creating too much pressure and pushing too fast. He also felt that the policy influencing role on MSs was important as some states are lagging behind and that the influence of FP7 is good in helping them to catch up.

CO2 savings are closely linked to markets and subsidies etc..

FP has a positive policy influence on EU, MS and companies.

.Low carbon energy r+d has increased over the last 3-5 years.

The FP has had a role in this increase but the scale is hard to know.

Well presented FP projects can be very helpful in illustrating policy goals.


One participant felt that the increased scale of projects was a positive factor in terms of their ability to influence policies. A new MS participant, from an energy utility company, reported that his company had been actively encouraged to take part in the FP by their government.

Larger projects with higher profile help in terms of policy influence.


5 Consistency

5.1.1 Definition

The extent to which positive / negative spill over into other social, economic or environmental policy areas are evolving.


The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opinions that we have collected. How can the impact and added value of collaborative research that cuts across scientific disciplines, industrial sectors and policy fields be further enhanced with a view to better address large societal challenges? The survey highlighted that participants are aware of the benefits that they should receive regarding boosting and protecting their research capacity but a low percentage considered their project would bring benefits in terms of equality in science. Sexual equality was a low priority for projects, though it was agreed that this issue was appropriate for consideration and could be more systematically measured and recorded, for example measuring how many women actually worked on a project when completed. The issues of energy cost, and the potential increases that particularly renewable energy can bring, were an issue that was picked up as missing from the objective tree and it does appear that this issue is not actively considered in many projects. However it was pointed out that the purpose of the funding is to improve the cost effectiveness of the technologies and that energy prices and charging mechanisms are an issue beyond the control of the FP, and the EU. The project participants were asked to comment on any spillovers from their projects into wider socio-economic policy areas. In general this revealed a number of positive and interesting benefits from projects, but as with the other benefits and impacts many of these were felt to only become apparent some time after the projects have completed. While the participants recognised these benefits they were generally felt to be tangential to the main purpose of the projects and as such it was not felt that they should receive significant additional monitoring. Many of the described benefits could be seen to clearly related to (i.e. stem from) the standard outputs expected from a project. Some of those questioned agreed that more could be done in this area by projects, for example educational benefits from projects going into schools etc. However, such activity would divert the focus and resources of the project team, and is not their core competence, and should therefore not be something

Aware of equality issues but a low contribution expected.

Energy costs should reduce via project success but its beyond the scope of the FP to influence these.

Positive spill over is expected into skills, health, env, etc, but these should not be subjected to additional monitoring efforts during project operation.


that project partners do. If such benefits are required it would be better pursued via another programme

5.2 Project Survey Findings

In respect of contributing to wider EU policy objectives over 50% of those surveyed expected their project to have a significant impact. This is particularly the case in respect of quality of life and health and safety. Impacts on employment are lower but still over half expect significant employment impacts. Spillover into wider EU science and technology benefits was strongly expected by respondents, with over 75% expecting impacts of medium or high significance on policy objectives in this area from their project. The one exception to this was gender balance in science with a much more mixed view of the impact of their projects. The survey respondents also thought that their projects would have significant impacts on wider environmental objectives, this was particularly felt to be the case in the areas of GHG emissions reduction, energy efficiency and increasing the share of renewable energy.

5.3 Project Coordinator and Participant Interviews

The project participants were asked to comment on any spillovers from their projects into wider socio-economic policy areas. In general, this revealed a number of positive and interesting benefits from projects, but, as with the other benefits and impacts, many of these were felt to only become apparent some time after the projects have completed. While the participants recognised these benefits, they were generally felt to be tangential to the main purpose of the projects and as such it was not felt that they should receive significant additional monitoring. Reflecting this opinion one coordinator stated that projects “need to guard against unrealistic expectations of the impact on society as any social impact may take a long time to emerge”. He also suggested that “In order to capture evidence of wider impacts perhaps the Commission could produce a set of case studies which provide concrete examples”. Some of the positive socio-economic spill over described included the following:

• Creating jobs • Improving skills – in research, manufacturing and operation and maintenance. • Improving health (via improved air quality from reduced use of fossil fuels.) • Influencing the pattern of economic development (e.g. away from crowded urban

areas and creating more activity in rural areas). • Economic benefits – captured by industrial revenue from downstream commercial

sales. • Environmental benefits – correlated to the CO2 savings. • Technology transfer to new countries. • Informal network building

Surveyed project participants expect positive spill overs from their work.

Participant interviews confirm that positive spill overs are expected.

.. e.g. jobs, skills, health, income and environment.


As can be seen, many of these benefits are clearly related to (i.e. stem from) the standard outputs expected from a project. Some of those questioned agreed that more could be done in this area by projects, for example educational benefits from projects going into schools etc. However such activity would divert the focus and resources of the project team, and is not their core competence, and should therefore not be something that project partners do. If such benefits are required it would be better pursued via another programme The issue of sexual equality in projects was raised and it was agreed that this issue was appropriate for consideration and could be more systematically measured and recorded, for example measuring how many women actually worked on a project when completed. Some examples of good practice were offered by projects, including a high level of interest from girls when carrying out science promotional work in schools and a growing renewable energy company (ABNT) where over 60% of the workforce is female. Although one coordinator felt it was not important to promote equality within the programme and other mechanisms such as promotion of science at a young age, promotion of programme results as subject in the secondary schools, etc. were more appropriate.

These benefits could be increased but this risks tangential activity.

Sexual equality is recognised as relevant but its hard to address in a male dominated sector More guidance on best practice would help.


6 Allocative / Distributional

6.1.1 Definition

The extent to which disproportionate negative / positive distributional effects of policies are evolving.


The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opinions that we have collected. Some of the issues of balance of funding are also discussed under relevance. Measuring the coverage of research subjects relevant to objectives of FP7 and identification of gaps. The analysis of programme data has revealed some interesting issues, these include:

• The vast majority (89%) of the programme budget to date has been spent in the first 4 calls. The 11% that has not been spent is mainly due to a reduction in funding during negotiation to a number of projects. It is assumed that this can roll forwards into the remaining calls.

• The number of applicants to each call has dropped significantly over the 4 years of operation, from 231 in the first call to 43 in the third and 45 in the fourth, This relates to a major reduction in the number of areas open for bidding.

• The chance of a bid being supported has increased from 11% to 33% or 1 in 3. • Over the 4 calls the average project value has increased from €6.1 million to €17.6

million. • The industrial match funding has increased over time.

The programme analysis indicates that there are more renewable energy focused projects than energy efficiency. In part this is thought to relate to the better fit of renewable energy to large demonstration projects than is the case for energy efficiency projects. It was also pointed out that energy efficiency is picked up and supported by other parts of the FP. An important reason for the relative lack of energy efficiency projects was also felt to be a failure for market mechanisms that support deployment to develop in the same way as they have for renewable energy. As far as country participation is concerned, providing figures for each member state and associated countries, identifying those which are less involved and factors to explain why, together with recommendations to implement in order to eventually bring them aboard.

More RES than EE projects are funded but the split is complex.


The survey illustrated that some 55% of the participants surveyed have taken part in FP projects before, though some 38% were entirely new to it. This illustrates that the programme is still attracting a healthy amount of new entrants, from what is a relatively small target group, although the assumption that past participants have a higher success rate appears to remain true. Germany and Spain have received the largest amount of funding to date with the members of the EU15 taking 95.1% of the funding to date. Measuring the proportion of SMEs participation and their role, providing with findings on the specificities of SMEs in the sector of energy technologies, including in terms of competitiveness and employment The issues of SME and new Member State participation are much discussed in evaluations of this nature. Although the smaller number of larger projects being supported means that the number of SMEs participating is reduced, the rate is reported as 19-20% which is higher than the FP target of 15% and higher than FP7 average level of 13%. The generic barriers of attracting SMEs to participate in the programme are well known and many were repeated in the consultations. These include the conflict which often exists between the desire to see quick and commercial results that SMEs have in comparison to the longer time scales that are typical for FP projects and amongst research partners. However it was pointed out that SMEs can often bring a high level of innovation to projects, particularly in clean energy where many of the technologies are, by industrial standards, new. There were a number of suggestions made by project coordinators on how to encourage SMEs to participate in the FP. Over half of the ten coordinators interviewed though that some enhanced level of funding support for SMEs would help attract them. The suggestions included higher support rates or an increased share of the project budget. Two coordinators thought this could be done by limiting the number of participants, perhaps via a specific stream of funding. Such an SME orientated strand could also offer a streamlined approach, with less onerous requirements and quicker, even pre, payments. It was suggested that a reduction in the administrative efforts required in the application process would make the FP more attractive to SMEs, but it was accepted that there would always need to be some minimum threshold of information required. In conclusion, the majority of project coordinators and participants interviewed agreed with others stakeholders and felt that the focus must be on getting the best team in order to achieve the highest standards of R&D, irrespective of quotas for SMEs/MSs/gender etc and that one should not include a SME just to satisfy a quota requirement. Similar points were made regarding new Member State participation, with the additional point that their applicants have not had the opportunity to build up the capacity and experience over numerous FPs that participants from the old MSs have.

Over 1/3 of participants are new to the FP.

Old MSs dominate funding >95%.

SME participation rate is high.

SMEs are useful but there is no desire for a quota.

NMs have a low representation, but this is hard to address.


6.2 Programme Data Analysis

The FP7 Energy programme has already evolved since 2007, with the SET plan one of the major factors in its evolution in this period. This evolution is evident through the number of topics called each year as shown in the table below. In each year the number of topics called has declined, from the 2007 call with 45 topic areas open to the 2010 call with only 8, as the programme has moved towards having a tighter strategic research focus. This move also appears to have contributed, by the 2010 call, to an increase in the proportion of called topics with at least one project funded from 42% to full coverage of 100%, and a consequent decrease and elimination in 2010 of the number of topics called where no projects were funded. Table 6.1 Analysis of topics called and funded

Call > 2007-TREN-2

2008-TREN-1

2009-ENERGY-2*

2010 ENERGY-2* Total

Topic Areas Open 45 18 10 8

Project proposals evaluated 231 130 43 45 449

Projects approved for funding 26 22 10 15 73

No of call topics with at least 1 project funded 19 8 7 8 42

As % of total called 42% 44% 70% 100% 51.9%

No of call topics with no projects funded 26 10 3 0 39

As % of total called 58% 56% 30% 0% 48.1%

No of projects funded in non-call topics 2 2 0 0 4

* 2009 and 2010 figures are provisional as these projects remain at the negotiation stage

6.2.1 Total Allocations

Our analysis of the 73 projects already approved (or in negotiations) for funding from the first 4 calls of FP7 Energy shows the following: Total EU Funding of €446.3 million has been allocated across 73 projects in the first 4 calls. This represents an allocation of 89% of the €501.4 million budget. Within this there was variation in funding allocation with the 2007 call allocating less than 70% of its budget while the 2008 and 2009 calls allocated over 95%, and 2010 over 100% of their budgeted allocations. The main reasons at this stage appear to be a reduction in funding allocated to 6 projects funded under the 2007 call, for each project these cuts represented at least 50% of the original budget, 1 project (2ndVEGOIL) was also funded under the 2007 budget but is now allocated to DG MOVE and therefore outside the scope of this evaluation. It is unclear if this process will be repeated in future calls as projects progress, funding has so far been reduced on one project (EMPYRO) from the 2008 call.

Move to more focussed calls in fewer areas with larger projects.

89% of the budget to date has been spent.

Much of this relates to projects not proceeding post negotiation.


The number of applications submitted has gone down considerably from 231 in the first call to 45 in the fourth. This is thought to mainly be a result of the significantly reduced number of areas ‘open’ for bids and requests for larger demonstration projects in the second, third and fourth calls. This reduction in the number of applicants has led to a clear increase in the chances of a submitted bid being approved, with the success rate increasing from 11% to 33% or around 1 in 3. The funding the EU has provided has been supported by around €378 million of investment by the project partners, so that total investment across the 74 projects totals over €824 million. On average the EU investment has secured matching commitments from project participants equal to around 85% of the EU funds invested. Alternatively it can be said that EU funding represents around 54% of the total value of the funded projects. Finally the average project value has increased through each project call as budgets remain of similar size but fewer projects are funded, over the 4 calls the average project value has increased from €6.1 million to €17.6 million. Similarly the total value of the largest funded project in each call has grown significantly. Table 6.2 Analysis of project funding

Call > 2007-

TREN-2 2008-

TREN-1 2009-

ENERGY-2* 2010

ENERGY-2* Total

Budget (€ mio) 128 147 100 126.4 501.4

Projects approved for funding 26 22 10 15 73

Success % (applications / approved)

11.3% 16.9% 25.6% 33.3% 16.5%

EU funding of approved projects

86.7 142.5 85.8 131.4 446.3

% of budget allocated 67.7% 96.9% 85.8% 104.0% 89.0%

Other funding of approved projects

72.1 107.3 66.3 132.0 377.7

Total value of approved projects

158.8 249.8 152.1 263.4 824.0

Average project value (€m) 6.1 11.4 15.2 17.6 8.9

Highest Project Value 28.3 21.7 58.0 71.6 *2009 and 2010 figures are provisional as these projects remain at the negotiation stage

DG ENER also part financed joint FP calls with other DGs. To date these have been:

• FP7-ENERGY-2009-BIOREFINERY 15 MEUR

• FP7-2010-NMP-ENV-ENERGY-ICT-EeB 15 MEUR - Energy Efficient Buildings

• FP7-OCEAN-2010 1MEUR - Ocean (horizontal action call falling under activity 10)

The projects partly supported by DG ENER under the joint calls have been excluded from the other analysis to avoid fractions of projects and participants that would not help analysis.

No. of applications has gone down significantly.

Average project value has almost tripled and industry match has increased.

Some very large projects are now being supported.

DG ENER have contributed some €31m to other FP calls in combination with other DGs.


6.2.2 Allocation by Activity

In the first 4 calls funding has been allocated across the 8 activity areas as listed in table 6.3. Only 72 projects are listed here as 1 project, SET-Plan_se2009.eu, was an ad-hoc project that funded the SET Plan and fell outside the activity categories. Table 6.3 Activity areas funded in first 3 FP7 Energy calls

Activity Code

Description Projects Funded

Total Project Value (€m)

% of Total Value

2 Renewable Energy Generation 21 253.4 30.8%

3 Renewable Fuel Production (excluding contribution to bio refineries)

12 144.3 17.5%

4 Renewables for Heating and Cooling 8 44.2 5.4%

5 CO2 capture and storage technologies for zero emission power generation

0 0.0 0.0%

5+6 Cross cutting technologies 5 & 6 3 12.5 1.5%

6 Clean Coal Technologies 6 64.1 7.8%

7 Smart Energy Networks 9 185.1 22.5%

8 Energy Efficiency and Savings 12 118.2 14.3%

9 Knowledge for Policy making 1 2.2 0.3%

10 Horizontal Actions 0 0.0 0.0%

Total 72 823.9 100.0%

Figure 4.1 shows the proportions of funding allocated to each activity area. This clearly shows that activities 2, 3 and 4, that cover the renewable energy categories, account for over half (~54%) of all projects funded, while work on cross cutting technologies and clean coal accounts for around 9% of projects funded, and smart energy networks and energy efficiency savings account for just over 1/3 of funding (~37%). This breakdown highlights a clear commitment to renewable energy and energy efficiency technologies which is generally consistent with the FP7 objectives and priority areas. Figure 4.2 shows the amount funded by activity area for each of the first 4 calls. There are a few key changes evident between the calls. Funding for activity 2, renewable energy generation has remained relatively stable through the calls but substantial funding for activity 3, renewable fuel production, in 2007 and 2008 was reduced to zero in 2009 and returned again in 2010. Cross cutting activities were funded in 2007 and 2010 but not 2008 or 2009, funding of clean coal activities was absent in 2007 but present all subsequent years. Funding for smart energy networks and energy efficiency activities was a substantial part of each of the calls but the balance between the 2 elements has swung with each call, from fairly even in 2007, to primarily energy efficiency in 2008, to primarily smart energy networks in 2009 and to only smart energy networks in 2010.

Appears that RES gets more support than EE.

But the split is not simple, e.g. smart networks are relevant to both.


Figure 6.1 Funding by Activity Area – Total of 2007, 2008 & 2009 calls

FP7 Energy Funding Allocated by Activity Type - Total First 4 calls

Renewable Energy

Generation31%

Renewable Fuel Production

18%

Smart Energy Networks

22%

Energy Efficiency and

Savings14%

Cross cutting technologies

2%

Renewables for Heating and

Cooling5%

Clean Coal Technologies

8%

Knowledge for Policy making

0.3%

Figure6.2 Funding by Activity Area – By Call (EU Contribution only)

0

20

40

60

80

100

120

140

160

2007-TREN-2 2008-TREN-1 2009-ENERGY-2 2010 ENERGY-2

Call (Year)

Euro

s (M

illio

ns)

Know ledge forPolicy making

Energy Eff iciencyand Savings

Smart EnergyNetw orks

Clean CoalTechnologies

Cross cuttingtechnologies

Renew ables forHeating andCooling

Renew able FuelProduction

Renew ableEnergyGeneration


6.2.3 Specific Activity Area

The 73 projects funded under the first 4 calls have focused on a number of the 38 specific activity areas. Table 4.4 presents the number of projects and total value of projects at this level. Within Renewable Energy Generation the largest share (30%) goes to cross-cutting issues, this encompasses 2 projects approved in the 2010 call, one is a solar power – desalination demonstrator and the other a solar CSP plant with CHP, mixed fuels and fresh water applications. These cross-cutting projects aside Photovoltaics, Biomass and Ocean Energy take the next largest shares, each over 15% of the total EU contribution to project value, and also each representing around 5% of total FP7 Energy project value. Within activity 3 Renewable Fuel Production, projects in Second generation biofuels from biomass dominate the project value, with around 2/3 of all EU funded value in this activity. This represents 6 projects worth €51 million, the second largest funding of a single area across all FP7 Energy areas, representing over 11% of all DG Energy FP7 Energy project funding. Within activity 4, Renewables for Heating and Cooling, funding is split relatively evenly across 4 of the activity areas. As noted previously, there is no funding for CO2 storage under activity 5 but there is some funding in these areas under the cross-cutting activity 5+6. Within activity 6, Clean Coal generation, the primary area of funding has been conversion technologies for zero emission power generation, with €41 million allocated over 6 projects. Under activity 7, Smart Energy Networks, projects are split across all 3 areas but funding is heavily skewed to the Development of interactive distribution energy networks, this area representing the single largest funded value of any activity area at €83.4m, representing almost 19% of all FP7 funding to date. Within activity 8, Energy Efficiency and Savings, there are a number of projects in areas 8.1, 8.2 and 8.4, with the largest funding, of over €37m, going to 8.4 to support the integration of renewable energy and energy efficiency under the CONCERTO initiative. Table 4.4 Specific funding activity areas in first 3 FP7 Energy calls

Activity Code Description Area

Code Description Projects Funded

Total EU

Funding €

m

% of total

in activity

code

% of

whole

2.1 Photovoltaics 5 28.6 21.8% 6.4%

2.2 Biomass 4 22.6 17.2% 5.1%

2.3 Wind 5 14.1 10.7% 3.2%

2.4 Geothermal 0 0.0 0.0% 0.0%

2.5 Concentrated solar power 1 6.0 4.6% 1.3%

2.6 Ocean energy 4 20.7 15.7% 4.6%

2.7 Hydro-Electric 0 0.0 0.0% 0.0%

2 Renewable Energy Generation

2.9 Cross-cutting issues 2 39.4 30.0% 8.8%

Split within themes is complex – with many projects being cross cutting.

Clean coal covers carbon capture and storage (CCS).




Total EU

Funding €

m

% of total

in activity

code

% of

whole

3.1 First generation biofuel from biomass 1 5.0 6.5% 1.1%

3.2 Second generation biofuels from biomass 6 51.0 66.8% 11.4%

3.3 Biorefineries (Part supported by ENER) 0 0.0 0.0% 0.0%

3.4 New energy crops 3 19.2 25.1% 4.3%

3.5 Alternative Routes to Renewable Fuel Production 0 0.0 0.0% 0.0%

3.6 biofuel use in transport 0 0.0 0.0% 0.0%

3 Renewable Fuel Production

3.7 cross-cutting issues 2 1.2 1.5% 0.3%

4.1 Low/medium temperature solar thermal energy 3 6.1 24.8% 1.4%

4.2 Biomass 2 7.0 28.5% 1.6%

4.3 Geothermal energy 1 4.3 17.5% 1.0%

4.4 Cross-cutting issues 0 0.0 0.0% 0.0%

4 Renewables for Heating and Cooling

4.5 Cross cutting issues 2 7.1 29.1% 1.6%

5.1 CO2 Capture 0 0.0 - 0.0%

5

CO2 capture and storage technologies for zero emission power generation 5.2 CO2 Storage 0 0.0 - 0.0%

5+6.1 Power generation technologies for integrated zero emission solutions

0 0.0 0.0% 0.0% 5+6 Cross cutting

technologies 5+6.2 Cross cutting and regulatory

issues 3 8.6 100.0% 1.9%

6.1 Conversion technologies for zero emission power generation

6 41.0 100.0% 9.2% 6 Clean Coal

Technologies 6.2 Coal-based polygeneration 0 0.0 0.0% 0.0%

7.1 Development of inter-active distribution energy networks 4 83.4 84.8% 18.7%

7.2 Pan european energy networks 2 4.9 5.0% 1.1% 7 Smart Energy Networks

7.3 Cross cutting issues and technologies 3 10.0 10.2% 2.2%

8.1 Efficient energy use in the manufacturing industry 3 10.2 16.0% 2.3%

8.2 High efficiency polygeneration 4 14.7 23.1% 3.3%

8.4

Innovative integration of renewable energy supply and energy efficiency in large communities: CONCERTO

4 37.8 59.4% 8.5%

8.5 Innovative strategies for clean urban transport: Civitas-plus 0 0.0 0.0% 0.0%

8 Energy Efficiency and Savings

8.6 socio economic research and innovation 0 0.0 0.0% 0.0%




Total EU

Funding €

m

% of total

in activity

code

% of

whole

8.7 thematic promotion and dissemination 1 1.0 1.6% 0.2%

9.1 Knowledge Tools for Energy-related Policy Making 0 0.0 0.0% 0.0%

9 Knowledge for Policy making

9.2 Scientific Support to Policy 1 2.2 100.0% 0.5%

10 Horizontal Actions 10 Horizontal Actions 0 0.0 - 0.0%

TOTAL 72 446.2 100%

6.2.4 Geographic Spread

It was made clear in the inception phase of this evaluation that a balanced distribution of project funding between individual EU member states was not now regarded as a primary objective of the FP7 Energy programme. As discussed in more detail in section 2 the objectives of the programme are to fund high quality research, in areas of strategic need which have the potential to have significant short and / or long term impacts, regardless of location. At the same time the organisation and spread of funds between member states, associate countries and 3rd countries is relevant to FP7 objectives and commitments to international science and technology co-operation. Encouraging participation from newer member states is also a desirable outcome. Analysis of funding provided in the first 4 calls shows that over 90% of funds are allocated to beneficiaries in EU member states. Of the remainder around 7.7% is allocated to beneficiaries in associated countries5 and 1.9% is allocated to beneficiaries in 3rd countries. Of the funding allocated to beneficiaries in EU member states, approximately 95% has been allocated to beneficiaries located in EU-15 countries, and only 4.9% to those in the new member states (see table 3.5). This balance varied from 91% / 9% in the 2008 call to 98% / 2% in the 2010 call. This balance is broadly similar to the GDP split between the EU-15 and non EU-15 member states which is around 93% / 7%, but not the population balance which is closer to 80% / 20%. It should be noted that 2 of the non EU-15 member states (Latvia and Malta) have not yet received any funding under the FP7 programme. Table 4.5 Funding by Geographic Area – Total first 3 calls

Beneficiary group FP7 funded Value (€m)* % Member state

group FP7 Funded Value (€m)* %

Funding to EU-15

388.4 95.1% Funding to EU Member States 408.3 90.4%

Funding to non EU-15 19.9 4.9%

Funding to FP7 Associate Countries 34.8 7.7%

5 Associated countries are listed in ftp://ftp.cordis.europa.eu/pub/fp7/docs/third_country_agreements_en.pdf

Vast majority of funds go to the EU.

95% of funds to the EU15, matches GDP mix but not population.


Funding to 3rd Countries 8.4 1.9%

*- based on participant grant request in 2010 call – it is likely that some participant requests will be reduced – this is unlikely to materially affect these proportions.

Figure 4.4 shows the total EU funding by beneficiary country for the first 3 calls of FP7 Energy. This shows that by value, beneficiaries in Germany and Spain received the largest funding, receiving over €65 million in total. Other major national concentrations of beneficiaries include Denmark, Italy, the Netherlands, France, the UK and Belgium, with beneficiaries receiving more than €25 million in each country over the 4 calls.

Figure 4.4 EU Funding by Country of Beneficiary – Total first 3 calls

0 10 20 30 40 50 60 70

BrazilKazakhstan

ChileThailand

South AfricaAustralia

AlgeriaTunisiaEstoniaCyprus

MacedoniaSerbia

VietnamIndia

BulgariaUkraine

ChinaSlovenia

RussiaSlovakiaRomania

IsraelCroatia

LuxembourgTurkey

Czech republicUSA

EgyptPolandIreland

LithuaniaHungaryPortugalSw edenAustria

Norw aySw itzerland

GreeceFinlandBelgium

UKFrance

NetherlandsItaly

DenmarkSpain

Germany

Cou

ntry

of B

enef

icia

ry

Euros (millions)

.Germany and Spain are the biggest recipients – by a large degree.


6.2.5 Network Analysis

Annexe 9.8 of this report presents a detailed network analysis of the project participants (based on the first 3 calls of FP7) and their interlinkages for both FP7 and FP6. There are a number of interesting findings from this analysis. As the diagram below indicates the EU 12 remain the most networked of the participating countries.

Figure: Collaboration visualization on country level (FP7, all links)

6.2.6 SME Participation

The issue of SME participation, and associated programme level indicators raised a number of interesting points. The objective of stimulating innovation capacity within energy SMEs was well known and accepted as justified. It was also pointed out that SMEs often have some of the most innovative ideas in what are, by industrial standards, very new technologies. As such their inclusion within the programme is valuable. However it was also pointed out that it is a difficult balance to strike between including SMEs and aiming to achieve world class research. The same point was made with regard to including partners from new member states. Although there are many exceptions to this apparently elitist view, i.e. world class research projects are very much possible with SME and new member state partners, any approach which forces projects to meet some ‘quota’ of SMEs or new member state participation was felt to run the risk of imposing extra costs and reducing the effectiveness of a project. The suggestion was made that the best way of solely supporting and expanding SME and new member state research capacity is via a dedicated strand or separate programme.

Old MSs are best networked in FP projects.

SMEs have positive attributes to bring to projects.

But quotas are seen as counter productive.


SME participation rates in the DG ENER part of FP7 are actually higher than typical for the FP as a whole and above the approximate target. Despite the perception that larger projects would be off putting to SMEs, their participation rate is relatively high, at 19-20% against a Commission target of 15% and a FP7 average level of 13%. Though the number of SME participants will have gone down, as for all types of participants as the number of projects has reduced. The route for SME inclusion is believed to typically be to provide a particular component or expertise within a consortia led by a large company or other organisation. This was contrasted with the position in DG Research where the perception is that SMEs are crowded out by research providers. It was also reported by DG officers that the inclusion of SMEs in a project application was viewed as a positive addition when assessing bids. It was also pointed out that other SMEs take part as sub contracting third parties to non SME participants. The point was raised that the demonstration projects now favoured by DG ENER need large industrial partners, particularly in the lead. The smaller organisations still have an important role – but typically not as the leader, e.g. SMEs to do specific components and RTD organisations measure technical outputs. It is also hard for smaller organisations to bring the scale of funds required to match fund the large demonstration projects now being pursued. A number of officers felt it was difficult for SMEs to be a project coordinator as in comparison to large organisations they will, by definition, have less administrative resources. The issue of SME inclusion raised a number of comments from the project coordinators. One UK coordinator felt that in the UK, SMEs are not coming forward to be involved in projects and there is no encouragement for them to target EU funds. He felt that this was leading to good ideas not being explored, with part of the problem being a perception among SMEs that the process of obtaining funding is too complicated or the chances of being successful are too slim. However the majority of the project coordinators thought that SME participation in projects could bring benefits to the SMEs, the projects and the economy of the EU. One SME participant felt that SME participation can help compensate for the domination by larger firms in some technologies. He also strongly felt that the discipline required to receive FP7 funding was a major benefit to SMEs in getting them to improve their systems to 'big firm' standards – this was particularly useful to document processes and to ensure that all useful knowledge generated by the project was captured. He felt that the Commission provided excellent support to SMEs to do this. Another SME participant felt that there were no particular barriers to their participation in FP7 (apart from the frustration over extended timescales). Another project coordinator stated that they had 2 SMEs in their project and that he felt they were highly competent and it was beneficial to project to have them involved. A participant from a research organisation felt that opportunities for SMEs to participate are available if an SME wishes to take the initiative. There were a number of suggestions made on how to encourage SMEs to participate in the FP. Over half of the ten coordinators interviewed though that some enhanced level of funding support for SMEs would help attract them. The suggestions included higher support rates or an increased share of the project budget. Two coordinators thought this could be done by limiting the number of participants, perhaps via a specific stream of funding. Such an SME orientated strand could also offer a streamlined approach, with less onerous requirements and quicker, even pre, payments.

SME participation rates are high.

SMEs are not best suited to the coordination role.

Participants are happy to include SMEs and feel the opportunity is these for those who see k it out.

Additional funding and networking might help attract SMEs.


It was suggested that a reduction in the administrative efforts required in the application process would make the FP more attractive to SMEs, but it was accepted that there would always need to be some minimum threshold of information required. A two stage application process was suggested as a good idea for SMEs as it gives them the opportunity to test an application with less risk. Another idea for attracting SMEs was to stage networking events where SMEs could meet FP7 partners. One participant felt that if the Commission asked for a project to reduce its number of partners then SMEs are often the easiest partners to cut. In conclusion the majority of project coordinators and participants interviewed felt that the focus must be on getting the best team in order to achieve the highest standards of R&D, irrespective of quotas for SMEs/MSs/gender etc and that one should not include a SME just to satisfy a quota requirement.

6.2.7 New Member State Participation

A programme level indicator which is well known and important for political reasons is the participation rate by new member states (NMSs). The political importance of this indicator was accepted and it was pointed out that there are some strong competencies in energy research in the new member states which should be supported. The general consensus was that NMSs do participate less than would be ideal. It was felt that this could partly be explained by the relatively small size of their network of potential FP collaborators, in comparison to the old MS applicants. Their (NMSs) lack of ability to provide the required match funding was also thought to be an issue, as was their relative lack of experience / capacity in the administrative requirements of bidding and running a project. Setting any type of formal target or quota for NMSs was not regarded as a good idea as the risk of their tokenistic inclusion was too large. Where they are included in projects they need to be relevant and performing a task which needs their skills, though it was agreed that it can be difficult to attract NMS partners without incentives such as a minimum budget. However, it was pointed out that there are areas of strengths for the new MSs which should be identified and exploited, and that there are also clear benefits to a consortium in terms of cost, e.g. the same component/expertise can be sourced cheaper from a new than old MS. This is also a benefit to EU competitiveness and consortium efficiency in up-skilling and including these partners. The project coordinators and participants were asked their opinion on the participation rates from new member states (MS). As with SME participation, all of those questioned felt that there was merit in including and encouraging partners from new MS. The lack of experience that new MS partners have in preparing applications and forming consortia was recognised as a negative factor for them. One coordinator with two new MS partners in his project reported that they were very good contributors and had been brought in for specific expertise and unique competencies to add to their project. A new MS participant stated that SMEs from NMSs were particularly hard to attract.

6.2.8 Gender Balance

Including women in projects was recognised as a legitimate policy goal. Officers felt that applicants are aware of this and complete the relevant gender aspects section in the proposal and related action plan. However it is recognised that the applications tend to include

Some SMEs put of by the administrative burden.

NMS participation can give benefits but they lack the experience and network of old MS participants.

Gender balance is recognised as important but hard to address.


standard responses and monitoring data, and therefore that there was some potential for improvement in this aspect. The suggestions made by the officers included a check list of gender inclusion issues and sign posting to best practice in this area in order to help coordinators improve their responses and projects. However it was also pointed out that it is difficult for projects to be strikingly original in terms of a 'gender aspects action plan' as what is possible depends on the field. On this point the opinion was raised that the majority of energy topics refer to a male dominated sector and imposing fixed tariffs, on a project tender mix, would be counter productive. As with fixed targets for any aspect of the partner or team mix it is more important to have the right (high quality) people involved.

6.3 New vs. Repeat Applicants

The survey responses showed that around 1/3 of participants are involved in an EU Energy related funding programme for the first time, but also that over half have been involved in a previous FP programme, while almost 75% had received funding nationally. The high proportion of those with previous FP experience points to this being beneficial to making a successful application, but at the same time the significant minority without any prior FP experience, highlights that prior experience is not an essential requirement, and that the programme remains accessible to new entrants. Analysis of the survey responses also showed that it was rare for an organisation to have previously applied for EU research funding but not national research funding, the majority had received both, and a sizeable minority (~20%) had previously received neither. Based on the survey evidence FP7 Energy can be said to remain accessible to participants, with a healthy intake of new participants to the programme and retention of previous participants. On this basis the programme has no disproportionate negative or positive allocation or distribution effects. Project coordinators were questioned regarding the importance of previous FP experience in putting together a successful application. The coordinators agreed that such experience did help but did not feel that this should necessarily be viewed as a problem as long as good projects and organisations were supported. One coordinator felt that if previous projects had been successful this was a positive indicator that should go in favour of an applicant and that it could be the case that additional funding would be needed to help move projects out of R&D towards commercialisation.

1/3 of participants are new to the FP.

Appears accessible to new applicants.


7 Sustainability

7.1.1 Definition

Will the effects achieved last in the medium or long term? In addition to these issues of persistence we have presented the views on the future direction of the programme under this heading. These issues could arguably be placed under relevance, as they relate to maintaining relevance into the future, but we have placed them under this heading to aid report balance.


The TOR contains the following key issues and questions which we have classified under this heading. Each question is followed by our opinions based on the data, analysis and opinions that we have collected. To what extent has the Commission actively collected and synthesised data on project and programme outputs, results and impacts or contribution to objectives? To what extent have the results of the actions been properly disseminated to stakeholders and the public? What is their exploitable potential, and to what extent can one say that this potential has been fully exploited? With regard to improving project dissemination, the project participants had mixed opinions on the benefits of increased networking between related FP projects. Some were supportive but others, some with experience of trying such an approach, felt that competition between companies and the risk of a lack of synergies made the idea not worth pursuing, with existing networking and conference arrangements being sufficient. There was more support for the idea of some collective dissemination of projects, possibly funded and organised by the EU. This idea would involve the presentation of related FP projects at high level sectoral trade shows / conferences. The collective profile raising benefits of this suggestion appear sound and as such it appears worth trialling. A number of project participants felt that some additional funding should be made available to project post completion to help with commercialisation issues. If this is difficult the procedures and circumstances under which additional funds can be provided, at project end, should be clarified. What are the major challenges for energy demonstration and industry-led activities in the coming year? How could FP7 address them in the best way?

Also considered programme future.

Inter FP project networking may work but its risky.

Collective, EU budgeted, promotion of projects at high profile conferences.

Clarification of project funding extension needed.


There were a wide variety of suggestions made as to what areas should be targeted for future support. A number of these related to follow on work in specific technologies (in which those questioned were active). Some more general areas suggested included fuel cells, biofuels and wave with support for manufacturing expertise and capacity highlighted in a number of areas. A number of cross cutting areas were also highlighted including smart grids and the links between biofuels, hydrogen production and electric vehicles. All of these technologies and issues are the subject of the ongoing work by the technology platforms and the SET plan. Given that the results of this evaluation will be directly put into practice for the elaboration of the work programme 2012 and 2013 (which will benefit from an increased budget), what would be the most suitable activities to include and why? What would be the most suitable funding options to open reaching a balance between demonstration activities and policy research? The SET plan was recognised by all those consulted as being key to the future focus and nature of FP energy R&D. There appears to be something of a misunderstanding among current FP participants that the SET plan will only support large scale demonstration projects led by large companies and will do nothing to support more fundamental research and SMEs. This misconception should be addressed. The Implementation plans for the industrial initiative aspect of the SET plan contain a number of KPIs that illustrate the development areas and goals for a number of technologies. These plans and the KPIs should be clearly referenced in the future FP calls, with alignment between FP Energy and SET expected to increase, as the plans become detailed and the level of success of SET becomes more apparent. There were varying opinions on the degree to which DG ENER support under FP should be solely directed via the SET plan. The most common views was that if the SET plan succeeds this is likely to happen anyway, with the FP perhaps able to fund those projects that MSs were less keen to support because of their EU wide nature, or that companies were less keen to support due to their higher risk. There is a clear and logical hope that the SET plan can learn from the experience of the FP in terms of programme direction, procedures and operation.

7.2 Strategic Energy Technologies (SET) Plan

As has been made clear elsewhere in this evaluation, FP7 has evolved over its life with the decision to make the calls more focussed and influenced by the priorities made in the Strategic Energy Technology (SET) Plan. The annexe to this report contains a detailed description of the SET plan and its recently released Implementation Plans. Its is clear that for energy related research the remainder of FP7, and the potential FP8 will be strongly influenced by the ongoing content, aspirations and success of the SET plan. Therefore, although it is not the purpose of this report to evaluate the SET plan its current and future importance has led us to seek views on it during our work.

Potential future priorities included follow on, manufacturing expertise and x cutting issues.

The SET plan is key to the future focus of the FP.

KPIs in SET plan are becoming clearer and this should improve over time.

DG ENER could only direct FP funds via SET but focus on projects which ‘fit’ FP criteria.

SET should learn from the FP.

The DG ENER FP7 programme is already linked with the SET plan.


7.2.1 Consultation Views on the SET plan

The SET plan was described as a step change in the approach to low carbon research and development compared to the last 3 years (and before). The key aspect was felt to be the industrial initiatives, which should enable activity on the scale required to make a real impact, by pulling in and directing industry money and industrial ability to drive market introduction. However it is not yet clear exactly how the industrial initiatives will be achieved and some industries have expressed reservations due to the amount of money they are expected to commit and the feeling that they have been presented with a fait accompli regarding the choice of technologies. In terms of the scope of the technologies in the SET plan it is arguably right to keep the scope as wide as possible and to keep the plan as a living document. This would enable the plan to develop as it becomes clearer which technologies are the most appropriate, this is very difficult to predict currently, given the market complexities involved. It is recognised that the SET plan is duplicating MS activity to an extent, though the point was raised by one consultee that the SET plan process was started in advance of large scale activity and commitment by most MSs, and other international competitors such as the USA. It is now the case that the US, Korea, Japan and China now all have equivalents to the SET plan and their nation status means they are able to instigate their plans quicker than the EU. The SET plan could ideally help MSs avoid duplication by being very clear on the areas it intends to fund and the roadmaps to 2013 do show the areas that are the expected focus. MS level implementation of clean energy research is however a complex and fast moving field and it difficult to try and complement this activity. There is potential for it be used to help coordinate and supplement MS level activity, for example the SET plan has called for 12 CCS schemes around the EU and this could take the form of industry / public cooperation in a number of MSs. The boundary between EU and MS funded research programmes is an interesting area and not one that has been much covered, one consultee, with a management role in the UK’s Carbon Trust (who fund research into low carbon) felt that exploring and explaining this boundary would allow better EU-MS links. The key performance indicators (KPIs) in the SET plan were described as having been hard to agree. They are intended to be steps on the way to ultimate goals (of reduced Carbon – via more res and improved energy efficiency). Another consultee pointed out that there is a marked variation between technologies with clearly specified targets for some and much vaguer aspirations in others. This poses a risk that technology areas with vague aims will lead to vague projects. Future for FP8 and the SET plan The future focus and nature of energy research under the FP is, in the view of those consulted, clearly closely linked with the SET plan. FP7 (and in time 8) will face a challenge regarding where it fits into the new landscape, if only because the level of funding it has is small in comparison to the total of the SET plan (in the order of 5%). The research focused objectives of the FP also imply that it should not all be used in the SET plan, as this is more focused on large scale adoption of renewable energy and energy efficiency. It was reported by one consultee that some current FP applicants / beneficiaries are not happy about SET as it represents a complete change from what they know, and the procedures and approaches of the FP from which they have learnt to take maximum benefit. In the opinion of

The concept of the SET plan is very welcome.

The SET plan is evolving quickly.

SET plan links to MS, EU and private funds are not yet clear. .

The SET plan approach is being used in other parts of the world.

SET plan KPIs are useful but vary in detail.

The FP future in SET is a challenge to define.

This change is not welcomed by all.


this consultee in order for the SET plan to work it needs the best research centres and companies to get involved, and this poses a risk to those more middle ranking universities and companies who currently tend to benefit from the FP. The SET plan will also be a difficult area for SMEs and new member state applicants to participate in. If there is still a desire to support these groups in low carbon energy research there is an argument that they may need a different support mechanism. In the view of one consultee it is not credible to have a programme which aims to support world class, cutting edge research but that also supports capacity building in SMEs and NMSs. Attempting to combine this in a particular project imposes costs and risks, reducing effectiveness. The SET plan itself will also need to link with the activities that will be planned under the auspices of the European Institute of Innovation and Technology (EIT) and its energy focused Knowledge and Innovation Community (KIC). There are a number of areas where there is a clear potential for linkages, such as smart cities and the need to define the FP8 priorities / nature etc. This is an issue that will need to be kept under review as the decisions on SET, KIC and FP8 are all made in the relatively near future. Profile of the SET plan There was a general consensus among those consulted that the profile of the SET plan is not very high. From a MS perspective this lack of profile and a perception of vague aspirations and lack of clarity on why these issues should be best considered at an EU level has led to a lack of allegiance to it. Another consultee felt that those who should know about it, research staff in major companies, were aware of the SET plan but agreed that in terms of political profile the SET plan is less known. The consultee felt that this low political profile was in part related to the difficulty the Commission have in communicating their aims, and there habitual use of mild language – to avoid confrontation. In addition there is the political problem of the Commission lacking an obvious spokesman for a programme such as the FP, which contrasts with the US where the president can make bold claims for their R+D programme. Increasing the profile of the SET plan was thought to need a wider co-operation between public and private partners. The opinion was raised that the current lack of public funding (due to the economic downturn) should help attract public authorities and their projects particularly those looking to replicate technologies. One consultee, felt that there was actually no need for the SET plan to have a high public or political profile as long as it was known of an understood by the target audience – energy companies and research groups. The project coordinators interviewed were asked their opinions on the future links between the FP and the SET plan. This revealed some interesting and varied perceptions. A coordinator from a very large research institute raised concerns that the rumours were that the EC plans to focus their research support solely on large multinational companies and move away from supporting SMEs. This approach was not popular as it was felt that FP would lose the ability to support growing but resource constrained SMEs. They believed that continuing to work with SMEs should be central to the SET plan and the FP. Another coordinator was also under the impression that the SET plan objectives were very focussed on demonstrations and as such he felt that funding and focus should also remain for fundamental R&D. There was also concern that too high of a reliance on private funding would put too much emphasis on cost effectiveness which could be detrimental.

The very large projects will be harder for SMEs and NMs to enter.

The SET plan is one of many new and evolving mechanisms to support low carbon R+D in the EU.

The profile of the SET plan is not high, though this may not matter.

The economic downturn may help attract public authorities to participate.

Some perceive SET as only supporting demonstration and not research.


One coordinator felt that the EC technology platforms were a good mechanism for influencing the future direction of FP and SET and that the current process offers plenty of chances to influence/contribute to the policy debate. One coordinator felt that the FP funds should be used as part of the SET plan and should focus support on the best technological solutions independently from their proposers (SMEs or Large Enterprise). The involvement of MS funding was considered necessary in order to create a “fund association” with the European Commission and others creating Public Private Partnerships in order to support large scale demonstration projects as well as smaller projects. They referenced the Marguerite6 programme as being a good example of such an approach.

7.2.2 Officer Views of the SET plan

All of the officers consulted agreed that the SET plan, if at all successful, would be the major influence on the future of energy research under the FP. However they also pointed out that the success of the SET plan is not yet guaranteed and the response from industry and member states will only become apparent over the next few years. For example the fact that not all MSs are (yet) actively participating in SET complicates the issue. Synergy and avoidance of duplication, with national programmes would be good as the case could be made that some FP projects could have been funded by MS funds. Though EU added value projects will remain an area that MSs are unlikely to fund, which should help avoid duplication and leave a ‘space’ for the FP to fill. A number of those consulted pointed out that the SET plan has two main aspects, the Industrial Initiatives and EU Research Alliance. It was thought that the FP has a role in both. With regard to the industrial initiatives, given the very large scale of funds envisaged it is clear that FP funds can only be a relatively small contributor and the risk exists of diverting the FP from its original aims. However it was suggested that the FP has a potentially important role in helping to influence the direction of the industrial initiatives. One officer described this role as being the ‘brain’ of the programme, for example helping to identify relevant projects and helping to define indicators of success. Another officer echoed this point suggesting that there should be lessons to be learnt from the history of FP to help the SET plan. It was suggested that in order to retain its focus on innovation the FP should support projects (possibly within SET) which are higher risk, related to proof of concept and higher EU value added (i.e. with potential for impact and replication across the whole EU). Such an approach was suggested as being one which would help clarify where FP sits in relation to SET. It was also pointed out that given the intention of the Industrial Initiatives in the SET plan to continue to target future funds on large demonstration projects, it will become even more difficult to try and give an equal spread of funds across the member states, as the large companies who will tend to dominate large demonstration projects are more common in certain MS, typically in the EU 12. Though some opportunities may arise from this approach to SMEs and NMSs as useful consortium members,

6 The Marguerite fund is a pan-European infrastructure fund for long term institutional investors to finance the implementation of

strategic European policy objectives and projects in the Transport, Energy/Climate and Renewables sectors. The main objective of the fund is to contribute to the European Economic Recovery Plan by financing the implementation of strategically important European policy objectives in Energy/Climate, Renewable and Transport sector infrastructures.

Technology platforms should continue to influence priorities.

The FP should retain its focus within SET.

The success of the SET plan is nit guaranteed.

Projects with significant EU added value offer a role in SET for the FP.

The SET plan contains research as well as demonstration goals.

SET can and should learn from the FP.

The FP should retain its identity within SET.


There was also support for the EU Research Alliance aspect of the SET plan as there was a strong belief that there remains a need to support more fundamental research, with the research capacity it enables, and the longer term technologies it can invent and research. These aims were described as being closer to the traditional aims of the FP. A number of those consulted also raised the suggestion that some FP funds should be held back from SET related projects and used to support other types of intervention. However, these other interventions should be of a similar nature to what the FP has traditionally supported rather than any refocusing away from this, e.g. it should not focus more on SMEs and should continue to support projects led by major companies.

7.2.3 Framework Programme Development

A number of issues related to the development of the FP as a whole were also mentioned by those consulted. The ongoing FP wide efforts to simplify procedures were mentioned and generally supported, with the caveat that there is an inevitable balance to strike between reducing burdens on applicant s and beneficiaries but also ensuring that public money is wisely and honestly spent. It was also mentioned that there are a number of other EU innovation support programmes of relevance, which the FP needs to complement: These include:

• New research and innovation strategy (under DG RTD was DG Enterprise) which is developing sectoral roadmaps, e.g. energy efficiency in buildings. There are possibilities here to combine supply and demand side funds, which for DG ENER implies FP and IEE plus others.

• The European Institute of Innovation and Technology (EIT) and its three Knowledge and Innovation Communities (KICs), one on Sustainable Energy and one on Climate Change Adaptation and Mitigation, were also mentioned as a recent development of clear relevance. The KICs are virtual networks of research excellence who will be seeking funding from a combination of funding (national, private etc.). It can be expected that these groups will be bidding for, and seeking to align with the activities of the SET plan and the future FP.

• The Joint Research Centres – clearly remain relevant and are strongly involved in the SET plan, for example they are running SETIS - the SET plan information system. The purpose of which is to “efficiently collect, harmonise, validate, analyse and disseminate information on the priority energy technologies identified by the SET Plan, across the EU. The goal is to provide undisputed and transparent data and methodologies to support the strategic planning, monitoring and evaluation of the European Energy Technology policy”7

• There continue to be other areas of the FP which share some objectives with the part supported by DG ENER, For example the DG ENV areas covering eco innovation, resource efficiency and environment, although formally excluding energy there will inevitably be some common areas of interest..

7 See: http://setis.ec.europa.eu/setis-main

A number of those consulted felt the FP should retain some funds outside of SET.

There are multiple EU led initiatives of relevance to FP ENER. DG ENER needs to keep its involvement in the development of these policies


The current participants were asked their opinions on where DG ENER should focus its future FP funding. Their suggestions fell into the following categories Specific Technologies – typically follow on ideas from existing projects Ø Solar generation from industrial processes; Ø Explore the use / upgrade of naphtha by products from bio refining. Ø Collecting, handling and pre-treatment of residues for lignocellulosic ethanol Ø CO2 capture using algae Ø Biofuels from high cellulose materials;

Wider Technology Areas – again typically related to their activities, though the need to support and develop manufacturing abilities was mentioned by a number of participants. Ø All aspects of PV development. Particularly manufacturing efficiency, excellence,

productivity, low cost, competitiveness etc; Ø Fuel cells, in particular advanced industrial manufacturing capacity; Ø New production technologies and control systems – to reduce costs and improve

efficiency; Ø To move towards a standard approach / design for wave / ocean energy. In the same

way as happened for wind; Ø Nuclear - new and more secure nuclear technologies, address the problem of

radioactive waste, etc; Ø A solar energy focussed call combining heating, chilling and power to reflect the

scale of potential demand for this technology in hot countries. Ø Grid asset management, particularly in Eastern Europe.

Cross cutting - enabling technologies or those linking energy technologies Ø Smart networks - ICT-related developments, grid connections; Ø Synergy between biofuels / hydrogen and electric cars. E.g. Fuel cell cars using

hydrogen from gasification and shift conversion processes. Non technical issues – not typically issues which the FP can directly influence. Ø Stability of the legislative framework including long-term targets – e.g. the strict

retention of the 10% biofuel requirement for 2020 in the Renewable Energy Directive by all Member States..

Nature of the Programme Ø In terms of future areas / technologies, the Commission should give further

consideration for potential FP calls to mix member states with developing countries (outside EU). There is an opportunity to build markets / products with these countries where the future demand for energy is likely to be higher.

• There was some direct contradiction between project coordinators in terms of whether the focus should be close to market or more fundamental research

7.2.4 Longevity of Impacts and Improving Dissemination

The project coordinators had the following comments on the probable persistence of the project effects. One coordinator felt that dissemination is generally very good but commercialisation can be more 'hit and miss,' depending on participation and how the outputs link to the business of the industrial partners. Another coordinator was more critical stating that he thought “ the programme will not really produce long-lasting effects for a number of

Participant suggestions for future focus areas include:.

Specific technology / follow ons.

Wider areas – PV, fuel cells, wave, multiple solar ,including manufacturing expertise.

Cross cutting areas including networks and biofuel – h – fuel cell – electric car links.


reasons including the fact that SME participation is not as strong as it could be and also because there is a tendency to avoid high risk projects (which are likely to have a greater potential impact)". A third coordinator thought that dissemination was good and that it was easy to find information on projects if you look for it. The project coordinators were asked for their opinions and suggestions on how project dissemination and long term impacts could be improved. One suggestion that was tested out with a number of the coordinators was the idea of increased inter project networking. There were a variety of contrasting views on the benefits that such an approach could bring. One coordinator did not see any benefits from inter-project networking, citing his own experience of trying to network four laser projects which failed due to a combination of competition between the projects and a lack of synergy. He felt that existing networking via academic channels and conferences was sufficient which was an opinion echoed by other consultees. In contrast another participant thought there should be more inter-project networking and had never been involved in any previous attempts. Another coordinator also thought that the idea of joint conferences was a good one and would offer the benefit of increased networking between projects. A third coordinator suggested a two-day workshop where all relevant FP7 energy projects were presented. A coordinator who had been involved in a joint conference involving CCS researchers felt that it had been useful and was a good idea for certain research areas. With regard to conferences another coordinator stated that discussion was limited to published results due to confidentiality and IPR protection, though this was the accepted norm and he thought that any requirement on FP projects to network with each other would be a somewhat artificial approach. The retention of some IPR by project partners was seen as a very valuable benefit from participation and one that should not be lost in efforts to more widely publicise results. However another project participant commented on a reluctance of industrial partners in his project to share knowledge and learning with other partners. He felt that there was a need for stronger guidelines at application stage on knowledge sharing. The issue of disclosure was described as complex, as if FP7 requirements are more towards full disclosure, it would put partners off participating from fear of losing their IPR. However, he thought that the balance doesn't seem to be quite right at present. One participant raised the idea of additional follow on funding to assist in dissemination and commercialisation. The amount he thought would be required need not be large (may be only a few thousand Euros) but it could make the difference between a project stopping or actually achieving market entry and as such it would be an investment worth making. Another coordinator was less sure on the idea of follow up funding , stating that: “There may be good reasons why a project shouldn't continue or why it should be considered as a 'test-bed' and should be taken forwards outside of EC FP activity”. They also pointed out that there is already some flexibility to secure funding extensions, though they believed that this was not consistently applied to all projects. One current participant felt that a separate contract for post project activity would be a complex arrangement One coordinator suggested that the EU should develop a dissemination strategy when products are mature and use highly recognised international events like Inter Solar or Solar Energy to present their FP7 projects. He felt that this would improve the credibility of the projects and help commercialise them. Such a collective approach was thought to be more efficient, in terms of impact per € spent than lots of individual project actions. This

Inter FP project networking works for some but does not appear a strong prospect.

Existing procedures and routes suffice for academic dissemination.

Some misunderstanding of the availability and purpose of additional funds to complete projects.

Good support for a combined promotion of projects at high level trade shows / conferences.


suggestion appears strong and was described as one which the CT in the UK have adopted with some promising results. One coordinator stated that his project has agreed to keep the project website alive for 1-2 years following project end to assist dissemination, he felt that this was good and it may be a good idea to oblige other projects to do this. One project coordinator mentioned security concerns regarding publicising the results of his project as it concerned the electricity infrastructure which is seen as an area of concern.


8 Conclusions and Recommendations

The conclusions for each evaluation question are presented at the start of each chapter. These recommendations stem from those conclusions and have been presented under sub headings in line with the programme life cycle. Each recommendation is followed by a brief discussion and suggestions as to how it could be practically realised.

8.1 Programme Relevance / Strategic Fit

1. DG ENER should maintain an input to the focusing of DG RTD calls and vice –versa in order to maintain coherence between the research focus of RTD and the demonstration focus of DG ENER.

This reflects the need to maintain a link between the energy demonstration projects and the more fundamental research projects, which could one day become demonstration projects, and / or which needs to reflect any barriers highlighted by demonstration projects. Ideally this link / input needs to occur at several levels, ranging from the technology platforms, to head if unit to informal gatherings between project officers.

2. The SET Industrial Initiative implementation plans and their KPIs, plus other relevant SET plan documents that are developed over time should be clearly referenced in the future FP calls, with alignment expected to increase as the plans become detailed and the level of success of SET becomes more apparent.

This reflects the support for the alignment of the FP and SET and the increasing detail available on the SET plan targets, such as the Industrial Initiative plans. The call for proposals needs to have clear reference to the technology specific SET plans and make it clear that applications should attempt to show how their projects will contribute towards the KPIs and other SET plan targets.

3. Energy efficiency needs to be clearly articulated as an objective of the ENER FP programme. The whole of the FP should be analysed to ensure that that other parts of the FP are indeed picking up relevant projects.

Energy efficiency is a vital high level objective of energy efficiency so any projects or ideas that can genuinely help should have some chance of receiving FP funding. There was a widespread opinion that energy efficiency is being covered elsewhere in the FP but this should be verified.

8.2 Information to Applicants


4. More effort should be made to clearly articulate the programme objectives to applicants in a clear and simple way in the application pack.

This is to help ensure that the applicants have as good an understanding as possible of the mix of objectives that the programme is intended to achieve. This could be achieved by including the objective tree with a few simple examples of where typical projects match the objectives. For example: A project to demonstrate a new design of wave energy device – would help demonstrate the feasibility of a commercial scale application, which would help lead to an increase in low carbon power generation capacity, the support and development of EU manufacture, installation and maintenance expertise and the improvement of practical and scientific knowledge on performance. A project to develop real time information on grid demand supply from household level and match it to generating capacity (i.e. a smart grid project) would develop approaches and technologies which could enable increased use of renewable (i.e. low carbon) generation, thus enabling CO2 emission reductions and supporting those sectors that produce such generating capacity.

5. There appears to be something of a misunderstanding among current FP participants that the SET plan will only support large scale demonstration projects led by large companies and will do nothing to support more fundamental research and SMEs. This misconception should be addressed.

This could be achieved via clear explanations in the programme documentation but is also an issue which DG ENER should feed into SET plan development and publicity.

8.3 Programme Management

6. There were a number of detailed suggestions made that could help improve the

efficiency of programme operation, some of which were well supported by other consultees and appear to be worthy of uptake, if only on a trial basis. These are:

a. A switch to remote as opposed to on site evaluations – as most officers consulted felt the ability they offer to attract better experts (due to increased flexibility) outweighed the concerns over document security.

b. Formalised pre submission meeting procedure – to ensure a level playing field between applicants. Clarify the procedures at information days.

c. Formalised applicant interview, particularly for large projects – to enable a better understanding of the project objectives and avoid easily explainable misunderstandings. For projects over a certain budget with the choice of which partner staff left to the coordinator.

d. A more extensive end of evaluation meeting – involving external experts and extended discussions to arrive at the agreed approved project list. If the resources exist to support such a meeting.

e. A set of project management KPIs – to enable easier and more formalised ongoing assessment of project progress. A mix of procedural and output quality indicators.

f. Increased use of external advisors to assess interim and final reports – to bring additional technical expertise and scrutiny to the reviews of project progress. Clarify the availability and increase if required (and affordable).


8.4 Project and Programme Impact

7. In order to enable the vital assessment of project impact and success that virtually all

consultees agree is needed, the authors of this report suggest that some form of relatively low cost payment could be offered to project coordinators as an incentive to provide short update reports at fixed periods after project completion.

This requirement should be part of the original contract and be structures such that some relatively small fixed amount (say €1000 - €2500) is paid to the coordinator each year (or every other year) for some 5 to 6 years after project completion to supply answers to a short pro-forma on post project developments. The questions should include: current status – ranging from all work stopped, through continuing research – directly or on a particular aspect, to small scale production and mass production. Questions on related patents should also be asked as should questions on employment and income, as well as scientific and commercial collaborations with contacts made directly (and indirectly) through the project.

8. The majority of those consulted felt that given the importance of CO2 savings, and the clear link which should exist between demonstrating low carbon technologies and large scale replication, project applicants should be very clearly requested to estimate the scale and nature of the CO2 savings that their projects could enable. There should be simple standardised methods on how applicants should do this and it should receive no more emphasis in the evaluation procedure than it already does.

It should be made clear to the applicants that a high figure will not necessarily increase the likelihood of the project being scored highly. It is more important that the submission illustrates their understanding of how the work leads to carbon savings and the technologies and competitors it will face if the supported project is successful.

9. Trial the collective presentation of completed and ongoing FP projects at a well regarded trade show / conference.

The current portfolio of project should be reviewed for a group of successful recently completed / very near to completion projects in a related field and the coordinators contacted to ascertain their interest in such an approach.

10. The procedures and circumstances, under which additional funds can be provided, at project end, should be clarified. Such funds may be useful to aid dissemination and commercialisation.

The circumstances under which project extensions are allowed should be highlighted to the approved projects and those projects going through the negotiation phase. It should be made clear that there is no guarantee of such funding and that it is awarded only under exceptional circumstances.


8.5 Programme Future Direction

11. There was wide consensus that the technology roadmaps under development via the

SET plan Industrial Initiative should be more explicitly used by the FP7.There is a debate to be had on how and if the FP should retain its identity within SET.

This recommendation overlaps with nos. 2 and 3 with the additional implication here being that DG ENER need to ensure that their voice and opinions are continued to he heard in the ongoing development and operation of the SET plan. The ENER FP could direct all its funding into SET, or target those projects with match FP ideals, e.g. EU added value, higher risk, more SMEs involved, or it could give some funds to SET and retain some separate funds. These are complex issues and the optimum choice depends on the policy objectives that are required and how they are prioritised.

12. There is a clear hope that the SET plan can learn from the experience of the FP in terms of programme direction, procedures and operation.

This recommendation overlaps with recommendation number 10 and illustrates the extent to which the future of FP funding by DG ENER is linked to the SET plan. Therefore it is clearly in the interest of both if the lessons learnt by DG ENER are utilised in order to help deliver the SET plan.

13. There are a number of emerging and evolving policy mechanisms for low carbon research and development. If the FP wants to retain its identity it needs to be actively involved in the policy debate around these mechanisms.

A number of DGs and programmes are becoming involved in this area. In order to avoid duplication and maximise synergies efforts need to be made to ensure that the information flows between the DGs are as free and deep as possible.


Annexes


1 Context

1.1.1 High Level Policy Drivers

In order to define the objectives of the group of projects which are being evaluated it is necessary to review all levels of relevant EU policy.

The DG Energy supported energy research component of the Seventh Framework Programme (FP7) needs to be seen within the overarching content of the EU’s objectives as a whole:

“The Community shall have as its task to promote throughout the Community a harmonious, balanced and sustainable development of economic activities, a high level of employment and of social protection, equality between men and women, sustainable and non-inflationary growth, a high degree of competitiveness and convergence of economic performance, a high level of protection and improvement of the quality of the environment, the raising of the standard of living and quality of life, and economic and social cohesion and solidarity among Member States.8

EU supported research goals are therefore aimed at serving “higher” economic, societal and environmental goals as well as the theme specific objectives, including meeting the challenge to turn Europe into a knowledge-based society: a society where the knowledge triangle of research, education and training, and innovation is fully mobilised. Some of the most relevant related EU policies and programmes include the:

The EU Sustainable Development Strategy: adopted in 2006 is the overarching strategy for all EU policies and sets out "how we can meet the needs of present generations without compromising the ability of future generations to meet their needs." The strategy covers 7 key areas including climate change and clean energy. Among the principles highlighted are the need to involve business and citizens and to work towards an innovative knowledge based economy. This is at the root of all ERA work in the energy sphere.

EU Energy and Climate Change package: which sets out the 20-20-20 by 2020 agenda for a 20% reduction in CO2 emissions, renewable energy to make up 20% of all energy consumed and 20% gains in energy efficiency by 2020. Innovation, research and education and training will all play key roles in the development, installation and operation of the technologies necessary to meet these targets.

The Environmental Technologies Action Plan (ETAP): is very closely related to the


successful creation of an ERA. Its objectives are to improve eco innovation and the competitiveness of EU environmental technology industries. By creating standards, technology centres, networks and supporting markets it expands the market for the research being sponsored under ERA. It also works at the next steps beyond ERA in fully commercialising technologies. Ecorys have carried out a review of the implementation of ETAP.

Internal EU gas and electricity market: the EU's stated policy goal is "to set up a truly competitive internal market for gas and electricity to offer consumers a real freedom of choice at fair, competitive prices, to stimulate clean energy production and to improve security of supply." Deficiencies in progress towards this goal have been identified and led to the recent adoption of a third package of measures with the primary aims to continue to open markets to competition. While ERA outputs may be focused on technological issues for networks there is also scope for increased cross-border research as the market increasingly becomes trans-national.

European Energy Programme for Recovery: this initiative by the commission is in response to the economic downturn and has the goals of stimulating economic activity while contributing to strategic energy policy objectives. €4 billion of funding until 2010 has been allocated in 3 areas, gas and electricity infrastructure, offshore wind and carbon capture and storage (CCS). These areas, particularly CCS, will all draw upon new research and innovation stimulated within the ERA.

Trans-European Energy Network Programme (TEN-E): is an initiative targeted specifically on energy infrastructure that aims to improve the internal EU energy market by encouraging energy diversification and interoperability to increase energy security, promoting and strengthening economic and social cohesion and supporting sustainable development. By focusing funding on feasibility studies there is a close relationship to the utilization and demonstration of energy technologies such as those developed through the ERA network. An evaluation of the TEN-E programme has been carried out by Ecorys.

The Seventh Framework Programme (FP7) is a key pillar in delivering the wide policy goals described. Its broad global objectives are grouped thus9:

• Cooperation – support for transnational activity at an appropriate scale across the EU and beyond, in thematic areas corresponding to major fields of knowledge and technology where research should be supported and strengthened to address European social, economic, environmental, public health and industrial challenges, serve public good and support developing countries.

• Ideas – activities to be implemented by the European Research Council (ERC) in developing “high-level frontier research” at EU level, building on excellence in Europe and raising its profile at international level.

8 Treaty Establishing the European Community (Consolidated Text), Official Journal C 325 of 24. 9 European Parliament and Council Decision No. 1982/2006/EC of 18 December 2006 concerning the Seventh Framework

Programme of the European Community for research, technological development and demonstration activities (2007-2013)


• People – individuals to be stimulated to become professional researchers, be encouraged to remain in Europe and the best researchers from outside Europe attracted here. Mobility is a key dimension; for professional and personal development for also for knowledge sharing and transfer of knowledge between countries.

• Capacities – the use and development of research infrastructures should be optimised, innovative capacities of SMEs strengthened and regional research-driven clusters supported. Science and society should be brought together in European society and support given to coherent research policies at national and Community level and horizontal actions and measures undertaken in support of international cooperation.

It is important that our approach reflects the overarching objectives of the FP7 as a whole, (since judgements on the success of the energy component must also concern its contribution to the programme as a whole), without loosing its focus on energy. The general FP7 objectives are diverse and reflect a very wide range of policies and strategies. The ex-ante assessment of FP7 provides a useful starting point in understanding them:

It is also important to bear in mind the relevance and potential cross over with a multitude of other relevant EU policy activity. A recent European commission communication10 'Reviewing Community innovation policy in a changing world' referenced the FP as well as the SET-Plan, the Action plan on Sustainable Consumption and Production and Sustainable Industrial Policy and the Eco-design Directive as key policies in triggering market uptake of innovative products. The same communication also mentions standardisation policy, public procurement and the Lead Market initiative as being important in this area. This will lead to the adoption of an EU Innovation Action Plan in 2010. In terms of financial support for research innovation the FP was highlighted along with Cohesion policy, Rural development policy and the CIP. There are also clear links between innovation and policy activity in education and skills where the key aims are also enhancing competitiveness and promoting and supporting the knowledge economy.

1.1.2 Energy research under FP7

Energy in FP7 covers a wide range of subject areas and disciplines, reflecting the multidisciplinary and cross-sectoral research required to support policy-making in this important field. The DG Energy specific energy component of FP7 is described in the table below:

Table 1.1 Energy research in the Seventh Framework Programme (2007-2013)

Seventh Framework Programme (2007-2013) (EUR 53.2 BIO)

EC Programme (EUR 50.5 BIO)

Cooperation (EUR 32,365 MIO)

Energy (EUR 2,350 MIO) DG TREN and Research

10 Communication from the Commission to the European Parliament, the council, the European Economic and Social committee of

the Regions. 2.9.2009. Com (2009) 442 final.


• Hydrogen and fuel cells • Renewable electricity generation • Renewable fuel production • Renewables for heating and cooling • CO2 capture and storage technologies for zero emission power generation • Clean coal technologies • Smart energy networks • Energy efficiency and savings • Knowledge for energy policy making

Energy - DG Energy (EUR 501.4 MIO in the first 4 calls)

FP7-ENERGY-2007-2-TREN 128 mio • 45 topics

FP7-ENERGY-2008-TREN-1 147 mio • 18 topics • Post Strategic Energy Technology (SET) plan. (7

key energy technologies to 2020, 7 other technologies to 2050).

• More industry commitment • Focus on core topics • Intensive consultation on needs and bottlenecks • More funding per project

FP7-ENERGY-2009-2 100 mio • 10 topics

FP7-ENERGY-2010-2 (open) 126.4 mio • 8 topics • Call opened July 09. • Deadline for applications March 2010

The FP7 research programmes have been constructed thoroughly and carefully, and it is therefore important for the evaluation that our team has a genuine knowledge of the rationale for inclusion of the various research domains; and is familiar with current research and development, together with the relevant critical issues and debates.

The themes supported by DG Energy have evolved over the life of FP7. This development during FP7 reflects past practice, for example during FP5 the decision was taken to stop supporting projects designed to improve the extraction of oil and gas, on the basis that the oil price had risen to the extent where subsidising oil companies could not be justified. The calls are described in more detail in section 3.

1.1.3 Division between DG Energy and DG RTD

The Directorate General (DG) Energy – formerly Transport and Energy (TREN) - and Research both fund a variety of energy research programs. They both support FP projects however they focus on different aspect. Below we describe some of the distinguishing aspects of both DGs in terms of the research they support.

DG Energy DG Energy provides and manages the funding for a variety of energy research initiatives for trans-European networks and technological development (like the pilot cases for CCS) and innovation. Fundamentally the focus of DG Energy is on commercialisation and


demonstration projects. This means that from a typical technology life-cycle perspective the technology development is past the basic research and pilot phase. As the technology is close to commercialisation private organizations are more willing to take the lead in this stage of the development process. Industries commit themselves to the technology as they see the potential for it to result in profits for them. Moreover, the risks of investment shrink as a result of (successful) pilot projects. However, as there are still a large number of risks (technical, economic, etc.) the public funding through DG Energy is still very important. Furthermore, the technologies funded under DG Energy are expected to penetrate the market in the relatively short term. The scale of the technology at this point in the life cycle is potentially being adopted at such a pace that it would qualify as widespread technology diffusion. This implies that these industrial led projects can potentially have a significant impact on the 2020 targets relating to emissions and efficiency.

DG Research DG Research provides and manages the funding for a variety of energy research and technological development initiatives. Fundamentally DG Research focus on the first stages of the life cycle of technology development. This means that fundamental research would be eligible for support. As such research is relatively far from commercialisation (compared to DG Energy projects) it takes a longer term perspective in what it supports. Furthermore, it implies that these research projects focus on future technologies.

1.2 Lessons from previous evaluations

When approaching evaluation assignments, some of the key reference documents are the most recent previous reports in the evaluation cycle of the programme in question and of closely related programmes. There are a number of reports of such relevance to this assignment, as follows:

Ex post of FP5 and FP6 DG RTD Energy projects has recently been completed - summary The Ex-Ante Evaluation of the 7th Framework Programme for RTD in the field of Energy and Transport (2006-2010) was carried out by ECORYS in 2005.11 It looked at the whole programme of DG TREN FP7 activities, including Nuclear and Transport. In setting out the rationale, objectives and delivery mechanisms for the programme it made a number of key recommendations and developed the underlying intervention logic. The ex-ante evaluation aimed to support the discussion on DG TREN’s role in FP7 and the contribution of the research programme to the objectives of European transport and energy policies. The main question related to the value added for DG TREN of being involved in FP7. Five intervention strategies were defined according to the level of involvement: full package, focussed approach, programming only, independent and no RTD. The pros and cons of these strategies were assessed with a multi-criteria analysis. For the first four intervention strategies recommendations were formulated to assure an appropriate effect. For the latter intervention strategy the recommendation was to not adopt it. The recommendation was that with a little

11 ECORYS (2005) Ex ante Evaluation of the 7th Framework Programme for Research and Technological Development in the field of

Energy and Transport (2006-2010), Rotterdam. Delivered under the previous evaluation framework contract


additional effort the strategy could be ‘programming only’ which clearly generated additional benefits. Parallel to this study a number of research priorities for FP7 were established containing new orientations that envisaged an improved response to the developments in the energy and transport sector. This also applied to the domains that were to be continued in FP7, such as CIVITAS and CONCERTO, road safety, intermodal transport and intelligent transport systems. The research themes (priorities) that were defined were:

• Energy efficiency and renewable energy sources • Clean coal technologies • Nuclear waste management • Air traffic management • The clean and safe vehicle • European systems for satellite navigation • Efficient and interoperable rail transport • Measures to profit for the findings of research

Other main conclusions of the study involved the main critical success factors in translating successful projects into useful results and impacts, these were:

• Policy take up of RTD results • User involvement and attention to commercialisation and regulatory aspects • Dissemination of results. • Alignment of EU policies versus national interests and interest of participants. • Continuity of knowledgeable EU staff

The Ex-Post evaluation of Non Nuclear Energy (NNE) Projects supported by DG TREN under FP5 was carried out in 2007 by Ecotec (Ecorys in the UK). The sector and technology scope of the FP5 ex-post evaluation is the same as for this mid-term evaluation, reviewing non-nuclear energy RTD projects funded under a framework programme. The key conclusions from this report were:

• Effectiveness: that there was a good match to objectives with only minor issues around improving awareness in new member states and data collection.

• Efficiency: in balancing programme administration it was difficult to satisfy both applicant and audit requirements. Issues were raised around the with-holding of funds.

• Utility: A good match was found between project effects and needs and specific value added was found in encouraging trans-national research co-operation and enhancing the global standing of European energy research.

• Sustainability: that projects that took a holistic approach to research and development were in general more successful than those focused on single issues, that the energy price and relative market subsidies were major factors in success and that firms directly involved and with a commercial interest in the market were also more successful.

From this the following key recommendations were made:

• To definitely continue and potentially expand the programme. • To continue to work closely with industry led technology platforms and let applicants

know of the flexibility in the process.


• To improve data management and collection by DG TREN. • That marketing did not need to expand but could be targeted on newer member

states. • The benchmarking indicators around administration, particularly payment timings

should be developed. • To investigate with DG Research the use of external project technical assistants and

agency to manage the programme more effectively and efficiently. • Strengthening the risk appraisal process and to consider the use of break-point

contingencies and making the extension process clearer. • To consider additional follow-up activity 2-3 years after ex-post to significantly

improve evaluation of sustainability objective. • Areas for further research to include, SME participation rates, more sector specific

analysis, benefits of an agency approach and optimum methods to integrate new member states.

It is clear from the ToR that many of these issues remain relevant (e.g. integration of SMEs and new member states) and that the evaluation of measures implemented as part of FP7 to address these issues is an important element of this mid-term evaluation. The intervention logic developed as part of the ex-post evaluation is shown below. This is largely similar to the objective tree in the ex-ante of FP7 (pp. 21) but it more clearly highlights the objectives around industrial competitiveness and R&D policy and delineates the objectives at the various levels. The literature review in the ex-post evaluation of FP5 reflected that there was still scope to improve the intervention logic being used and that a more systematic approach could improve the quality and usefulness of subsequent FPs and evaluations.


Figure 1.1 Intervention logic from Ex-post evaluation of Non-nuclear Energy Projects supported by DG TREN under FP5

Reduce global warming andenvironmental impacts of energy

Secure Diverse, high qualityand low cost energy services

Sustainable development -EU and worldwide

Improved industrialcompetitiveness

Reduce Energydemand and

imports

Minimise environmental impactof energy production and use

Competitiveenergy markets

Increase uptake ofRES

Increased levels and improvedquality of R+D, displaying- Community added value- Social contribution- In areas of economicdevelopment with prospects

Energy efficiency ofequipment and services

RES levels - %and absolute

Lower CO2 andother emissions

Transparency and diversityof energy market

Critical mass (human and financialresources) in R+D

R+D contrbuting to implemetingcommunity policy and addressing

EU level issues

R+D contributing to employment,quality of life and health

R+D in expanding areas with goodpros[ects, in areas where EU

businesses can and must improvetheir compet

New product / processwith improved efficiency

/ reduced impact.

Improved awareness /uptake of RES andenergy efficiency

Costsavingsachieved

Strengthening EUenergy industry

Improvement oftechnical

knowledge /perforamnce

New contacts made and follow oncollaborations - strengthening ofcapacity , improved networking

R+D which overlaps with other policygoals

Enhance skills of R+D staff, developskills in target areas

Registration of patents, publication ofjournal articles

Global Objectives

Intermediate Objectives

Specific Objectives

Operational Objectives

The ex-ante evaluation in particular discusses the overall need for the FP and the rationale for selecting the specific thematic areas and so will be especially useful input for this task. The ex-post evaluations of FP6 (including the Report of the Expert Group published in February 200912) also contains a useful discussion on intervention logic, noting that, “a more explicit ‘programme logic’ would have produced a more robust overall FP design”. The Ex-Post Evaluation of FP6 by an Expert group report (all themes) – was released in February 200913. The evaluation recognised the common issue that longer term impacts are difficult to identify, even for completed projects, with the accepted compromise being that the link between outputs and results and how these lead to outcomes can be described. This confirms our experience and assumption that given the interim nature of the evaluation we are bidding for the impacts will be even harder to identify - as none of he projects will have completed yet.

12 http://ec.europa.eu/research/reports/2009/pdf/fp6_evaluation_final_report_en.pdf 13 http://ec.europa.eu/research/reports/2009/pdf/fp6_evaluation_final_report_en.pdf


The report also stated that dissemination and exploitation remain key to the achievement of impact and as such they need to be well resourced. This implies that our evaluation should specifically investigate how well this issue is planned into the projects we review and if any efforts have been made to begin dissemination from the start of a project, rather than leaving it all to the end as is commonly the case. Another finding of the report was that active links and networks between project partners, and between projects (including networks of Excellence), are key to technology transfer, project success and behavioural change- this confirms the value in the use of network analysis tools (which will be discussed in the next section). The report recommendation highlighting post project funding for dissemination is an interesting and valuable idea but not one which an interim evaluation is well placed to examine - as no FP7 projects have completed yet. This report (and others) provide evidence of overburdening of Commission project officers – this will be a key issue for the efficiency aspect of our evaluation and is a key aspect at interim stage. The finding that past project coordinators and participants have a great deal of knowledge on best practice, raises the question of the degree of re-participation from these past applicants and the extent to which new partners are able to learn and benefit from their experience. At both project and programme level there are methods to enhance the policy-research links, both for policy makers to pass on the areas of research they would benefit from and for researchers to make, and present, their work in a more policy relevant manner. This is an issue that the relevance and sustainability evaluation questions need to address. Although it covers DG Research as opposed to DG TREN projects the report Evaluation and Impact Assessment of the European Non Nuclear Energy RTD Programme14 has a number of points of relevance to this work, such as: The differences between FP5 and 6, summarised as in FP6 the whole value chain was addressed via larger projects with multiple participants, whereas FP5 projects dealt with one issue at a time. The overall RTD budget decreased from FP5 to 6 by some 30%. Industry participation reduced from 42% to 31%. In terms of results the continuity of support for technologies was identified as an important factor in project (and area) success. In terms of relevance the general conclusion was that the larger scale of the FP6 projects was not as productive as the more focussed nature of FP5. In terms of policy impacts the production of roadmaps was regarded positively partly due to its ability to internationalise the research agenda. The positive contribution of projects and the programme to the policy debate in this area was also highlighted.

14 Evaluation and Impact Assessment of the European Non Nuclear Energy RTD Programme. Development and implementation of a

methodology for evaluation and impact assessment of the energy programme of the fifth and sixth Framework Programme of the European Community for RTD activities. EPEC. October 2009


In terms of economic impact the presence of industrial engagement was highlighted as a positive factor. Distinct variations were also reported between the sectors with solar energy performing well.

1.3 Programme History and Evolution

Energy in FP7 covers a wide range of subject areas and disciplines, reflecting the multidisciplinary and cross-sectoral research required to support policy-making in this important field. The DG Energy specific energy component of FP7 is described in table 1:

Table 1. Energy research in the Seventh Framework Programme (2007-2013)

Seventh Framework Programme (2007-2013) (EUR 53.2 BIO)

EC Programme (EUR 50.5 BIO)

Cooperation (EUR 32,365 MIO)

Energy (EUR 2,350 MIO) DG TREN and DG Research

• Hydrogen and fuel cells • Renewable electricity generation • Renewable fuel production • Renewables for heating and cooling • CO2 capture and storage technologies for zero emission power generation • Clean coal technologies • Smart energy networks • Energy efficiency and savings • Knowledge for energy policy making

Energy - DG Energy (EUR 501.4 MIO in the first 4 calls)

The themes supported by DG Energy have evolved over the life of FP7. This evolution continues past practice, for example during FP5 the decision was taken to stop supporting projects designed to improve the extraction efficiency of oil and gas, on the basis that the oil price had risen to the extent where subsidising oil companies could not be justified.

The Directorate General (DG) of Energy and DG Research both fund a variety of energy research programs. They both support FP projects however they focus on different aspects. Fundamentally the focus of DG Energy is on commercialisation and demonstration projects. This means that from a typical technology life-cycle perspective the technology development is past the basic research and pilot phase. As the technology is close to commercialisation private organisations are more willing to take the lead in this stage of the development process. Industries commit themselves to the technology as they see the potential for it to result in profits for them. Moreover, the risks of investment shrink as a result of (successful) pilot projects. However, as there are still a large number of risks (technical, economic, etc.) the public funding through DG Energy is still very important. Furthermore, the technologies funded under DG Energy are expected to penetrate the market in the relatively short term. DG Research focus on the early stages of the life cycle of technology development. This


means that fundamental research would be eligible for support. As such research is relatively far from commercialisation (compared to DG Energy projects) it takes a longer term perspective in what it supports.


2 Strategic Energy Technology (SET) Plan

2.1 Overview

FP7 has evolved over its life with the decision to make the calls more focussed and influenced by the priorities of the Strategic Energy Technology (SET) Plan. The SET Plan was put forward in 2007 by the Commission in order to accelerate the development and deployment of cost-effective low carbon technologies. It comprises of measures relating to planning, implementation, resources and international cooperation in the energy technology field. The SET plan proposes a new method for managing research on energy technologies based on joint strategic planning. The policy measures are designed to stimulate a more intensive international collaboration strategy. The SET Plan is focused on both short term research close to commercialisation as well as supporting long term technology developments in the field of low carbon technologies. With regard to shorter term research, the SET Plan aims to lower costs, improve performance and stimulate commercialisation of existing technologies. With regard to longer term research the SET plan supports the development of next generation technologies relevant to low carbon. Actions are focused on improving specific technologies such as renewable technologies, energy storage, fusion energy, etc. In October 2009 the European Commission called for governments, firms and researchers to combine efforts to develop energy technologies (in line with the priorities of the SET Plan for a number of reasons: tackling the climate change problem, safeguarding the energy supply in the EU and to guarantee the competitiveness of the EU economies. It would, furthermore, contribute to advancing the fast growing industry in the EU as well as creating jobs. The Commission estimated an additional investment of approximately EUR 50 billion for research on energy technologies. It was suggested that this finance should come from different sources, including government and the private sector at both the national and EU level (primarily from the Framework Programme) and the allocation should be centrally coordinated.

The technologies targeted in the SET Plan were divided into three groups, in order to cater for the variety of needs. These technology groups were:

• Close to market competitiveness. Technologies in this groups are on-shore wind, solar heating, solar photovoltaics (c-Si), first generation biofuels, electricity networks (transmission), nuclear fission (G III+), combined heat and power (CHP) and energy efficiency in buildings, transport and industry.

• Emerging technologies on the verge of mass market penetration. These include off-shore wind, solar photovoltaics (thin film), concentrated solar power generation, carbon capture and storage, second generation biofuels and electricity networks (distribution).


• New technologies, such as nuclear fission (Generation IV), hydrogen and fuel cells, ocean energy and geothermal.

There are seven SET plan initiatives described in the SET-Plan Communication of November 2007:

• European Wind Initiative, where the focus is both on- and off-shore applications. In this initiative rapid expansion of wind will be supported across Europe. This will be done by coordinated measurement campaigns, development of testing facilities for new turbine components, demonstration projects of next generation turbines, and testing of new logistics strategies and erection techniques in remote environments.

• Solar Europe Initiative, including photovoltaics (PV) and concentrated solar power (CSP). The aim is to become more competitive and gain mass market appeal in this technological area. To support the development of PV, long-term and large-scale demonstration projects will be undertaken, with up to five pilot plants for automated mass production and a portfolio of demonstration projects for centralised and decentralised PV electricity generation. For CSP, the initiative mainly consists of building up to ten demonstration power plants.

• European Electricity Grid Initiative responding to three interrelated challenges: creating a real internal market, integrating a massive increase of intermittent energy sources and managing complex interactions between suppliers and customers. The focus in this initiative is on the development of a smart electricity system, including storage, and on the creation of a European centre to implement a research programme for the European transmission network.

• Sustainable Bio-Energy Europe Initiative, to bring the most promising bio-energy technologies to commercial maturity. Up to 30 pilot plants will be developed for demonstration.

• The European CO2 capture, transport and storage initiative. Carbon Capture and Storage (CCS) technologies will be widely commercialised. The full CCS chain for a representative portfolio of different capture, transport and storage options will be demonstrated at an industrial scale. In addition, a comprehensive research programme will deliver improved components, integrated systems and processes to make CCS commercially feasible.

• The sustainable nuclear fission initiative, which is mainly focused on the development of Generation IV reactors. These reactors will be designed to maximise inherent safety, increase efficiency, produce less radioactive waste and minimise proliferation risks.

• The Smart Cities Initiative will support ambitious and pioneer cities that would transform their buildings, energy networks and transport systems into those of the future, demonstrating transition concepts and strategies to a low carbon economy. Participating cities and regions will be expected to test and demonstrate the feasibility of going beyond the current EU energy and climate objectives.

The cost estimates of these initiatives are as follows: • European Wind Initiative (EUR 6 bln), • Solar Europe Initiative (EUR 16 bln), • European Electricity Grid Initiative (EUR 2 bln), • Sustainable Bio-Energy Europe Initiative (EUR 9 bln), • European CO2 capture, transport and storage initiative (EUR 10.5 – 16.6 bln), • Sustainable Nuclear Fission Initiative (EUR 5 – 10 bln), • Smart Cities Initiative (EUR 10 – 12 bln).


Additionally a set of technology roadmaps were presented by the first European Industrial Initiatives (EII's), the Initiative on Smart Cities and the Energy European Research Alliance (EERA) during the next 10 years. The roadmaps cover a number of key research areas with cost estimates, as follows:

• Wind energy (EUR 6 bln), • Solar energy (PV and CSP) (EUR 16 bln), • Bioenergy (EUR 9 bln), • Carbon capture and Storage (EUR 10.5 – 16.6 bln), • Electricity Grid (EUR 2 bln), • Sustainable Nuclear Energy (EUR 5 – 10 bln), • Smart Cities (EUR 10 – 12 bln).

Notes The Wind, Solar (PV and CSP), Electricity grid and CCS European Industrial Initiatives were launched in June 2010. The launch signalled agreement and commitment to the technology roadmaps, implementation plans and governance arrangements. The implementation plans and technology roadmap highlight a number of key objectives, targets and indicators for each technology, these can be summarised as follows:

2.2 Wind

Technical KPI's have also been developed from the technology roadmap released in October 200915, the summary technical KPIs for wind from this are shown in the table below.

Activity / Area KPI

Strategic Key Performance Indicator:

Average wind energy electricity production cost reduced by 20% by 2020 N.B. – EII IP – this is to be calculated from Levelised Cost of Electricity and separately for onshore and offshore.

Activities Key Performance Indicators

New turbines and Components

Manufacturing costs of wind turbines and their components reduced by 20% by 2020 Transport and erection costs of on- and offshore wind turbines reduced by 20% by 2020

15 SEC (1295) available at

http://ec.europa.eu/energy/technology/set_plan/doc/2009_comm_investing_development_low_carbon_technologies_roadmap.pdf


Activity / Area KPI

Offshore structure related technologies

Installation costs of offshore wind turbines reduced by 20% by 2020 Maintenance costs of offshore wind turbines reduced by 20% by 2020

Grid integration Virtual capacity factor of wind farms reaching 80%

Resource assessment and spatial planning Wind resources and conditions predicted with an uncertainty of less than 3%

Source: SEC (1295) – these summary KPI's further developed and detailed in Wind EII Implementation Plan The wind energy EII implementation plan16 sets out a detailed action plan to achieve the objectives set out in the technology roadmap. The implementation sets out one overarching Key Performance Indicator (KPI) for the Wind Energy roadmap which will be used by SETIS to monitor its implementation. This is the Levelised Cost of Electricity (LCOE), which will be calculated separately for onshore and offshore wind. A complex formula drawing in all relevant elements has been utilised to calculate the LCOE, this is fully detailed, with assumptions, in the implementation plan document.

The implementation plan has significantly expanded on these objectives, with a large number of draft (to be validated by the wind EII) technical objectives and related KPIs listed in the implementation plan. Action points and a limited number of more project specific KPIs have been devised. The majority of these indicators contain numerical and chronological targets such as increased rotor blade length, weight to power ratios, turbine generator capacity and efficiency gains by specific dates. The measures / indicators can be grouped under the following headings:

• Turbine improvements and design development • Reliability of turbines and wind farms • Optimisation and testing of wind turbines new different terrains. • Standards for large scale wind turbine components • Manufacturing and logistics for large scale turbines. • Demonstration sites • Safety Standards • Grid Connection and Interconnection between MS countries • Offshore and Onshore Wind Solutions

16 http://ec.europa.eu/energy/technology/initiatives/doc/wind_implementation_plan_final.pdf


• Power Plant requirements • Market Integration and Penetration • Research

2.3 Solar (PV)

Solar Energy – PV roadmap set a strategic objective of establishing photovoltaic's as a clean, competitive and sustainable energy source which can provide up to 12% of electricity demand by 2020. The technology roadmap set this objective, along with technical KPI's in October 2009. A summary of the technical KPIs for solar PV from this report are highlighted in the table below.

Activity / Area KPI


PV system

Reduced conventional turnkey PV system cost to <1.5€/Wp by 2020 Reduced concentrated PV system cost to <2€/Wp by 2020 Increased PV (module) conversion efficiency to >23% by 2020 Increased conversion efficiency to concentrated PV to >35% by 2020. Increased crystalline silicon and thin film modules lifetime to 40 years.

Integration of PV-electricity generation Increased inverter lifetime to >25 years by 2020 Battery storage cost <0.06€/kWh and life >25 years

In May 2010, an up-date to this report was launched but the Solar Implementation Plan17 (Solar Europe Industry Initiative) has not significantly expanded and/or refined the original KPIs established in 2009. The primary objective of the KPIs is to support the original strategic objective and support increased market penetration by reducing the cost of PV within the commercial market. Instead, five year targets have been established for each of the indicators in order that real goals for PV technology and electricity can be achieved, and support the interaction between other energy sources.

2.4 Solar (CSP)

Concentrating Solar Power (CSP) has a strategic objective set out in the Roadmap of contributing around 3% of European electricity supply by 2020 with a potential of at least 10% by 2030. Technical KPI's for CSP have been developed from the Roadmap which was

17 http://ec.europa.eu/energy/technology/initiatives/doc/pv_implementation_plan_final.pdf


released in October 2009, the summary technical KPIs for wind from this are shown in the table below.

Activity / Area KPI


Increase efficiency & Reduce cost

Increase solar to electricity conversion efficiency by at least 20% (relative). Reduce cost of installed products and O&M by at least 20% compared with state of the art in commercial plants in 2009

Increase dispatchability Increased performance of storage and hybridisation by at least 20%

Improve the environmental profile Substantial reduction of water consumption with only minor loss of performance Substantial reduction in land use per MW installed

The CSP EII Implementation Plan18 (Solar Thermal Electricity European Industrial Initiative STE-EII) was launched in May 2010. The implementing plan aims to expand on the original technology roadmap objectives, and groups CSP work under four strategic objectives which are; Reduction of Generation, Operation and Maintenance Costs, Improvement of Operational Flexibility and Energy Dispatchability, Improvement in the Environmental and Water-use Footprint, and Advanced Concepts and Designs. The plan has reformulated the CSP but there has been limited variation from the KPIs set out above. Instead, they have been refined and expanded upon into the following two categories:

Activity / Area KPI


Increase efficiency & Reduce cost

Increased efficiency: Increase solar to electricity conversion efficiency by at least 5% (relative). Increase Heat Collecting Fluid Steam Temperature. Reduce costs: Reduce cost of installed products and O&M by at least 10% compared with state of the art in commercial plants in 2009 Lifetime Levelized Electricity Cost €/MWh Number of 'down-time' hours per year (plant reliability).

18 http://ec.europa.eu/energy/technology/initiatives/doc/csp_implementation_plan_final.pdf


Activity / Area KPI

Increase dispatchability

Increased performance of storage and hybridisation by at least 5% Investment cost of storage, €/MWht of stored energy Increase efficiency of storage, % as well as time dependency Decrease size of storage, M3/MWht Increase number of operating hours, based on maximum storage capacity Decrease the cost of produced energy (compared with a similar plant without storage

The recently launched implementing plan demonstrates that the original KPIs have been refined and developed to reflect the specific of the Solar Thermal Electricity sector. The targets have been reduced but still aim to contribute to the 2020 targets and enhance market penetration of the CSP sector.

2.5 Electricity Grids

Resulting from the Technology Roadmap KPI's have been developed for the Electricity Grid sector 2009 and are shown in the table below.

Activity / Area KPI


Number of customers involved (at least 1.5 million)

Greatly increased capacity to host RES electricity from central and distributed sources (to at least 35% of electricity consumption) including readiness for massive offshore wind integration

Increased overall quality of electricity supply (by a 2-10% reduction of energy not supplied)

Reduced peak to average load ratio (by 5-10%) and thus reduced need for investments

Full integration of customers in market mechanisms promoting energy efficiency and active demand practices.

The European Electricity Grid Initiative (EEGI) Implementation Plan19 has sought to build upon the original KPIs but has yet to finalise the draft indicators set out in the table below.

19 http://ec.europa.eu/energy/technology/initiatives/doc/grid_implementation_plan_final.pdf


The EEGI considers there is a need to distinguish between the costs/benefits for grid users (internal) and cost/benefits for externals. The benefits and KPIs will be further examined with the SETIS (SET Information System.

Benefits Potential KPIs

Increased Sustainability Quantified reduction in CO2 emission

Adequate transmission / distribution grid capacity

Hosting capacity of DER Reduction in DER cut-off due to congestion Allowable maximum injection of power without congestion risks

Harmonization and standardisation of grid connection

Reduction in time to connect new user Uniform grid connection rules

Higher security and quality of supply

Reduction peak demand ratio Increased share of renewables Reduction in interruptions per customer Increased voltage quality performance Increase in coordinated operation between TSOs and DSOs Increased efficiency in preventive and emergency control Coordinated restoration after emergency

Enhanced efficiency and better service

Reduction in network losses Increased demand side participation Enabling energy efficiency by end users Hosting capacity of EV Increased availability of network components Actual availability of network capacity Availability of ancillary services across transmission and distribution grids

Effective support for pan-European electricity markets

Increase of cross-border interconnection capacity

Once complete, the KPIs will represent; Level 1 (2020 EU Energy Policy), Level 2 (The Programme), and Level 3 (The Projects).

2.6 Carbon Capture Storage (CCS)

The Technology Roadmap proposed and agreed in October 2009 set a twin strategic objective for CCS EII of enabling cost competitive development of CCS beyond 2020 and assist further develop technologies to allow application in all carbon intensive industrial sectors. To support this, technical KPI's were developed from the technology roadmap released in October 2009, and a summary of the technical KPIs for CCS is presented in the table below.


Activity / Area KPI

Strategic Key Performance Indicator: Cost of CCS technology (including capture, transport and storage) in power plants reduced by 30-40%


Average annual load factor of at least 80% in CCS power plants by 2020

Average annual CO2 capture rate of at least 90% in CCS power plants by 2020

Net efficiency of coal fired power plants equipped with CCS (pulverised coal with post-combustion, IGCC with pre-combustion capture or oxyfuel) higher than 40% by 2020.

Net efficiency of conventional coal fired power plants higher than 50% by 2020.

Publication of an accurate atlas of CO2 storage sites in Europe and an outline of the European transport infrastructure by 2020

First commercial deployment of CCS technologies to energy intensive industrial applications by 2025.

The CCS Implementation Plan20 seeks to expand on the original objectives and associated KPIs. The KPIs are yet to be validated by the CCS team but can be grouped as follows:

Activity / Area KPI

Demonstration and Lighthouse Projects

Cumulative number of Final Investment Decisions taken Gross installed cumulative capacity of CCS projects across Europe Specific capital investment Fixed and variable operating costs Availability Construction time Plant efficiency CO2 capture rate CO2 avoided CO2 stream composition

20 http://ec.europa.eu/energy/technology/initiatives/doc/ccs_implementation_plan_final.pdf


Activity / Area KPI

CO2 transport and storage

Length of pipeline network built Number of Final Investment Decisions Assessment of European Storage Potential Amount of CO2 stored

Public awareness and support Eurobarometer poll

Health Safety, Environmental and Knowledge Sharing KPIs

Number of permits to CCS projects Number of projects in the Project Network


From COM (519) - SEC (2009) 1295 Investing in the Development of Low-Carbon Technologies (SET-Plan) A Technology Roadmap


3 Evolution of calls

The table below lists the subject areas open for bids in each of the four calls launched to date. Table 2 summarises the key points of this: Table 2 – Summary of Calls

Call Budget (€m) No. subject areas open Projects approved

FP7-ENERGY-2007-2-TREN

128 45 26

FP7-ENERGY-2008-TREN-1

147 18 23

FP7-ENERGY-2009-2 100 10 11 under negotiation

FP7-ENERGY-2010-2 126.4 8

Deadline March 2010

15 under negotiation

The change in programme focus following the first call is clear in that the number of subject areas ‘open’ for submissions reduced dramatically. The detail of the subject areas open in each call is shown in annexe one. Table 3: Calls For Proposals by Activity, Area and Topic

2007 2008 2009 2010Level 1 Level 2 Level 3

Called? Called? Called? Called?2.1.1 Enhancing strategic international cooperation initiatives in the field of concentration photovoltaics

2.1.2 Manufacturing and product issues for thin-film photovoltaics

x

2.1.3 multiple benefits of PV systems. x

2.1.6 Alternative approaches for crystalline silicon PV

2.1.7: secure, reliable and affordable supply of feedstock for the pv industry

x

2.1.8: improved production equipment and cost reduction

x

2.1.9: innovative/improved pv manufacturing processes

x

2.1.10: development and demonstration of standardized building components

x

2 Renewable energy generation.

2.1 Photovoltaics

2.1.11: multiple benefits of pv systems x



Called? Called? Called? Called?

2.2.1 Biomass to electricity from energy crops and recovered fuels

x x

2.2.4: large-scale cofiring x

2.2.5: novel solid biofuels for electricity generation x 2.2 Biomass

2.2.6: high-efficiency medium-to-large scale electricity generation from biomass

x

2.3.1 Demonstration of large scale systems for on- and off-shore wind farms including their cost effective grid integration

x

2.3.2: Support to the coordination of stakeholders’ activities in the field of wind energy

x

2.3 Testing, standards and certification for wind energy systems

2.3.4: demonstration of large scale systems for on-and off-shore wind farms

x

2.3.5: integration of collaborative project x

2.3 Wind

2.3.6: wind mapping for offshore applications x

2.4. Geothermal

2.4.1 Increased electricity production from Enhanced Geothermal Systems and from low enthalpy geothermal sources

x

2.5.1.:Improve the environmental profile of the csp installations

x

2.5-02 Using CSP for water desalination 2.5.3: low cost, high efficiency daily storage systems x 2.5.4: improve the environmental profile of the csp installations

x

2.5.5: csp: innovative heat transfer concepts x

2.5: Concentrated solar power

2.5.6: intermediate size, lower concentration ratio csp systems

x

2.6.1. Ocean: demonstration of innovative full size systems

x 2.6.: Ocean energy

2.6-03 Pre-normative research for ocean energy 2.7 Hydro-Electric

2.7-01 New or improved hydro components and concepts

2.9.1. Storage for intermittent electricity x 2.9-1: Demonstration of innovating multipurpose solar plants

x 2.9. Cross-cutting issues 2.9.1 Deep off-shore multi-purpose renewable

energy conversion platforms for wind/ocean energy conversion

3.1.1:Biofuels from high moisture content biomass – biomethane production

x x 3.1: First generation biofuel from biomass

3.1.2: biodiesel from oil crops, animal tallow and used cooking oils

x

3.2.1 - 07 Identifying research needs and technological opportunities for biofuels production in Latin America

3.2.2.:Bioethanol production from lignocellulosics x

3.2-03 High purity syngas cleaning technologies for biofuels

3.2.5: synthetic biofuels via gasification x

3.2 Second generation biofuels from biomass

3.2.6: hydrogenation of oils and fats x

3 Renewable fuel production

3.3 3.3.1 Sustainable Biorefineries x




3.3.2 Enhancing exchange of information, synergies and crossfertilisation between projects in the field of Biorefineries

x Biorefineries (Part supported by TREN) 3.3.3 Developing biorefinery concepts 3.4 New energy crops

3.4-1: Biofuels from algae x

3.5 3.5-01 Fuel production using solar radiation 3.6: biofuel use in transport

3.6.1: demonstration of liquid and gaseous biofuels use in transport/ vehicles

x

3.7.1 Harmonisation of biomass resource assessment

3.7.3: standardisation and sustainability issues x 3.7: cross-cutting issues

3.7.4: promotion and dissemination x 4.1.1 Low/medium temperature solar thermal systems for industrial Process Heat

x

4.1.1: collector design and components x 4.1.2: small scale thermal cooling units x 4.1.3: small distributed systems for seawater desalination

x

4.1 Low/medium temperature solar thermal energy

4.1.4: large scale systems for industrial heat processes

x

4.2: Biomass 4.2-1: demonstration of a new generation of boilers and stoves

x

4.3.1: improved ground source heat pumps x 4.3: geothermal energy

4.3.2: improved underground systems x

4.4: cross-cutting issues

4.4.1: advanced compact storage systems x

4.5.1 Hybrid systems based on solar thermal heating/ cooling, backed up by biomass or geothermal to compensate heat load intermittence

x

4. Renewables for heating and cooling

4.5. Cross cutting issues

4.5-1: support to the coordination of stakeholders' activities in renewables for heating and cooling

x

5.1.3 Advanced separation techniques 5. ? 5.1 ? 5.1.4 Separation techniques in gaseous fuel power

generation (RTD)

5&6.1:Power generation technologies for integrated zero emission solutions

5&6.1.1. :Feasibility and engineering study for development of an integrated solution for a large scale zero emission fossil fuel power plant

x

5&6.2-1: extending the value chain for ghg emissions other than co2 collaborative project

x x

5&6.2.2: support to regulatory activities for zero emission power generation

x

5.2.3 CO2 capture and storage - public acceptance 5&6.2.4: initiating a CO2 value chain in the energy sector using early opportunities

x

5&6

5&6.2: cross cutting and regulatory issues

5&6.2.5: extending the value chain for ghg emissions other than CO2 associated with coal production and use

x

6. Clean coal technologies

6.1 Conversion technologies

6.1.1 Efficiency increases in existing and new build pulverized coal power plants with a view to CCS.

x




6.1.1. :Advanced fluidized bed combustion technology

x

6.1.1: solid fuel gasification development – improvement of gasifier technologies

x

6.1-1: efficiency improvement of oxygen-based combustion collaborative project

6.1.3.:Efficiency improvement of oxygen-based combustion

x x

for zero emissionpower generation

6.1.4.:Advanced gas turbines for solid fuel gasification processes

x

6.2: coal-based polygeneration

6.2.1: poly-generation concepts for coal fired power plants

x

7.1.1: Optimisation of the electricity grid with large scale renewables and storage

x 7.1: Development of inter-active distribution energy networks

7.1-1: large scale demonstration of smart distribution networks with distributed generation and active customer participation

x

7.2.1 Simulation and state estimation of smart electricity transmission networks

7.2.3.: Diagnostics, Surveillance, Maintenance and Control of Power Transmission and Grid Connections

x

7.2: Pan european energy networks 7.2.4.: Assessment of needs for reliable and flexible

future European gas networks x

7.3.1 HTS Devices for Electricity Networks

7.3.3 Stepping up the cooperation of national and regional research activities on Smart Energy Networks

7.3.4: analysis and scenarios of energy infrastructure evolution

x

7. Smart energy networks

7.3: cross cutting issues and technologies

7.3.5: more efficient integration of renewable energy into future infrastructures

x

8.1.1: Energy efficiency in energy intensive industry x 8.1.1: manufacturing industry: wastes and waste heat recovery and transfer

x

8.1.2: manufacturing industry: smes energy innovation

x

8.1: efficient energy use in the manufacturing industry 8.1.3: manufacturing industry: innovative energy

efficient manufacturing processes x

8.2: High efficiency polygeneration

8.2.1.: High efficiency polygeneration - renewable energies for applications in industry

x x

8.4: 8.4.1.: CONCERTO communities: the way to the future

x

8.5: innovative strategies for clean urban transport: Civitas-plus

8.5.1: testing innovative strategies for clean urban transport

x

8: Energy efficiency and savings

8.6: socio economic research and

8.6.1: support action for evaluation and monitoring civitas-plus

x




innovation

8.7.1: promotion and dissemination x 8.7: thematic promotion and dissemination

8.7.2: support action for coordination and dissemination

x

9.1.1 European energy infrastructure networks and systems transition planning

9.1.2 Energy behavioural changes 9.1 ?

9.1.3 Energy technological foresight and scenario development

9.2.1 European scientific multidisciplinary "think-tank" to support energy policy and to assess the potential impacts of its measures.

x

9: Knowledge for energy policy making 9.2: scientific

support to policy

9.2.2: energy technology watch x

10.1.1 Future Emerging Technologies (FET)

10.1.2 Novel materials for energy applications (Joint Call NMP)

10. ? 10.1 ?

10.1.3 Trans-national co-operation among NCPs

SETPLAN

TOTALS 45 18 10 8


4 Participant Survey

A high response rate was achieved in the survey of current FP7 Energy participants from the 2007 and 2008 calls. Of 477 valid contacts, 221 responded, equal to a 46% response rate. The following table presents a breakdown of the respondents by type.

Partners Co-

Ordinators SMEs

Non-SME's

Companies Non-

Companies TOTAL

TOTALS 191 30 57 164 144 76 221

As% of total respondents

86.4% 13.6% 25.8% 74.2% 65.2% 34.4%

TOTAL in FP7 2007 &

2008 455 48 118 385 - - 510

Respondents as %

42.0% 62.5% 48.3% 42.6% - -

This shows that a response rate of at least 40% was achieved in the partner/coordinator and SME/Non-SME sub-groups. The co-ordinators subgroup with a sample of only 30 would ordinarily be of limited value as it is nominally only a small figure, but as it represents over 62% of the whole sub-group this is mitigated, so that the findings remain strongly relevant. Non-companies in this case encompass all sorts of publically funded institutions including local municipalities, universities, research institutions and NGOs. There is some uncertainty with the data and therefore the definition of respondents in this regard, this should be borne in mind when considering this factor.

4.1.1 Project Information

The first 4 survey questions established the basic project information relevant to the survey including the respondents name, organisation, the funded project and project website. Questions 5 and 6, presented below with their results, were designed to provide valuable insight into the various core dimensions of the projects. This is important to enable evaluation of the types of projects being supported against the FP7 Energy objectives. Q5. How would you describe your project in terms of the following dimensions?

• Strategic importance for society • Strategic importance for your organisation • Technical complexity


• Commercial risk • Technical risk • Cost

How would you describe your project in terms of the following dimensions?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Cost

Technical risk

Commercial risk

Technical complexity

Strategic importance for yourorganisation

Strategic importance forsociety

Low Medium High N/A

Q6. How would you describe your project in terms of its proximity to market?

How would you describe your project in terms of its proximity to the market?

Fundamental research

4%

Applied research24%

Prototype development

22%Product testing4%

Demonstration46%


It was expected that a number of trade-offs e.g. cost v risk to become evident from these questions. Among the clearest results identified is that the funded projects are felt to have high strategic importance for society and the participant organisation. The strategic importance for society was felt to be slightly higher than that for the organisation itself, perhaps highlighting that funded projects are felt to be targeted more closely on societal objectives rather than organisational objectives. Over 95% felt their project was of medium or high technical complexity. A significantly smaller proportion (less than 70%) felt that their project was of medium or high commercial risk, while around 80% felt their project was of medium or high technical risk. These findings may point to confidence among researchers that despite projects being complex they still have the resources to overcome the technical and commercial hurdles they face. In respect of cost around 60% of respondents felt their project was of medium cost, 10% low and almost 30% high. In respect of proximity to market it was clear that the 3 main project types supported are demonstration, prototype development and applied research projects, together constituting over 90% of the total. Almost half of respondents considered themselves to be at the demonstration stage, which is in line with the project types as classified in the call analysis.

4.1.2 Relevance and Effectiveness

This section examined participants understanding of the nature and objectives of FP7 and how their project fits with this. The intention of Question 7 was to establish the level of understanding of programme participants when applying, to evaluate the relevance of the programme and how well the objectives of the FP7 Energy programme are presented and known. Questions 8, 9 and 10 examined the level of match between the project and different sets of wider objectives. Identifying the level of match is important, as if the FP7 Energy programme is to achieve its objectives, there should be a match between the objectives of the projects it funds and the programme objectives. The identification of which project objectives most closely and commonly match programme objectives is also interesting. As a follow-up to this Questions 11 and 12 provide direct comment from respondents on the relevance of the FP7 Energy programme to achieving the stated objectives and how this has evolved over time. This final element is particularly interesting to note the perceived evolution of the programme and what respondents views on this are. This is useful to evaluate the perception among respondents of any changes due to increased alignment with the SET plan and what they think of these changes – this is also relevant to evaluating FP7 Energy consistency and acceptability. Questions 13 and 14 provide information on whether the respondent has been a previous recipient of European or national level funding for energy research and development. This is important to identify the extent to which past participants re-apply and the relationship between national and EU funding. This can inform an evaluation of additionality and the complementary nature of FP7 Energy support.


Q7. How well did you feel you understood the FP7 Energy programme objectives when applying?

How well did you feel you understood the FP7 Energy programme objectives when applying?

Basic understanding

23%

Moderately well45%

Very well31%

Poorly1%

Not at all0%

It was expected that participants would have at least a basic understanding of the FP7 Energy programme, the results show that almost 99% of participants claimed to have at least a basic understanding of the programme, almost 1/3 believed they understood the programme objectives very well. This points to widespread recognition and understanding of the FP7 Energy programme objectives among applicants. Q8. How closely do your project objectives match the following high level FP7 objectives?

• Achieving a secure and diverse EU Energy Supply • Reducing GHG emissions and increasing the sustainability of energy supply • Improving the economic competitiveness of EU industries


How closely do your project objective match these high level FP7 objectives?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Achieving a secure and diverseEU Energy Supply

Reducing GHG emissions andincreasing the sustainability of

energy supply

Improving the economiccompetitiveness of EU

industries

No match Slight match Some match Good match Very close match N/A

It would be expected at this high level that there should be at least a good match between high level FP7 objectives and ultimate project objectives. Responses from participants showed a belief that high level objectives matched very closely, particularly in respect of reducing GHG emissions and achieving a secure and diverse energy supply. Responses were more mixed on the match with improving economic competitiveness but still over 75% felt there was a good or very close match between project and programme objectives. These results point to a high level match in purpose between project and programme objectives and therefore that the funded projects should make a contribution towards achieving these high level FP7 Energy objectives. Q9. How closely do your project objectives match the following energy specific FP7 objectives?

• Increase energy generated from renewable energy sources • Improved energy sector competitiveness • Contribute to achieving wider energy policy goals e.g. 20-20-20 targets • Developing technologies that can achieve short term market penetration • Increase whole energy generation system efficiencies • Build interfaces between new and existing technologies


How closely do your project objectives match the following energy specific FP7 objectives?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Increase energy generatedfrom renewable energy

sources

Improved energy sectorcompetitiveness

Contribute to achievingwider energy policy goals

e.g. 20-20-20 targets

Developing technologiesthat can achieve short term

market penetration

Increase whole energygeneration system

efficiencies

Build interfaces betweennew and existing

technologies


The different level of objectives in this question would be expected to produce a more nuanced set of results and draw out particularly strong or weak objective matches. Against the energy specific objectives, in every case at least 65% of respondents felt there was a good or very close match between their project and the FP7 programme objectives. The objective with the lowest % of good/very close matches is the 'developing technologies that can achieve short term market penetration' objective, highlighting that this is not necessarily a primary project objective. This is understandable given the results of Q6 which although showing high proportions of projects at demonstration and product development stages also show a significant minority of around 30% that are at fundamental or applied research stages and therefore further from market. An interesting result is that only 13 of the 222 respondents to this question believed there was no match between their project and the objective of increasing energy generated from renewable energy sources. Given that around 40% of project funding is directed at non-renewable energies the proportion here could have been expected to be higher. This may point to projects in areas such as energy efficiency and smart networks seeing their role as an enabler for renewable energy. Q10. How closely do your project objectives match the following commercial and research related FP7 objectives?

• Enhance EU research and development capacity • Leverage private investment • Support trans-national co-operation • Provide tools and information for policymakers • Securing industry commitment • Specific technology development


How closely do your project objectives match the following commercial and research related FP7 objectives?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Enhance EU research anddevelopment capacity

Leverage private investment

Support transnational co-operation

Provide tools and information forpolicymakers

Securing industry commitment

Specific technology development


The results from this set of objectives show varying levels of match. The strongest objective match, with over 88% seeing a good or very close match, is supporting transnational co-operation. Specific technology development and enhancing EU research and development capacity were also believed to strong matches, with around 75% seeing a good or very close match in each. The other 3 objectives each saw this good / very close match level fall to around 60% of respondents with providing tools and information for policymakers seeing the lowest match. The policymakers information objective also saw the highest combined levels of no or only slight match at 14% of respondents, highlighting that this seemed to be the least relevant objective at project level. Leveraging private investment saw the highest numbers of respondents (11 out of 222 or 5%) declaring that there was no objective match. Q11. What is your view on the appropriateness of the FP7 Energy programme to achieving the objectives above, given the wider economic, social and environmental context?


What is your view on the appropriateness of FP7 Energy to achieving the objectives above, given the wider economic, social and

environmental context?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Appropriateness

Not appropriate Moderately appropriate Highly appropriate Don't Know

The results show that over 70% of respondents feel that the FP7 Energy programme is appropriate to achieve the objectives listed in the previous questions. None of the respondents felt the FP7 programme was inappropriate. This would appear to be a very strong endorsement of the FP7 energy programme as an instrument to achieve the stated objectives. Q12. And how has this changed since:

• Previous Framework Programmes (FP4, FP5, FP6) • The start of FP7

And how has this changed since:

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Previous FrameworkProgrammes (FP4, FP5,

FP6)

The start of FP7

Become less appropriate No change Become more appropriate Don't Know

The results show that a significant proportion of respondents, over 50% in the case of previous Framework Programmes and over 40% in the case of FP7, are unsure of any changes in appropriateness. Of those that did have a view, in the case of previous Framework Programmes over 75% believed that FP7 had become more appropriate than previous FPs.


Since the start of FP7, those that had a view were split approximately 50-50 between those that believed the programme had become more appropriate and those that believed there had been no change. These results point to an understanding among previous participants that the programme is becoming more appropriate to achieving the stated objectives but that a similar change since the start of FP7 is less clear. At the same time a large proportion of participants don't know either way how the appropriateness may have changed, this may in part be due to only participating in FP7, or that they feel it is too early to give a view on changes since the start of FP7. Q13. Has your organisation received any EU Energy research funding prior to your FP7 project?

Answer Options Response Percent

Response Count

No 37.4% 83

Previous FPs (FP4, FP5, FP6) 54.5% 121

Other energy programmes (e.g. Intelligent Energy Europe) 22.5% 50

Other programmes (please specify) 12.6% 28

answered question 222

The results show that over half of the respondents had taken part in previous FP programmes and that in total over 60% of respondents had received some form of EU Energy research funding prior to taking part in FP7. It also shows that over 1/3 of participants are new to EU Energy research funding. Q14. Has your organisation received any national funding for energy related research in the five years prior to this application?


Response Count

No 26.1% 58

Yes, 1-5 projects 43.7% 97

Yes, 6-10 projects 4.5% 10

Yes, > 10 projects 18.0% 40

Don't Know 7.7% 17


The results show that almost 75% of respondents had received some form of national funding for energy related research in the last 5 years. Nearly 1 in 5 respondents had received funding for over 10 projects. Summary Overall the results of Q8, 9 and 10 showed that the level of match between project and programme objectives was rated good or very close was over 80% in most cases, and the lowest level of match at this level was still 58% (for providing tools and information for


policymakers). It could therefore be clearly said that respondents felt their projects had a good or very close match to FP7 Energy objectives in the great majority of cases. Questions 11 and 12 indicated that the FP7 Energy programme is strongly felt to be an appropriate way of achieving the stated objectives and that it is more appropriate than previous framework programmes. The results from questions 13 and 14 show that around 1/3 of participants are involved in an EU Energy related funding programme for the first time, but also that over half have been involved in a previous FP programme, while almost 75% had received funding nationally. The high proportion of those with previous FP experience points to this being beneficial to making a successful application, but at the same time the significant minority without any prior FP experience, highlights that prior experience is not an essential requirement and that the programme remains accessible to new entrants.

4.1.3 Efficiency

The questions in this section were designed to produce data on the administrative load of preparing a bid for FP7 Energy funding and, following participants success in this, in project delivery. This is useful to examine the level of administrative burdens which will be indicative of the efficiency of programme delivery. Comparison to previous FP programmes will enable analysis of the trend for efficiency. To achieve this questions 15 and 16 query time spent in preparing the FP7 Energy project application and questions 17 and 18 on time spent to administer the project since it began. Question 19 explores participants satisfaction with specific elements of programme implementation and question 20 how participants experience of the programme compares to previous Energy Framework Programmes. The qualitative measures used in a number of these questions allows for analysis of project participants relative perception of administrative burdens, which is also an important factor in evaluating efficiency. The questions in this section can also assist in evaluating relevance and effectiveness Q15. How much work was required to prepare the application? (Approximate number of person days)

Answer Options Response Count answered question 201

Average 45 days Median 25 days

Q16. Regarding the time you spent on the application, did you find this?

Answer Options Low As

expected High Don't Know

Response Count

Did you find this?

4 121 75 11 211

The results of the questions on the time taken to prepare the applications show that on average each organisation had to spend an estimated 45 person days to prepare their applications. This figure is somewhat skewed by a number of high responses including one respondent that took almost 2 years (730 person days) to prepare their application. It is noteworthy that 20 respondents reported requiring over 100 person days to prepare their


contributions to applications, yet 58 respondents required 10 days or less. Taking the skew from the highest estimates into account the median value of 25 person days may give a more representative view on approximate person days per application. Over half of respondents felt that the time spent on the application was in line with their expectations, but around 35% found the time spent higher than expected. Q17. How much work has been required so far by your organisation relating to Commission required management and administration of the project? (Approximate number of person days)

Answer Options Response Count

answered question 191 Average 44 days Median 20 days

Q18. Regarding the time you have spent on administration, did you find this?

Answer Options Low As

expected High Don't Know

Response Count

Have you found this? 5 129 68 9 211

The results of the questions on the time taken to satisfy FP7 management and administration requirements show that on average each organisation had to spend an estimated 44 person days annually. This figure is again somewhat skewed by a number of high responses including two respondents that took around 2 years (700 and 800 person days) to administer their projects, though both of these regarded this length of time 'as expected'. It is noteworthy that 23 respondents reported requiring over 100 person days for project administration, yet 75 participants required 10 days or less. Taking the skew of these higher entries into account the median value of 20 person days may give a more representative view on approximate person days for project administration. Over 60% of respondents felt that the time spent on project administration was in line with their expectations, but around 30% found the time spent higher than expected. Q19. How satisfied have you been with the following aspects of programme implementation?

• Application procedures • Proposal evaluation procedures • Time from project approval to contracting. • Project payment arrangements • Overall project administration and communication • Strategic orientation • Scientific and technological areas covered • 'User friendliness' of FP7 compared with other schemes


How satisied have you been with the following aspects of programme implementation?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Application procedures

Proposal evaluation procedures

Time from project approval tocontracting

Project payment arrangements

Overall project administration andcommunication

Strategic orientation

Scientific and technological areascovered

User friendliness' of FP7compared with other schemes

Very dissatisfied Dissatisfied Neither Satisfied or Dissatisfied Satisfied Very Satisfied No Opinion

The results from this question show that generally participants are satisfied or very satisfied with these aspects of programme implementation, with combined satisfaction rates of over 60% in all but 2 of these areas. Participants were most satisfied with the scientific and technological areas covered, the strategic orientation of the programme and project payment arrangements, with over 70% either satisfied or very satisfied. The 2 areas with lower satisfaction reported are the time from project approval to contracting and 'user friendliness' of FP7 compared with other schemes. While the lower satisfaction rates for user friendliness can largely be explained by a large proportion of respondents with no opinion (n.b. over 60% of those with an opinion were satisfied), the time from project approval to contracting appears to be the main area of dissatisfaction. Only just over 40% were satisfied or very satisfied with the time from project approval to contracting, while 22% were either dissatisfied or very dissatisfied. Although these figures appear high it is interesting to compare them with the replies to the same questions from out ex-post of TREN FP5 energy projects. In that survey 39% were ‘not satisfied’ with the payment arrangements, significantly more than the 22% not satisfied in this survey. Q20. Overall, how does this compare to your past experiences, if any, of the FP Energy programme?


Overall, how does this compare to your past experiences, if any, of the FP Energy programme?

Much worse

Worse

No Change

Better

Much better

Of the 119 respondents with past experience of FP Energy funding over 57% felt their experience this time was better, and for 6% much better. So around 2/3 of respondents believe their experience of programme implementation has improved. Around 30% believed there had been no change from their past experience. Around 6% believed their experience was worse than past experiences, but not believed their experience was much worse. These suggest the majority view among participants that their experience of the programme is improving, whether this is down to their own learning from past experience or improvements in programme implementation is unclear from this data. Summary The findings of the survey indicate that participants in the FP7 Energy programme typically spend between 20-45 person days each in preparing their application and meeting the management and administration requirements of the programme. For many the burden is lower but for a small minority of participants (around 10%) the time required is over 100 person days. Overall the programme participants appear broadly satisfied with the various elements of programme implementation and particularly with the scientific and technological areas covered, the strategic orientation of the programme and project payment arrangements. There was one clear area where satisfaction was much lower and this was the time from project approval to contracting. For those with prior experience with the Energy Framework Programmes a majority believe that their experience of FP7 is better than in the past and only a very small minority believe it is worse. The consistency in categories and questions asked here will allow further analysis through cross-referencing the findings of this survey against previous FP evaluation surveys, to chart any improvement or decline in programme efficiency over time.


4.1.4 Results and Outputs

The questions in this section were designed to give insight into the nature of expected results and project outputs. It is important to have views on the expected outputs at project level as these should correspond to the programme level intervention logic. Looking beyond this to the potential scaling up or global impacts a project could have is also an important objective and therefore evaluation criteria. This is particularly relevant in respect of the top level EU and FP7 Energy objectives. Questions 21 and 22 provide insight into the types of expected outputs, these are key factors in evaluating overall programme effectiveness, efficiency, utility and distributional effects. Questions 23 and 24 explore the potential energy saving and/or CO2 reduction benefits of projects, and also the potential for the scaling up of these benefits. Q21. What specific types of output(s) is your project likely to produce?


Response Count

Prototype, Demonstrator or Pilot 69.5% 146

New or improved process or technique 65.7% 138

Publication in refereed journal (peer reviewed) 50.5% 106

Conference 46.2% 97

New or improved product 41.4% 87

Policy and/or technical advice or recommendations to external bodies 40.5% 85

Exploitation product (e.g. patent, copyright, trademark, license, registered design) 29.0% 61

Production of electronic outputs (please specify reports, data sets, codes, shareware, other software via web, CD-ROM etc.)

29.0% 61

New or improved service 24.3% 51

Other (please specify) 12.4% 26

Other

Sharing of knowledge and experience, building of networks

Others were provided but have not been reported in order to protect anonymity.

The results of this question showed that almost 70% of respondents expect to produce a demonstrator, prototype or pilot as a result of their project. Around 65% expect their project to produce a new or improved process or technique, while over 50% expect their project to produce articles published in peer reviewed journals. Among the other expected outputs an interesting result was that only 29% expect an exploitable product to be among the project outputs. There are a variety of reasons that this may be the case, which may be worth exploring further. Among the other types of outputs listed by respondents the key outputs were informational – items such as training, guidelines, country reports, case studies and standards. It is interesting to compare these expectations of results with the actual results reported by FP5 participants. In the FP5 survey some 40% had published in a peer reviewed journal, so if


the 50% who expect this to occur do achieve it this will be an improvement. If 30% of the products are able to lodge a patent this will be a significant improvement from the actual results of FP5 where only 13% did this. Q22. In your expectations, what will be the main technical result(s) of your project?


Response Count

Bring demonstration technology closer to large scale deployment 56.2% 118

Validation of energy technology concept through demonstration at full scale 51.9% 109

Improvement of existing commercial energy technologies 32.4% 68

Integration of new and existing technologies e.g. Buildings integration of Solar PV 31.9% 67

Advances over state of the art – i.e. completely new technology, significant breakthrough 25.2% 53

Other (please specify) 8.1% 17

None 3.3% 7

Don't know 2.4% 5

Other

assessment of new technologies for long term emissions abatement


In narrowing the scope of the question to the expected technical outputs the 2 most expected outputs relate to demonstration technologies, which as Q6 showed is the most numerous type of project funded. Over 56% of participants expect their project to bring a technology closer to large scale deployment and around 52% expect their project to validate a technology concept through full scale demonstration. Around 30% each feel their technical outputs will be improvements of existing commercial technologies or the integration of new and existing technologies. Just over ¼ of participants feel that their outputs will represent an advance over state of the art, representing a significant breakthrough or new technology. Interestingly around 5% of respondents expect no technical outputs or don't know what their outputs will be. The shift in the nature of the programme towards demonstration is apparent when comparing these answers to those recorded for FP5 where it was the case that many more of the projects thought they had produced an advance over state of the art. Q23. Are you able to estimate the CO2 emissions savings and / or the energy savings your project itself will enable (if successful)?

Answer Options Response Percent Response Count

Yes 30.0% 63


No 70.0% 147

If Yes, can you provide an Approximate figure (tonnes of CO2 or GWh) 49

Approximate figures

60 Mte/annum of CO2

CO2 savings 500 to 750 megaton per year in 2050

Not yet, too early

5000t CO2/y

70% GHG emissions reduction in the Hybrid concept (existing facility plus 2nd generation)

shall provide the reference economic evaluation method & tool for CCS projects: without a good tool, no CCS.

5 GWh, assuming 10 year lifetime

No

12 million tonnes of CO2 per year for Sweden, about 36 million tonnes for EU

112 MWh/day

CO2 emission savings: - 2 ton CO2/ton EtOH

50'000 tCO2 per year per plant (of 350MW size)

0,6 GWh

~ 90.000 tn/year

at least 1 000 000 tonnes CO2

At end of project when sustainability modeling tools are ready

54.000 t CO2 per year

~50000 tonnes CO2/year

13000 tons/year

640 tonnes of CO2 / year

900000 t/year

25000 CO2-e/year at full capacity

Demonstration plant 30-40 000 tons, dissemination of the technology, millions of tons

7000 ton/year

3000 tons CO2 / year

there will be very high savings but it is too early now to provide figures

4000

Up to 40% energy and CO2 savings compared to conventional technologies.

80% per tractor

In commercial plant over 90% of CO2 produced will be captured

Presented by VTT in the application

"Well-to-wheel" analysis performed for full-scale technology (~95 CO2 reduction), but not for pilot scale size.

Tens of Millions of tons when applied


not yet, it depends on the technology

5000CO2 t/a

1800 tons

Reduce the carbon footprint of Wave Energy Converter manufacture/installation by a factor of 50% due to the anticipated increase in annualised average electricity outputs mean fewer devices will be required. The number will depend upon the footprint of each specific device.

10 000 000.00

Unknown yet

4000

40.000 tonnes of CO2/year, 6,2 tonnes/person 6-7.000 new CO2-neutral citesents

30 million tonnes CO2 / a

5 tonnes per year

I would prefer co-ordinators answer to this

21,3 GWh/year for the demonstration in Italy

0,7 tCO2 avoided per MWe

200000 tonnes of CO2

3% total energetical fossil fuel reduction (e.g. about 2 Mio. t Diesel/a in Germany)

N/A yet

The results to this question show that only 3 out of 10 respondents were able to estimate what the carbon or energy savings that may occur as a result of their project. Of the 49 respondents that provided an estimate of some form, only around 20 of these provided any useful quantitative estimates. For only 10% of programme participants to be able to estimate in this way is somewhat worrying, especially considering that reducing carbon emissions is one of the key, perhaps the most important, high level objectives of FP7 Energy. Further analysis of the responses for this question shows that of the 30 project co-ordinators that responded to this question, 13 of them (43%) were able to estimate the CO2 reductions or energy savings that would result from their project. Of the 191 partners that responded, 49 (26%) were able to estimate the CO2 reductions or energy savings. This appears to show that project co-ordinators have a better understanding than project partners of the likely outcomes of their project in terms of CO2 reduction or energy saving, though less than half were still able to provide an estimate. Q24. What is your view of the approximate EU and global potential scale of replication of your project?

Answer Options x1-10 x10-100 x100-1000 x1000 + Don't

Know Response

Count

National 80 27 19 12 66 204

EU 26 59 27 31 66 209

Global 14 37 32 45 74 202


Other (please specify National / EU / Global) 14

Other

Not applicable: our project was a concerted action aimed to share existing information


This question also highlighted a gap in knowledge among participants, with around 35% of respondents in each case unsure (don't know) of the replication potential of their projects. There were indications from respondents that some did not understand the question. Of those that were able to estimate the replication potential of their project the answers showed that most respondents (58%) felt that nationally the replication scale of their projects was only x1-10. At the EU level it was felt by over 40% of participants that replication scale was in region of x10-100, though over 20% put this at x1,000+. At the global level over 35% felt that replication scale was over x1,000+, and over 25% each in x10-100 and x100-1,000. The survey returns in questions 23 and 24 are not sufficient to provide a clear view of overall potential carbon or energy savings of FP7, given the survey audience there is potential double counting and other factors to consider. These questions though potentially do give insight into some important issues around projects not having clear estimates of the carbon or energy saving potential of their project or the wider potential of the project outputs. It is probable that a number of respondents did not understand the question, this in itself is worthy of comment as it suggests that the idea of the project being to ultimately reduce carbon emissions is not the way in which they think about their project.

4.1.5 Project Impacts

These questions were designed to capture the expected longer term impacts of the projects and although for most participants it will be early in their project life, they will still have expectations of impacts. Evaluating the fit between these and the programme intervention logic and objectives is important. In addition, aggregation of impacts at EU level in areas such as high level EU policy, science and technology, commercial competitiveness and environmental policy are also important evaluation criterion. Given the nature of the projects supported by FP7 Energy, i.e. relatively near to market, it is expected that there would be a spread of benefits across the various areas, with the commercial elements expected to be particularly relevant. These elements will be central to evaluating the programme against the economic and competitiveness objectives. In evaluating the broader impacts of the programme it is necessary to consider that at an individual project level the responses will reflect low levels of significance but at an aggregate level these effects may be multiplied. Questions 25 and 26 explore the expected significance of the impacts of projects on participants organisations in both the scientific/technical and commercial dimensions. Question 27 further explores the commercial benefits, if any, of the FP7 Energy project. Questions 28, 29 and 30 go beyond the organisational benefits expected from project participation and explore the extent of wider impacts in EU and FP7 Energy policy and objective areas.


Q25. How significant do you expect the impacts of the project on your organisation to be in the following scientific/technical areas?

• Enhanced reputation and image • Ability to carry out new activities or enter new areas • Enhanced ability to produce or deliver new products, processes or services • Other (please specify)

How significant do you expect the impacts of the project on your organisation to be in the following scientific / technical areas?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Enhanced reputation andimage

Ability to carry out newactivities or enter new areas

Enhanced ability to produceor deliver new products,processes or services

N/A Insignificant Low Medium High Don't know

Other field of impact

Enhance cooperation with other EU research organisations


The results to this question show clearly that participants believe that their project will have a significant impact on their organisation with over 80% expecting a medium or high impact, in each of the three scientific and technical areas, and particularly in enhancing their scientific/technical reputation and image. Looking more closely at the responses to detect any variation in response by whether the organisation was a partner / co-ordinator, SME or Non-SME, or Private / Public shows that in every case the project co-ordinators are more confident (though not by a lot) than project partners in positive outcomes in the science and technical areas. The same is also the case for public organisations compared to private companies. This is even more so in comparison between public organisations and private companies, with the public organisations 15-30% more confident of highly significant outcomes. In contrast to this, for SME's there is little noticeable difference in expected outcomes, only in the case of enhancing the ability to


deliver new products, processes or services was a small difference found, with non-SME's slightly more confident of medium significance outcomes. Q26. How significant do you expect the impacts of the project on your organisation to be in the following commercial areas?

• Improved competitive position through increased market share or access to new markets

• Increased profitability • Enhanced productivity • Improved linkages with other organisations • Formation of new business entities • Other (please specify)

How significant do you expect the impacts of the project on your organisation to be in the following commercial areas?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Improved competitive positionthrough increased marketshare or access to new

markets

Increased profitability

Enhanced productivity

Improved linkages with otherorganisations

Formation of new businessentities


Other field of impact

Remark: we are a government (and hence a non-profit) organisation


The results to this question show a more mixed view of the potential commercial impacts of the participants project involvement for their organisation, this is not unexpected with many of the organisations involved being publically funded or non-profit universities or research institutions. The most significant impacts are expected in improving linkages with other organisations and improved competitive position through increased market share or access to new markets, with 87% and 62% respectively, expecting impacts of medium or high significance. The transnational and consortium approach of the FP7 programme would be


expected as a matter of course to improve organisational linkages, it is interesting to note that participants believe this will have significant impact on their organisations. The widespread expectation of significant impact on competitive position is also an interesting result, suggesting the FP7 Energy programme will have impacts on high level competitiveness objectives. Set against this are more mixed expectations in respect of the significance of impact on increasing profitability, enhancing productivity and the formation of new business entities. In each of these areas the number of participants who feel that the impacts will be low or insignificant is around 35% while those who believe the impacts will be medium or high is around 45%. So although there is some bias towards significant impacts there is a large minority of participants that don't expect significant impacts. Further analysis on the basis of respondent type reveals the following against the various commercial areas:

• Improved competitive position through increased market share or access to new markets – Project partners were a little more confident than project co-ordinators of medium to highly significant returns in this area (76% to 61%), the same is also true for SME's in comparison to non-SME's (70%-60%). A significant difference was found between private companies and non-companies, with private firms much more confident than non-companies of medium or highly significant impacts in this area (71% to 47%).

• Increased profitability – Project partners were much more confident than project co-ordinators of medium or highly significant impacts in terms of profitability (60% to 32%). SME status had very little impact on the responses. In this case companies were slightly more confident of medium to highly significant impacts than non-companies (50% to 40%)

• Enhanced productivity – there was little variation from the total proportions within or between the subgroups in respect of productivity.

• Improved linkages with other organisations - there was little variation from the total proportions within or between the subgroups in respect of improved linkages with other organisations.

• Formation of new business entities - there was little variation from the total proportions within or between the subgroups in respect of the formation of new business entities.

Overall, it can be said that project partners are more confident than project co-ordinators of significant commercial impacts in respect of improving competitive positions and increasing profitability. A similar difference is also noted between private companies and non-companies, with private firms more confident of significant positive impacts against these 2 commercial objectives. SME status appeared to have little or no effect on the respondents expectations for commercial gain. In respect of productivity, linkages with other organisations and the formation of new business entities there was very little variation based on sub-groups. Contrasting these results with the similar analysis for Q25 appears to show that SME status has very little effect on the expectations of impacts of involvement in FP7 in relation to science, technical or commercial goals. A difference that is noted is that project co-ordinators


seem more positive about scientific/technical impacts than commercial, while the reverse is true for project partners. As would be expected given the classifications, non-companies (public) organisations are more confident of significant scientific/technical impacts, while private firms are more confident of significant commercial impacts. Q27. What type of commercial impact(s) do you expect your project to enable in the following markets?

Answer Options None

Improved market share

Improved profits

New partners to work

with

No direct commercial

return Don't know

Response Count

Local 47 35 28 64 40 36 205

Regional 36 40 36 81 35 34 204

National 18 65 46 98 26 30 207

EU 13 70 46 120 23 29 208

Global 17 51 36 80 26 55 201

The results show that at the Local through to the EU level around 15% of participants don't know what the commercial impacts of their project to be, at the Global level this increases to over 27%. Of those that did have a view on commercial impacts over 25% expected no impact at the local level, 20% none at the regional level and around 10% at all other levels. Among these, the EU level was the level with the lowest number (7%) that expected no commercial impact. The results here are broadly similar to those who expect no direct commercial return from their project involvement, which is a logical match. The most widely expected commercial impact was that project involvement would provide new partners to work with, this was particularly expected at the EU level (58% of respondents). The next most expected commercial impact was for improved market share but with only 17-34% of respondents expecting this at the various levels, again with impact expected to be greatest at EU level. Improved profits at the various levels were only expected by 14-22% of respondents. The results here point to commercial benefits being somewhat limited beyond improving the potential for partnership at various geographic levels. Q28. How significant do you expect the wider impacts of the project to be against the following high level EU policy objectives?

• Employment • Education • Quality of life and health and safety


How significantdo you expect the wider impacts of the project to be against the following high level EU policy objectives?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Employment

Education

Quality of life and health andsafety


The results of this question show that against each objective over 50% of those surveyed expected the wide impacts of their project to have an impact of medium or high significance. The expectation of an impact of this size is highest in respect of Quality of life and health and safety at 72%, while lowest for employment at 56%. For each of the 3 objectives the expectation of medium impacts was the same at around 35%, the main difference arises in expectation of high impacts. In each case around 7-10% of respondents didn't know what wider impacts their project may have against the high level objectives. Q29. How significant do you expect the wider impacts of the project to be against the following science and technology related EU policy objectives?

How significant do you expect the wider impacts of the project to be against the following EU science and technology related policy

objectives?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Equal opportunites for all andgender balance in science

Improved public acceptanceof new technology / approach

Develop international scienceand technology co-operation

Enhance skills of RTD staff

Development of world classexpertise in a research area

Enhancing EU Energytechnology leadership


The results from this question show that for every objective, except Equal opportunities for all and gender balance in science, over 75% of respondents believed that their projects wider


impacts would be of medium or high significance. This points to a very strong belief among respondents that their projects will make significant contributions to achieving the EU's science and technology policy objectives. Q30. Do you expect your project to have any benefits in terms of the following areas of environmental policy and what is the scale of this benefit?

Do you expect your project to have any benefits in terms of the following areas of environmental policy and what is the scale of this

benefit?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Improved energy efficiency

Reduced energyconsumption / demand

Increased share of new andrenewable energy source

Reduced impact onbiodiversity, soils, forestry

Reduced impact of energyproduction/use/waste

Reduced CO2 emissions

Disbenefit None Low Medium High Don't know

It would be expected that the energy projects funded by FP7 will have positive impacts on the environment. The results to this question highlight that in all areas, but to varying degrees, respondents believe their project will have environmental benefits. Importantly in the area of reducing CO2 emissions, over 85% of respondents expect their project will have medium or high benefits, with almost 68% of the total believing the benefits will be high. This is very strong backing for an expectation of carbon savings and contrasts with the findings of Q23 and Q24, highlighting that while estimates of savings and potential for scale-up are not clear to respondents, they expect their project to result in large scale CO2 reduction benefits. Similarly high proportions also expect their projects to have significant benefits in improved energy efficiency and increasing the share of new and renewable energy sources. Interestingly 3 respondents believed their project would be a disbenefit to the new and


renewable energy sources objectives, closer investigation reveals that these respondents are projects under the Clean Coal activity area. More mixed views on environmental benefits were found against the reduced energy consumption / demand and reduced impact on biodiversity, soils and forests objectives. In each over 50% still expected medium to high scale environmental benefits but at the same time over 35% believed that the environmental benefits would be either dis-benefits, no benefits or low benefits. The reasons behind these lower expectations is potentially an interesting area for further analysis. Overall there was evidence of widespread expectation of significant environmental benefits resulting from the projects.

4.1.6 Other comments

Question 31 asked respondents if they were willing to take part in an in-depth project profiling, over 50% of respondents indicated they would be happy to do this. Q32. Do you have any other comments on your experience with the FP7 programme and DG ENERGY? This question elicited over 30 responses as highlighted below. Any other comments.

I find that in some cases the delays in payment is much too long. I also found (in one project unrelated to the ...


4.2 Past Beneficiary Survey

This shorter and smaller survey was designed to examine the reasons that participants in the FP6 Energy programme have not been involved in the FP7 Energy programme. The first 3 questions established the respondents name, organisation and FP6 project, Question 4 confirmed that they participated in FP6 Energy but not FP7 Energy. It was also asked what the reasons for this were. Q4. Can you please confirm that your organisation participated in the FP6 Energy Programme but has not participated in the FP7 Energy Programme.


Response Count

Yes, participated in FP6 Energy but not FP7 Energy 80% 32

No - Please specify reason below (e.g. we did not participate in FP6 Energy, we are participating in FP7 Energy or a non-Energy FP7 project)

20% 8



Response Reason

NO We are part. in FP7

FZJ is largest research centre in Europe: other unit might participate

We are also participating on FP7 through the PV TP-SEC/ Grant agreement no.: 241377

We participate both in FP6 and FP7 projects

We are just under negotiation with a FP 7 Demoproject

We are participating in non-Energy FP7 project

we are looking to present a new proposal

Yes, Other We are mainly involved in NMP


Q5. Please rate the following factors as reasons for not applying to the FP7 Energy programme.

• No appropriate project has arisen • Organisational focus has moved away from R&D • Now focussing on private sources of R&D funding • Now focusing on national (public) sources of R&D funding • Now focusing on other EU sources of R&D funding • No FP7 funding in your area of research • Insufficient funding available from FP7 • Scale of FP7 projects now too large for your requirements • Scale of FP7 projects now too small for your requirements • Unable to find trans-national partners • Requirement for a consortium approach is a disincentive • Experience of FP6 is a disincentive • Other (please specify)


Please rate the following factors as reasons for not applying to the FP7 Energy programme

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

No appropriate project hasarisen

Organisational focus hasmoved away from R&D

Now focusing on privatesources of R&D funding

Now focusing on national(public) sources of R&D funding

Now focusing on other EUsources of R&D funding

No FP7 funding in your area ofresearch

Insufficient funding availablefrom FP7

Scale of FP7 projects now toolarge for your requirements

Scale of FP7 projects now toosmall for your requirements

Unable to find trans-nationalpartners

Requirement for a consortiumapproach is a disincentive

Experience of FP6 is adisincentive

Don't Know Not a factor Minor factor Major factor N/A

Other (please specify)

Too much bureaucracy: waste of time and resources: major factor

We have applied for two projects unsuccessfully are two other proposals are being considered. Very few possibilities exist within the calls.

No project opportunities for cities, no more Civitas or Concerto

I was not invited to participate to any project

Main reason for not participating was that either no appropriate topic were mentioned in the call or the way how they were mentioned was not appropriate.

Lack of internal resources to work up ideas to project stage.

General economic situation in ….does not give room for participation.

The results of this question show that by far the most important major factor in not applying to the FP7 Energy programme is that no appropriate project has arisen. A less major factor is that there is no FP7 funding in their research area. A number of the issues were identified as minor factors in not applying to FP7 energy including a focus on national (public) sources of R&D funding, that insufficient funding is available from FP7 Energy, finding a transnational partner and the requirement for a consortium approach. Some items were specifically identified as not a factor in not applying for FP7, particularly the scale of FP7 being too small but also organisational focus moving away from R&D and focusing on national funding.


Interestingly, 60% of respondents said their experience of FP6 was not at all a disincentive to apply for FP7 funding, through around 20% said it was a minor factor and 15% a major factor in not applying. Overall the most influential reasons for not applying appear to be based around specific project details such as appropriateness to the organisation or projects research areas called. Some operational elements such as finding partners and needing a consortium approach were highlighted as minor factors. Previous experience of the Energy FP did not appear to act as a negative factor to the majority of respondents. 6. How likely do you think it is that your organisation will participate in a future FP7 Energy programme?

How likely do you think it is that your organistation will particpate in a future FP7 Energy programme?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Likelihood of future FP7participation

Don't Know Very Unlikely Unlikely Neither Likely or Unlikely Likely Very Likely

The results of this question show that over 55% believe they are likely or very likely to participate in a future FP7 Energy programme. While around 13% thought it unlikely they would participate in a future FP7 Energy programme no respondents felt it was very unlikely. Q7. Do you have any other comments?

Any other comments

I think that if my city could implement in FP7 Energy - investments project, not research like is now, it would be more interesting for my authorities



5 List of Consultees

The expert panel members are as follows: • Garry Staunton – head of low carbon innovation and the Carbon Trust. • Collette Lewiner – Cap Gemini Energy and Utility Director • Jesus Isoird – as a representative of Euroace (employee of Accione – Spanish

construction and energy developer) • Ronnie Belmans – University of Leuven • Emilia Panayotova-Björnbom, Royal Institute of Technology, Stockholm (retired)

The stakeholders consulted are as follows: Commission Stefan Tostmann DG Energy Head of unit Glyn Evans RTD Deputy head of unit K1 Ms Constantina Filiou, DG Energy Mr Roberto Gambi, DG Energy Mr Alexandros Kotronaros, DG Energy Mr Kyriakos Maniatis, DG Energy Mr Pietro Menna, DG Energy Mr Jose Riesgo Villanueva DG Energy

Technology Platforms Mr Fichaux, Ms Radvilaite EWEA Mr. Landolina Eurec Advisory Group on Energy Ms. Haug University Hohenheim Ms. Bach Arsenal research Programme committee Mr. Martin DECC – UK Government Mr. Vaucher Euresearch

The FP Projects chosen for in depth interviews are as follows:

Interviews Project Acronym Co-ordinator Partner

1 METAPV 3E. (Be) - 2 SOLASYS Fraunhoffer(De) BP (UK) 3 STANDPOINT Wavebob (UK) - 4 LED Abengoa, Spain (ES) TNO (Be) 5 OPTIFUEL Inensys (De) -


6 ALONE Florence University (It) DLR (DE), Eurac, Solitem (De) 7 MEDIRAS Franuhoffer (De) PSE (De) 8 DECPO ENEL (It) 9 ECCO Sintef (Norway) Statoil (Norway) 10 SEETSOC City University (UK) NTU (EL), EMS (Serbia)


6 Stakeholder Interviews

6.1 Evolution of calls in FP7

Our consultations have revealed a number of interesting points regarding the way in which the calls under FP7 have evolved. The first call was described as being drafted with an FP6 mentality which led to a relatively high number of projects with a high number of participants being supported. There is no formal, specific evaluation of the projects supported by DG TREN under FP6 but it is clear from the opinions gathered that there were a number of problems. These problems can be summarised as follows:

• The high number of projects and the high number of partners within these projects resulted in a high level of administrative requirements for TREN, for example more partners increased the chances of partner changes, which is an admin heavy task.

• The high number of partners implied a low budget per partner which can increase the likelihood of the project receiving little attention.

One external consultee raised the point that the decision (to reduce the number of areas open and go for fewer, larger projects) could be justified solely for administrative efficiency reasons. These related to the very poor perception among some beneficiaries and applicants that developed regarding FP6. These could be summarised as stemming from lots of small projects, with lots of partners, leading to diluted efforts which acted as a disincentive to many applicants. The creation of this disincentive needed to be addressed and the approach adopted has achieved this.

6.1.1 Fewer Projects with Less Partners

The switch to fewer projects, with fewer partners, which began in the second call and has been retained since, raised a number of points. A benefit which was mentioned by both Commission staff and external consultees was the reduction in the administrative workload of commission staff. This brings a number of positives, including:

• The project officers have a more reasonable workload, so can do a better job. • The officers are able to focus more closely on the project, so learn about it, and from

it, more, for example each project should now get at least one visit from the project officer.

• Learning from the projects is useful as the project officers are involved in policy development so this direct and practical involvement with research is of clear benefit.


The ideal situation, in terms of project workload on an officer, was described as a portfolio of some large and high profile projects plus some smaller ones. The generally increased scale of projects (average project value up from €6m to €15m) means that the projects should be of sufficient scale to achieve high impact and visibility on the research landscape. Larger projects are also more likely to attract attention within the Commission which helps give profile to the policy goals and activity of DG Energy and can even help in terms of speeding up Commission administration. Industrial participation in the DG ENER part of FP7 is high, currently reported as over 70% which is much higher than the rates achieved under previous FPs. The high level of industrial participation has led to large sums of private funds being attracted to match the commission funds. This is believed to build in a positive cycle in that a large cash contribution from a private firm will increase and improve the management and input from the company, which makes for better projects with less management input required from the Commission. The major reduction in the number of topics open per call was also described as having the following benefits. As the total budget remains the same, the amount per open topic has increased. As the number of potential, credible, participants in a particular field is limited the implication, and reality, of the situation has been larger budgets per project and larger budgets per consortium member. This increase in budget is perceived as being attractive to potential applicants. This issue is illustrated by the reduction in the number of bids and the subsequent increase in the chances of a successful application. One consultee, who is employed by a large company that has participated in FP7 projects, agreed that fewer and smaller projects resulting in a higher success rate (for applicants) does make the FP more attractive to large companies, as they see a better chance of winning and the effort they expend is more likely to result in funding. The improved chances of success when bidding for funds, narrowing from 1 in 10 previously, to 1 in 5, or even lower, is more in line with the chances of success companies typically work with when bidding for commercial contracts. This motivates better bids, particularly from commercial organisations. Improved motivation also relates in part to the significant time input required in preparing a bid of this nature, for example the need to identify demonstration sites usually requires the agreement and buy in of a third party such as a local authority.

6.1.2 Large vs. small projects

With regard to the question of a general approach of many small vs. fewer and larger projects there were a number of interesting points raised. While it was agreed that in general projects with lots of partners are more difficult to manage, there are some cases and target markets where this approach is beneficial. For example in some building related projects the ideal consortium would have a representative from each part of the long (in comparison to renewable energy) supply / value chain. Although this implies a high number of partners many will only have a small, but important, involvement


6.1.3 Mix of technologies in approved projects and calls

Those consulted raised the following points regarding the mix of technologies covered in the calls and approved projects. The alignment with the SET plan was described as a sensible arrangement and one that has made the programme management more straightforward as it provides a justified framework for structuring the calls. The fact that each call description is presented as a combined document between DG Energy and Research is intentional as it gives a picture of a coordinated and larger approach than separating the focus areas between the two DGs. Many of those consulted felt that the current approach of supporting large, demonstration focussed projects presents an easier opportunity for renewable energy focussed projects than for energy efficiency focussed projects. This was believed to partly relate to the nature of the projects, for example a large scale installation of a new wind turbine is a simpler prospect (one site and one technology) than installing multiple small scale efficient appliances. The Concerto approach (considering multiple low carbon solutions on a spatial basis) was mentioned as a way in which this issue had been addressed in the past. With regard to energy efficiency, another project type which has been stopped is ecobuilding, the main reason for this was understood to be that it is difficult to judge how one building project is better than another. However there has been a recent call for proposals called the E2B roadmap. This is a repackaging of FP7 money, branded as a response to the economic downturn with a total value of €500m including €125m from DG energy. The call asks for projects focussed on particular building sectors e.g. ‘a school of the future’ concept. With regard to funding projects focussed on industrial energy efficiency, another problem with these is that there is a perception that industry should be pursuing these opportunities anyway, particularly as energy costs are so high, leading to a significant risk of a lack of additionality. Despite this concern some projects are funded, about 20% of the project portfolio. It was suggested that the ideal solution would be a cluster of projects to help get scale and momentum.

6.1.4 Mix of Technologies and SET Plan Alignment

The question was raised of balance between the objectives, with the assumption that a portfolio approach, where projects covering all potential areas of research are requested, will allow comparison between the technologies and multiple approaches to the same technology. This is a first step to 'picking winners' both within a technology area and between technology areas. Such a portfolio approach also allows cross sectoral issues to be picked up. This leads on to questions around the fundamental targeting of the programme, which revealed a number of alternative approaches and opinions, some of which are energy specific and some of which are more generic, though still relevant. The consultee from the UK’s Carbon Trust (CT) provided an interesting comparison between their approach and their perception of how the FP works. He described a high level split in the approach, such that a programme could either focus on known areas of strength or improving areas where the EU performance is currently weak, but improvements are


necessary / thought to be beneficial. The CT approach was described as being to offer a range of schemes to fit the variety of R+D needs. This approach was felt to differ from that of the FP which is perceived as starting from a final objective and working in a top down manner to specify the research it wishes to support. This consultee also raised the issue of the risks involved in funding a project and on commercialising low carbon technology. He raised the point that the perception of risk is industry is often not what public funders expect and that industry put a large weight on the perceived risk at certain stages on the route to market, such as the classic ‘valley of death’ risk profile as innovations come close to market – at which point the public sector sees the risk as diminishing but the private sector see it as very high. It is important that public supporters of R+D are aware of this difference in perception and are willing to accept how industry feels, and do what it can to help them. The CT aim to adopt a more private sector view on risks, and as such, the vast majority of their staff have a private sector background – and try to focus more on the issues of importance to the companies involved. The additionality that public R+D support should have is two fold – enabling projects which would not otherwise happen and speeding up others. For projects where the additionality is ‘speeding up’ the recipients are relatively passive in that they need help to do more of the same. Where the desire is to accelerate activity the assistance needs to look at the key barriers, often this shows areas where things should be happening but are not. The technology accelerator approach, that the CT use, is designed to help fill such gaps by speeding up the transition from university research to the market. One consultee described the current situation in terms of technology choice in low carbon research as still being at the ‘let all flowers bloom’ stage, i.e. the problem is so large and important, and the potential solutions so diverse that it is unwise to try and focus on particular technologies to the exclusion of others. Another reason not to remove any technologies and to continue to support all is that the policy environment in terms of member state and industry commitment to low carbon is still fragile. It is also the case that the technologies which eventually emerge as the most important may not be those which appear best placed at the moment, or those which are technically / theoretically the best, as this is how technology development works. With regard to the focus of the projects, one consultee stated that the focus on near to market research was understandable, as industry is motivated by the prospect of profits and there is a need to spur carbon savings as soon as possible. This needs to be balanced with a recognition that innovation does not finish when a product comes to market. Ideally there should be some funds available to support all stages of the research cycle, including product development. This point is of relevance to reducing energy use in the buildings sector, as here research needs to focus on the cost reduction of relatively mature technologies - i.e. more like product development but still research. Industry needs to have confidence that they can look for support to help them plan investments. With regard to the split between RES and EE one DG officer made the comment that if this split is done by project numbers it can be misleading as the large CONCERTO projects might have been 3 to 4 separate projects in the past but the focus on larger projects has led to them being combined. Analysing by funding can also be misleading as some of the efficiency projects will have an impact on a lot of buildings despite being relatively low budget in


comparison to RES projects. The building specific calls have helped to keep the balance – which is better than it appears. In general terms, heat related projects are less supported than electricity related projects, if the programme wishes to reflect the use of energy, this is an under representation. This also relates to the fact that heat is not a traded and transported commodity like electricity. Improving the efficiency of its use will require approaches that are attractive to consumers, which has been something of a market failure to date. The technology platform on heating / cooling should help develop such concepts. Energy efficiency is more cross sectoral than RES in terms of the technologies being developed. Much of the research / improvements available relate to controls and ICT and these projects would typically not come under DG ENER. Energy use in buildings is complex and many of the FP projects / calls are pursued in conjunction with other DGs, (2 calls of this nature were mentioned). One officer raised the logical point that any relative (to renewables) lack of energy efficiency demonstration projects, fundamentally relates to the lack of a winning commercial business model for energy efficiency. The technology is mostly well known and proven, but the economic models and incentives are still to be properly devised. Some of the DG officers felt that ideally the programme should look to retain some balance, within each project and across projects, between demonstration for commercial benefits and scientific benefits, with the projects selected for the scientific benefits typically being more cutting edge and riskier. This balance needs to start in the calls for proposals. On a similar note one officer felt that it would be good to retain some of the budget to support smaller projects. Another officer agreed that supporting some smaller projects reflects the industry approach as they will carry out smaller scale testing prior to large demonstration. With regard to supporting cross cutting technologies, a relevant technology area that was discussed was Carbon Capture and Storage (CCS). It was pointed out that CCS projects are also supported under the European Energy Programme for Recovery (EEPR) via grant aid of €1bn to be invested in six CO2 capture and storage projects. This was described as being in recognition of its major importance for the EU economy and also as an important bridging technology to a future renewable and hydrogen energy system. In order to help applicants clearly understand what ENER mean by ‘demonstration’ projects it was suggested that the definition used in the call for proposals needs to be very clear. This will avoid applicants submitting and presenting research projects as demonstration (not to say that research projects shouldn't be funded). The attempts in previous calls to define ‘demonstration’ could be improved. The EU has in the past lacked an overall strategy, in terms of which energy technologies to support and how, the SET plan has been the first attempt to address this. Some alignment of FP Energy funding with the SET roadmaps has already taken place and is likely this will increase over time. With regard to the balance between renewable energy and energy efficiency projects, the point was raised that many technologies that can improve energy efficiency are already developed and available, and that take up is lacking due to the market, rather than technology issues. Renewable energy is also generally perceived as more glamorous than energy efficiency.


With regard to the balance between demonstration and research, the view was given that it was appropriate to focus more on demonstration. Given the large investment such projects require it was to be expected that large companies should lead projects. SMEs can still take part but they should not lead projects. From the DG RTD perspective it was felt that DG ENER now managed the demonstration projects and DG RTD the research focused projects. It is important that research is continued to be supported as it drives innovation. With regard to putting both research and demonstration projects under DG Energy, this approach would be resisted by DG RTD as other DG's may also seek control of both aspects of the parts of the Framework Programme of relevance to them, e.g. DG Environment. One officer felt that the clear focus on demonstration had helped clarify the division between DG Research and DG Ener – this split (in terms of distance from market) had not always been obvious to external observers. The alignment of FP7 Energy with the emerging priorities of the SET plan was generally supported, with it being described as a way of adding credibility to both. The technology platforms continue to support the alignment with the SET priorities. However the change to fewer and larger projects is recognised as having a number of down sides. These include:

• A perception of a focus on large companies, though in terms of SME participation rates this does not appear to be true.

• Some past participants are unhappy that their opportunity to participate has reduced. This is most often the case for smaller research institutes, which are more concentrated in certain member states.

• There is a perception of decreased participation from new member states, partly due to potential applicants there lacking the administrative capacity and experience to submit bids.

• With regard to projects which are not focussed on demonstration, the overall approach of less and bigger projects is less obviously beneficial. In these more research based areas narrower calls will lose potential ideas and approaches. Bids of this nature are often easier to put together than large demonstration type projects, for example because there is no need to agree large sites, so applicants are more likely to be willing to accept a lower chance of winning.

• It is recognised that, although improved, the flexibility in the financial arrangements could still be better.

6.2 Quality of Supported Research

Views on the quality of the research supported were sought from Commission Officers and project participants. The global ranking of energy R+D in Europe was reported as an issue which is always considered when assessing applications. The point was also raised that as there are now fewer but larger) projects it becomes even more important, because the numbers of projects are lower, so more is more pressure on each individual project to deliver something of global significance.)


With regard to biofuel research and demonstration, the opinion was raised by a DG officer that FP support has helped put the EU at the front of global research. However, it is difficult to compare EU research with the position in the US, as their approach is different. The US approach involves more funding and more concentrated efforts on a technology once it has been selected, e.g. ten projects funded at one time in biofuels in order to increase the chances of success. This gives more intense coverage in some areas than the EU, but less in others.

6.3 Increased support for International (non EU) partners

The question of whether the quality of projects would be improved if it was made easier for non EU applicants to participate in projects drew a variety of comments. It was pointed out that there are already a number of other parts of the FP, and other EU funding sources specifically targeted on cooperation with partners from outside the EU. For example a joint RTD/ENER topic, under the Specific International Cooperation Action, focused on EU cooperation with India. There are also overseas aid programmes which can support energy infrastructure development and it should be remembered that the FP has a very different purpose to the aid budget. On international partners, the benefits of additional inclusion were felt to vary by sector. Where there are clearly global leaders who are not EU based, (e.g. biofuels in Brazil) there is a good case, but where the EU leads (e.g. wind), or where the project will result in developments outside of Europe the case is much less strong. For new sectors (e.g. wave) all developers have the same initial goal (i.e. a working large scale prototype) so it is possibly easier and more sensible to collaborate. Where non EU cooperation is supported it should be done on an equal basis (in terms of funding support) and care needs to be exercised with regard to leakage of technology and expertise, especially to countries who are actually at the cutting edge themselves, e.g. current coal generation plant installations in China are more efficient than in the USA and emit less CO2 per kWh produced. Contractual complexity also increases with the inclusion of non EU partners, which is another reason for limiting it. In conclusion the most commonly held opinion was that international partners can improve the quality of projects, but also bring risks and downsides so the current flexibility is a fair compromise.

6.4 Measuring progress

The DG officers that were consulted raised a number of interesting points concerning the measurement of project, and programme achievement. The following measures of success were suggested, with the proviso that they can only be measured some time after the project funding period has passed.

• The speed and scale of technology deployment. • The establishment of European companies as world leaders in their field (as has

happened with wind energy). • Achieving / contributing to the policy objectives, i.e. replication and the CO2 savings

each replication achieves.


• Improving the cost effectiveness of the technology, which requires clarity on the cost reductions achieved, even if only a forecast.

• Progress towards market for the technologies supported, this is more complex to measure and requires an in depth analysis by field.

Fundamentally each project should be evaluated on whether it achieved its goals. For example, for a project demonstrating a new plant – did it get the plant up and running? Did it achieve its targets (e.g. % efficiency)? Has it reduced the production cost? Has it led to replicate plants in the EU? However even if some (or all) of these goals have not been achieved the project can still produce useful results, although this would require an in-depth and project specific review. It was generally recognised that these issues are not very well picked up in the project monitoring process and that the applicants can be somewhat weak on specifying these measures of success in their applications. There was some concern that the follow up of projects was lacking and that the focus of attention is much more on launching and running projects and programmes rather than looking for their longer term / ultimate impact. It was suggested that metrics such as an index on related growth or 'build and run' related indexes, would help address this and illustrate programme sustainability. It was suggested that ex-post monitoring of impacts, could be a contractual obligation on the projects but this would imply long contractual periods which would be difficult to enforce once payment (for the bulk of the project) has been given. On a general level it was also pointed out that attempting to predict project success in advance is not really possible, partly because many of the factors which determine success or failure are external to the project. The use of patents and citations (resulting from FP funded projects) although interesting and positive, was not felt to be a very strong or suitable indicator of success for the projects funded. There were a number of logical reasons given for this opinion, as follows:

• Spurious innovations can be patented if this is seen as an end in itself. • Citations can also be manipulated. • For the demonstration projects which are now the focus of FP ENER, it would be

expected that much of the innovation being demonstrated is already patented. • Sometimes the innovation in a project is the scale, which is not really a patenting

issue. • Some innovations achieved by programmes cannot be effectively or efficiently

patented. • Increasingly ideas are not patent protected anyway, due to the difficulty of

international patent protection. The guidance documents on completing the application form ask about CO2 savings (although there is no explicit question in the form) and this is an important factor in assessing bids. Replication potential is also raised as relevant but it is recognised that this a more complex issue than CO2 as it requires a commercial understanding of EU market. Some officers also thought that estimating CO2 savings was also too difficult for applicants to estimate. These markets vary a lot by technology, e.g. retrofit vs. new and by MS and even region. For example with biomass the replication potential depends on resource availability – this varies widely and at a very local level – and is therefore very hard to know for the whole


of the EU. Estimation of replication also requires some prediction of the future, which is complex, though this could be improved, for example a view on future fossil fuel prices is key to estimating replication potential for the next 10-15 years. The majority of those questioned thought that the CO2 savings question is important as it quantifies the impacts and clearly links to the priority target for ENER – 20/20/20. Even though it is not necessarily an easy question to answer, applicants should be obliged to attempt to do so, as long as the method to do this is clearly explained, a rational view taken on any claims which appear extravagant, and it was not used as the only criteria for judging applications. It was also recognised that the ease and speed with which a low carbon technology becomes commercial is strongly influenced by factors other than the technology, i.e. financial incentives. This reflects the fact that at the present time most technologies targeted by the programme are still not competitive with conventional (fossil) solutions, though some are getting close (e.g. wind). The length of time that such financial incentives will be required is very closely linked to the fossil fuel price. However the support provided by the FP will improve the technologies and make them more competitive with fossil fuels. A number of the officers described it as a balance to strike between supporting the ideas which are very close to commercial and those which are higher risk. If projects are too close to market the support becomes subsidising what the companies should do themselves, if too risky, the implementation may be difficult to achieve and too few of the projects succeed. One officer felt that there are good examples of the balance being achieved in the past, the example the support given to the Concentrated Solar Power sector by the FP helped the sector get started by funding three large demonstration projects, two of which were successful and a now commercially successful company (Abengoa Solar) has been created. The reality of market and technology interaction driving large scale replication and market uptake of low carbon technologies is intended to be reflected in the SET implementation plans. The key performance indicators (KPIs) being developed in these plans should offer a clear future mechanism to help monitor this and as such it was suggested by a number of consultees that they should be clearly referenced in future FP calls. Using the SET roadmaps is the assumed intention for the future of FP Energy funding. This could also help avoid MS duplication, but is hindered by the fact that not all MSs are actively involved with SET. DG RTD also recognised a need to improve the measurement of project success, and hoped that this would improve under FP8, for example with the utilisation of the KPIs developed by the SET plan.


7 Project Participant Interviews

7.1 Relevance / Utility

7.1.1 Global Competitiveness

Most of the coordinators interviewed felt that FP7 put Europe at or near the top of the world rankings in terms of energy research, and that it was important, indeed critical, for EU competitiveness that that was the case, though one project co-ordinator felt Europe lagged behind global competitors somewhat. He did comment, though, that FP7 was helpful in somewhat redressing the imbalance. That said, few felt they had a sufficient grasp of the overall balance of energy R&D funding globally on which to make comparisons, though in their particular topics and fields of expertise, most felt the FP7 projects were cutting edge. A number commented that on renewables development and deployment, Europe was world leading, and particular applications such as laser treatment of photovoltaics was global state of the art. Another consultee also felt that the EU was currently leading in CCS but the US were now dedicating large resources to this area and could overtake the EU. The comparison to the US was made by another consultee who felt that the FP7 has smaller budgets than the US Department of Energy research programmes, both for research and demonstration focussed projects. One interviewee pointed out that there was a global increase in the level of interest in, and resources directed towards renewable energy sources, which he characterised as a global shift away from IT R&D towards renewable energy. This opinion was endorsed by another consultee, a large commercial partner, who commented that the absolute importance of EU or Member State funding of research had reduced somewhat in recent years, as companies have a sufficiently strong commercial driver to undertake research in house or with partners but without public funding, the benefits to firms being greater control over the research, the ability to progress more swiftly (not having to wait for a decision on funding award and subsequent contract negotiation), less bureaucracy and, perhaps most importantly, the ability to keep highly commercially sensitive work secret. The downside of purely commercial work is that results don't get shared. The implication is that the terms for commercial partners to participate need to be made sufficiently attractive such that strategically important work could be done within the context of an EU public/private project. One co-ordinator from an academic institute with extensive experience of FP projects considers that the FP7 programme has been very effective at meeting the goals of energy and R&D policy. It has been able to develop projects which are assisting in taking forward big step changes in technological developments. The finance provided by the Commission enables good sized projects to be devised but they feel the big ambitious goals are inhibited by too much emphasis on demonstration and not enough on the R&D element. In addition,


because FP7 only provides 50% of the funds, trying to secure the remaining leverage from industry or the private sector is viewed as very difficult. The particular technology in the project for this co-ordinator is a long way from market entry and the project is relatively high risk.

7.1.2 Measuring the Impact

There were very mixed views among the coordinators on the best metrics to use to measure the impact of the programme. For every participant who suggested that one or other of the proposed metrics (citation index, patents, overall R&D spend, and R&D spend per capita GHG emissions) is a good indicator, three or four other participants cited drawbacks with each one: Ø Citation index might be fine for an academic partner, but not for a commercial

partner; Ø Not all areas of research can generate the same level of patents – particularly those

projects that do not lead to a commercial product. Also, given the programme is focused nearer to demonstration, there may be fewer patents emerging (particularly where a project is focused on process improvements);

Ø Overall R&D spend was considered crude – it's not about the money, it's about the results;

Ø As far as spend per capita GHG emissions, there are many factors affecting emissions, and not all the change is driven by R&D. Also, this would favour large scale, near to market technologies such as CCS / Nuclear but will work against those such as ocean energy which are at an earlier point of scale and technological development.

Ø Cost is not the only barrier to technology deployment, for some technologies public acceptance is also key.

Picking up the theme of the importance of results, 6 interviewees said that the true impact of the programme can only be measured by the downstream impacts, be that sales volumes, number of production plants constructed, increased turnover of participating companies, number of new start up/spin off companies created, or GWh of energy saved or GHG emission reduction.

7.1.3 International Co-Operation

When questioned on international co-operation with organisations outside the EU, coordinator interviewees were split into two camps – those that felt in would be a positive development, and those who were against it. The main concerns of those opposed to international co-operation centred on the leakage of intellectual property and manufacturing capacity out of the EU. One of the commentators against co-operation argued there would be more value from encouraging more cross-country co-operation spanning different areas of Europe e.g. Scandinavia and the Balkans. He felt that Europe is very diverse and it would be beneficial for different areas to work together and learn from each other. Two interviewees in favour of international co-operation argued that "artificial geographic limits" do not reflect the reality of teams and production capacity being placed anywhere globally, even if the parent company is European. One argued that, given the global nature of the GHG problem, it was important to have global co-operation on solutions.


One partner in a project focused on increasing deployment of renewable energy in South East Europe felt that it was important politically as well as environmentally (given the large renewable potential) to involve neighbouring and accession countries such as Turkey. However, this partner felt that nothing would be gained by including non-neighbouring countries such as India and China, a view echoed by the co-ordinator of another project, who felt that opening up the programme to partners from the Far East would risk EU companies NOT participating, as they would be wary of the risk of knowledge transfer outside the EU. One partner felt there was a real benefit in keeping the programme within the EU as the distance and time to deployment of successful projects would be minimised. Conversely, one interviewee felt that collaboration with China would be very useful, given they have expertise in implementation at large scale. Of those in favour of international co-operation, a number commented in detail: Ø "Co-operation with non-EU partners is very important. We are starting to have many

contacts with India, Russia, South America, Libya, Tunisia Morocco etc. The Russians, for example, pay all the contribution costs while for example with India it is a real co-operation project. The EU brings the intelligent design dimension while the Indians can provide the workforce for developing and testing a component that in EU would cost too much. Such examples would double the funding and at the same time oblige the partner country to bring its added value to the project. But you need to ensure it is a proper partnership – my experience in partnering with [a North African country] was not a positive one."

Ø "The energy problems are global, so worldwide research should be complementary.

As suggested, foreign research (i.e. outside of the EU) is to have two governments making agreements on the ways of funding. In the present FP7 programme, it is possible to include a foreign institute, but only as a subcontractor. This means in practice EU money is flowing externally. An example of such an alternative structure is the IBEROEKA program, where the Latin-American authorities fund research in Latin America while the European authorities fund European research on their part accordingly (some specific calls include Japan).

Ø "The presence of international co-operation funds could improve the quality of future

projects. For instance, CCS (Carbon Capture and Storage) projects can address problems which go beyond European borders. In this case a mixture of funds could help the birth of cooperation projects which can put together international know-how. For example, in the specific field of CCS an important contribution could come from MIT (Massachusetts Institute of Technology), who are at this moment a very important partner. But other important international institutions can make a valuable contribution as well."

Ø A wider breadth of participating countries would help spread the costs, particularly

for very high cost technologies, such as CCS. Ø Involving partner s from ‘hot’ countries (e.g. Egypt, Algeria, Morocco) is appropriate

for solar technologies which would have a large potential market in these countries. Siting a demonstration project in these countries would be ideal when the potential customers are there.


Ø There are MS level programmes which are designed to enable direct cooperation

with non EU partners. E.g. the Dutch BE-Basic programme regarding bio-refineries which supports cooperation with Brazilian firms.

Ø One project reported difficulties with Moroccan and Jordanian partners.

7.2 Efficiency

7.2.1 The Application Process

In general, participants felt the application process was reasonable, though most had comments on specific areas where the process, or their experience, could have been better. Two co-ordinators admitted to using external experts to manage the application process. The comments below illustrate the wide ranging nature of comments on the application experience – with different applicants sometimes presenting opposite views on certain aspects: Ø "I found the application process straightforward with clear instructions. However,

once the project had been recommended for funding, the process became more complex as it was necessary to make some adjustments to the original proposal (in order to rectify errors) and a detailed justification was required. The support received from the EC officer was outstanding, as this person helped to identify issues and also resolve them."

Ø "I felt that the structure of FP7 projects is still very much focused on companies with national organisations and does not reflect the business processes of companies that are already organised on a European level. This results in an additional administrative burden during the proposal phase as well as during the execution phase, because the different R&D locations in other countries in Europe have to be included as third parties. This is not a substantial obstacle but it does not reflect the European concept and adds additional, unnecessary administrative burden without yielding any benefits in return."

Ø "There were some problems with the participant identification code – mistakes in the coding process had led to some of the participant numbers being linked to the records of different companies. This was frustrating and it took some time to get a response from the IT helpdesk and resolve the issue."

Ø "This was my first FP project (although the university has previous experience). My initial task was to coordinate the proposal inputs and build the consortium. The application process was not that bad. I had previously applied for US grants and found the process for these was more difficult."

Ø "Meeting the partnership conditions was challenging – assembling partners in the Balkan region can be difficult as there is interaction between business and politics. Their approach was to make contact with the industrial partners via the academic partner for that country. This worked well."

Ø "The new application process for FP7 is better than for FP5 & 6. It is much easier doing it electronically."

Ø "The two-step procedure is helpful as it reduces effort for 1st stage. However, the evaluation procedure for the 1st step is not clear as it doesn't give a score to show ranking position (it's just a pass/fail)."


Ø "As a partner, I wasn't involved in application process itself, so can't comment. I understand the Commission might want a mix of countries and indeed genders, but it could be too rigid/limiting. I could envisage a project where, all the best partners were from one country, but that wouldn't be allowed, so putting any form of artificial constraints do limit the absolute quality of the project in terms of its net impact."

Ø "From my perspective, the process and structures related to FP7 are generally fine and appropriate. It's important for FP to have some relatively strict, rigid conditions which all projects need to meet, otherwise how else do you select projects to either approve or reject?"

Ø "The application process was a severe burden. Not so much the actual application, but collecting all the forms and documents is: it is considered to be a full-time job. The consortium decided to subcontract an external agency to do this (hence, it is also difficult to evaluate the application process from the view of the partners). It is stressed that the negotiation part (after submission) is unsatisfactory as well. Because of this, it is not known in advance what exactly will be funded at which conditions, but the consortium has to sign a priori."

Ø "I felt the application process for FP7 was not too complex but we hired in somebody who had the experience with these procedures. It is important to note that the administrative process, especially the contract negotiations, takes a considerable length of time – which seems incomprehensible for an SME like us that is applying for the first time."

Ø "We found the application process fairly consistent between FPs but negotiation and management of projects is very time consuming and demanding. The negotiation process has very specific needs and this can get very difficult for SMEs. The move from FP6 to FP7 with a unified database is an improvement."

Ø "The procedures are too long, complex and risky: it is not easy for a small firm to work with FP7: a university has the capacity to put people to work for 2-3 months to prepare a proposal while for small firms this is impossible. This kind of procedure can last a year with the risk of not being awarded the project. There should be a pre-evaluation proposal of no more than 3 pages. The EC could already select the ideas which are in line with the project. In this way if an idea is not likely to be selected time and money will be saved. The pre qualification paper would help to select the good ideas that could come from smaller firms."

Ø "Another problem is that we never know by whom we are evaluated and some projects are not even taken into consideration while the same project presented for similar calls receive very good ranking."

Ø "I did not feel the application process was too complex or rigid. In fact, stricter requirements, such as having a signed consortium agreement before contracting would have helped, as in our case, one partner withdrew after the project was contracted with the Commission."

Ø "I see the application process as not too complex but of course there are too rigid conditions. One of them is for example the number of partners required. My opinion is that a proposal if well prepared and focused on technological improvement could be presented even to only one company instead of a large consortium. Provided that a strong commitment to dissemination and to publish results to the European Commission is assured (of course this must be secured by the European rules of the programme)."

Ø "We did not find the time to contract a problem. Rather, we felt that this was more a pressure from the Commission side with achieving 2020 goals in mind. In certain


rapidly changing sectors e.g. Solar PV, a delay in contracting could be problematic in terms of being left behind and that there was a need to act fast in these cases."

Ø A two stage application process would reduce the risk of preparing a large and detailed full application based on the relatively brief information contained within the call for proposals, only to find that the application was not in line with Commission expectations / requirements.

Ø It is become harder to gain access to Commission officers prior to submission. Ø The application process is not too complex and the inputs required are reasonable.

One participant felt the financing regulations were too strict. Shared activities in various Member States lead to an administrative burden; for instance (external) auditing of salaries is required for each country. It was suggested that more flexibility in resource allocation (leave it to the partners) should be pursued. Another suggestion was to have a dedicated project manager at the side of the Commission (being present at meetings, knowing the content, etc.) rather than an administrative officer."

7.2.2 Project Size

There were mixed views among coordinators about the move to larger projects, with the balance slightly in favour of large projects. Among those in favour of larger projects, the following comments were made: Ø "There are no real downsides to larger projects. However, if the trend is more to

demonstration projects, then R&D centres will find it more difficult to participate as can't find the match funding from own resources."

Ø "I prefer the funding of fewer but larger projects. It is possible to do more with a larger project and get better, more meaningful results. However, a downside is that there is now more competition – the larger grants, along with the current economic crisis, make the programme more attractive to industry."

Ø "Larger projects are definitely more attractive, given the nature of the project e.g. many technological firsts and physical scale of demonstrator – project has to be big. I feel the FP7 budget overall is large enough, but were some technologies e.g. CCS, Nuclear receiving more than their fair share? Wave energy should get more."

Ø "The reshaping towards larger projects is positive. The proposal quality is going to be improved and consortia will be stronger. A consortium including 9 to 10 partners is optimal (in the past, some consortia included even up to twenty partners, which is considered too much)."

Ø "The reshaping of FP7 energy towards larger projects makes it more attractive to industry and I think it generates higher profile projects. I do not see downsides, though I cannot judge the influence on SMEs or small R&D centres."

Ø "I'm positive about the evolution into the direction of fewer, but bigger projects and also the fact that you need fewer partners."

Ø "The budget is appropriate. A larger budget can produce a larger number of projects and more new and technological target solutions. The European Union must


encourage MS to reinforce National R&D programmes and budgets in order to offer further financial resources to the energy market (in particular the public funding combined with the development and sharing of common strategic research target will stimulate the mobilisation of industrial actors on larger projects)."

Ø Larger projects are preferred as the administrative burden (for applicants) stays the

same what ever the project size. Ø Larger projects are more attractive to industry as the scale involved is more realistic

with regard to actual deployment. Ø The project might well have been 5 separate, small projects under the old approach,

so one large project is more efficient and increases the chance and scale of learning between the partners.

Those less keen on the shift to larger projects commented: Ø "In terms of reshaping towards larger projects, one downside is the number of project

partners required to form an FP7 programme. The only impact this creates is an extra administrative burden, which moves from the Commission and onto each project. It is almost impossible for an FP7 project not to designate / appoint a professional coordinator. The other key impact it creates (due to the number of project partners) is it only takes one partner to slow down or even stall the project, so more participants increases that risk."

Ø "Larger projects are not necessarily more attractive to industry. The important thing

is they must be relevant to their needs, and able to move quickly – size is not that important. Larger does give higher profile projects for EU to shout about, but they're not necessarily better. We were involved in a previous FP project – it was a huge project, involving everyone under the sun. The overheads involved in organisation were considerable. Likewise, communication within the project was a big challenge, as was keeping the focus on the project goals – too many people doing their own thing and going down their own little paths."

Ø "Small projects can be more effective and more focused – they generally work more

efficiently. You get much better interaction between participants. Also it's better for the R&D centres – they actually do the research, rather than focus a lot of effort on project management. As far as risk is concerned, it's not immediately obvious it's more or less risky depending on size."

Ø "In future, the EC should refocus the FP7 programme towards smaller projects (with

a maximum of 5-7 project partners), with streamlined / standardised conditions or terms of reference for approval. This would improve the efficiency of FP programmes and enable a more effective and speedy approval and operation process."

Ø The larger scale can discriminate against newer ideas as it is not feasible to go

straight to large scale demonstration with untried ideas. And some saw both sides of the argument:


Ø "We need 2 streams of projects: (a) long term, very large projects (say €100M) on

visionary future goals, which need 5-10 year horizons. An example might be a totally new battery technology needs longer. Within these large "framework" projects, there can be smaller projects, as you can't map out a 10-year programme in advance in detail, so it's got to operate more like a platform, within which there would be specific projects and calls for proposals. (b) The second stream should be focused of smaller targeted projects, say €1-10 M. The two stream approach means you don't put all eggs in one basket (technology) but you enable step change progress for longer term potential. Larger industry players will have the longer horizons. SMEs of course won't – can't commit to 5+ years, but they can join individual projects. Risk is inherent in both types, but these vary by the two streams."

Ø "As regards higher profile projects, I agree they should be part of the mix, but don’t

do it to the exclusion of smaller R&D."

Ø "I am unsure about the impact of the move towards funding fewer but larger projects. Larger projects are more attractive to industrial partners. However, they can be more difficult to review, quantify impacts and achieve cohesion/integration. Large projects are still attractive to SMEs and they can bring unique skills to partnership. SMEs are included in partnerships but it would be interesting to explore what proportion of the funding they receive. On balance, I think it is necessary to find a balance between large projects and small high-tech projects.

Ø "I believe the reshaping of FP7 energy into larger fewer projects has made it more attractive for those planning demonstration projects but less attractive to more R&D focused type projects. It hasn't put us off applying, though." (from an R&D partner)

7.3 Effectiveness

7.3.1 FP7 Objectives

Not all coordinator interviewees agreed that the most important objective of the FP7 energy programme is the achievement of GHG emission reductions. Most recognised it as an overarching ambition, but one which was a by-product of the R&D activity. A significant minority thought the number one objective to be industry-focused – either strengthening manufacturing capacity within Europe, improving competitiveness, or addressing security of supply. Whatever their views on the priority objectives, many all interviewees were agreed that quantification of CO2 or GHG savings was either extremely difficult, or impossible, at the application stage. Comments included: Ø "It is not straightforward to produce an estimate of the emissions savings that a

project would generate. In general, this would require two simulations to be run to predict the emissions with the project and also without, but this is not easy.


Ø "It's very difficult to quantify the CO2 savings as they arise downstream. Of course, you can make an estimate of the forecast volume of sales as a proxy, but this is prone to massive error."

Ø CO2 savings could be easily estimated for our project, requiring a combination of

technical and market analysis. Ø "CO2 wasn't a requirement in application process. Still now, half way through

project, we can't put a figure on CO2, but can describe qualitatively (in the project impact section). We don't think that CO2 should be part of the ranking process."

Ø "We don't know what CO2 savings are, and don't think it's that important to know

precisely. If using it as a proposal selection criterion, the problem is that without a strictly controlled process for calculating the savings, you could create perverse incentives as to how optimistic the outcome is to increase chance of success. "

Ø "Yes, CO2 savings should be included in ranking but would need clear procedure on

how to do calculations for CO2 – maybe independent of the proposers." Ø CO2 savings are the most important objective, though there is clearly a debate on

how to best achieve these. Ø The current level of focus in the application forms on CO2 is appropriate as it is too

difficult, due to the market variations and uncertainties, to predict the potential savings in detail.

Ø "We consider it's almost impossible to record or estimate CO2 reductions – there are

too many uncertainties which could influence this result and distort the findings. The institute have tried to meet internally to give consideration as to how to do it but they can't agree on the best method to achieve this. It is also considered that there are more appropriate measures such as (kWh)."

Ø "Each applicant needs to demonstrate a strong, clear argument of how each project

application could achieve GHG reductions. The problem for FP7 is that if nuclear projects were applying for assistance, whilst they can show huge GHG reductions, there are other risks (in the case of nuclear) which need to offset this connection. I'm afraid I can't offer any suggestions as to how this could be alleviated but I'm very aware of the problems for EC."

Ø "We tend to concentrate on the production rate outputs of technology systems e.g.

kWh – this is our primary objective. From a commercial perspective, the improved efficiency, flow and operation of production systems from new energy efficiency / sustainable energy sources is a much more persuasive argument. By concentrating on kWh rather than CO2 savings the benefits are clearly visible, the market / product potential looks more viable. We consider there is a very fine line between research and supporting industry to develop new markets and products. It isn't so easy to gauge future market success / market growth on CO2 savings made by project alone."


Ø "Personally, I don't think that being able to quantify CO2 savings was that important, as these benefits were likely to flow anyway. What's far more important is the potential for industrial success and to contribute to energy security."

Ø "I've noticed that CO2 savings is a focus of applications for national level funding,

but don't feel that GHG reductions provides a level playing field for technologies as some can much more easily describe scaled savings e.g. nuclear, CCS, than a technology such as wave that is still at early development stages."

Ø "The impossibility to estimate CO2 saving in many projects must be solved because

this could limit the possibility to monitor the possibility to reach the first objective (GHG reduction). Of course the participants must be put into the condition to give the correct answers to precise and concise questions. Putting more emphasis on this issue is very important but I am not sure on what kind of indicator must be used to do so."

7.3.2 R&D benefits Vs. Commercial benefits

Most coordinator respondents agreed the programme has a mixture of R&D and commercial benefits, with the balance weighted towards commercial benefits. Predictably, academic or research institutes put more emphasis on the R&D aspects, while industrial partners were far more interested in the commercial benefits. Ø "The focus also depends on the nature of the consortium/project – e.g. if a project is

led by an industrial partner then it would be expected to have a more clearly defined commercial focus. Where the focus is on R&D, it will take more time to realise the commercial benefits. In general, it is not possible to separate R&D and commercial benefits as they are all part of the same process. However, commercial benefits are generally part of the impact generated by the process of high quality research activity."

Ø "Projects already place an emphasis on commercial benefits but that the programme

requires immediate rather than long-term effects. Generating important intellectual benefits requires a more long-term approach."

Ø "It's a combination between R&D partner and commercial players. Main benefit for

R&D partner would be another follow-on R&D project! Of course there has to be a commercial benefit to get partners on board. So need to get the balance right. Both are needed."

Ø Given that many of the recipients of funding are researchers they will be more

confident of their projects achieving research, as opposed to commercial, benefits as they understand research benefits more clearly.

Ø "Only way to reduce GHG emissions is if R&D is commercialised, so these are the

most important aspects of the programme. Purely scientific interest isn't a good enough justification. It should be about commercialisation through the R&D process."


Ø "In terms of commercial versus R&D benefits – it is a little too early to say for our project but our institute's previous experience has shown that the commercial benefits are much harder to achieve."

Ø "As regards the balance between R&D and commercialisation, we're very much

focused on the latter, as we need to attract commercial partners to the project." Ø "R&D is really important. If we don’t focus enough on R&D when the market is

stagnant we won’t bring any money back. We should be careful not to mix the R&D and the commercial part. The schema should be:

R&D => Demo-project => Review of product’s design => Commercialisation of the product.

Nowadays we tend to pass from R&D directly to the commercialisation and even taking money from R&D for the commercialisation. Japanese have developed the R&D in the 1990s and only today they are commercialising the products. We should bring back the manufacturing otherwise we will remain forever China-dependent."

Ø "I'm strongly convinced that commercial benefits should be at the heart of FP7

projects. Projects focusing on improving R&D capacity rather than commercial benefits were getting things the wrong way round. Projects receiving FP7 funding should be beyond the R&D stage and be demonstrating something that will create a platform to market."

Ø "The commercial benefits must be always taken into consideration. If an R&D

project works (I mean it produces results), its results must take in commercial benefits (or future commercial benefits). To solve this matter it is not necessary to measure patents but it is sufficient to have complete evidence of the results of the projects and of their applications from a commercial point of view. An example of a question that an applicant could be asked to get this commercial view could be to demonstrate in the next two years after the project – in a commercial phase – to give evidence of the annual emissions reduction, annual energy saving, CO2 saving, etc."

Ø One new MS participant highlighted, and appreciated, the networking opportunities

that project participation had given him. Ø One project participant suggested that willingness to pay for the results of a project

was a strong indicator of success.

7.4 Sustainability

7.4.1 Future focus for FP7

Suggestions for future areas of activity typically reflected the particular field of endeavour of the coordinators interviewed. Areas put forward included: Ø Smart networks or ICT-related developments;


Ø All aspects of PV development. In particular, to strengthen the EU PV industry, it needs to be more competitive, so future FP calls should be more about manufacturing efficiency, excellence, productivity, low cost, competitiveness etc;

Ø Batteries; Ø Fuel cells, in particular advanced industrial manufacturing capacity - current

manufacturing is largely manual or only partially automated. EU has to compete with manual Chinese manufacturing. We need EU knowledge on highly industrialised production in new fields such as PV, fuel cells and batteries;

Ø In the future, synergy could be sought with the hydrogen technology. Fuel cell cars require hydrogen that could stem from the gasification and shift conversion processes as well. The development of electric cars may go in parallel;

Ø Solar generation from industrial processes; Ø Explore the use of naphtha that is, alongside biodiesel, another product resulting

from the distillation process. Bio-naphtha requires upgrading to regular gasoline standards and the upgrading process may be different compared to fossil naphtha. This is already an R&D activity within this project but this might require further R&D beyond the schedule of this project;

Ø Grid connection as a priority area; Ø CCS; Ø New production technologies and control systems - these could help to reduce costs

and improve efficiency. It is very important to continue to finance demonstration projects;

Ø For future FP calls, the key issue was to move towards some kind of standard approach to wave / ocean energy. Comparison was made with wind energy in the early 1990s, where multiple turbine options were around and that eventually the shakeout led towards a fairly standard design. A similar process is needed in wave energy;

Ø A new area for subsequent funding may be on the raw materials side of Lignocellulosic Ethanol development - collecting, handling and pre-treatment of residues, CO2 capture using algae, etc demonstrations and first of a kind steps to commercialise the technology may need different support. Another topic may be the optimal use of by-products and finally making inquiries on the conversion of materials containing higher amounts of cellulose;

Ø New solutions for off-shore wind; Ø New applications of geothermal energy to low enthalpy (binary cycle); Ø New solutions for hydrogen application; Ø Marine renewable energy; Ø Nuclear - new and more secure nuclear technologies, solve the problems of

radioactive waste, etc; Ø In a world where the utilisation of fossil fuel is predominant, we need new and

definitive solutions to reduce emissions from thermoelectric power plants. Ø A solar energy focussed call combining heating, chilling and power to reflect the

scale of potential demand for this technology in hot countries. Ø Grid asset management, particularly in eastern Europe.

Other coordinator comments concerned the nature of the programme, rather than specific technologies: Ø In terms of future areas / technologies, the Commission should give further

consideration for potential FP calls to mix member states with developing


countries (outside EU). Given the focus of development in these countries and their need for more sustainable technologies, there is an opportunity to build markets / products with these countries. It is likely that this is where the greatest demand for energy technologies will be but much of the production / innovation experience is found in the EU. Currently, the FP7 programme doesn't enable this to happen as extensively as it could.

Ø The new calls need to be less 'narrow / constrained' in the view of eligible projects. Currently it is a little limiting and EC can be slightly 'blinkered' or risk adverse.

Ø Need to also focus on longer term technologies and fundamental research. One partner in the LED LED (Lignocellulosic Ethanol Demonstration) project made two important observations that R&D funding on its own was not the solution: Ø Critical for the development of the second generation biofuels (and first generation as

well) is the stability of the legislative framework including long-term targets – this is more important than FP7 funding. The 10% biofuel requirement for 2020 in the Renewable Energy Directive is being loosened by some Member States. As a result, various biodiesel companies went bankrupt in the recent past.

Ø Rather than fixed processes and complex administrative regulations, it is suggested the Commission has open discussions with applicants on ideas and to establish reasonable budgets and prices (i.e. what is a reasonable amount of money to achieve a desired outcome).

7.4.2 Strategic Future of FP

The project coordinators were asked their opinions on the future focus and nature of the DG Ener supported FP7 programme. One coordinator reported that he felt “the Framework Programme should aim for some continuity in its work programme – building on the previous call but retaining flexibility to develop areas of weakness in order to maximize the benefits. FP6 appeared less focused and incremental and this may limit its impact. FP7 has been more successful at getting stakeholders involved and identifying their needs. There were a number of specific technologies and areas suggested for future support, comments of this nature included:

• There is potential for major innovation in energy markets, e.g. materials and ICT, which have not yet been explored.

• The Future and Emerging Technologies (FET) programme is a good idea and there would be scope for a similar programme focusing on the energy market.

• RES grid connection and the smart grid platform • Local energy generation and distributive energy sources. • Needs to be focused on encouraging EU manufacturing competence &

competitiveness. Most production lines are turnkey, so can be sold anywhere in the world wherever the skills lie and wherever it's going to be most profitable.

• Felt that FP funds should be directed towards projects that can deliver wealth creation – in respect of this RES should be the main focus, as Europe has an IP lead in many technologies and there is potential to expand the economic benefit of this.


There was some direct contradiction between project coordinators in terms of whether the focus should be close to market or more fundamental research. The diversity of opinion is shown by the following:

• There should be more focus on research which is further from the market but this cannot happen given the current proposal evaluation process which penalizes this type of work for not producing short-term outcomes.

• Large 10-year projects should be part of the mix, to address the mega trends. • For EU competitiveness, need to focus on manufacturing capability – so preference

for near-market focus, rather than further from market. • FP programmes should be stepping beyond just 'demonstration' and onto supporting

market stimulation, assisting successful projects in the short-medium term after completion in order to avoid floundering and failure after FP funding ends.

• There is currently a gap between technology and distribution systems, so the programme should include applied technologies, i.e. more market oriented.

• An element of the programme budget for smaller projects, which would help SMEs and further from market technologies.

With regard to the ability of the FP to leverage in, and encourage more private sector R&D in this field, one coordinator stated that “it is not possible in a free market to force private energy companies to divert some profits into R&D”. He went on to state that “ The funding of R&D must be a prerogative of EU and MSs which are working together with private companies to find the best innovation for the future. In other words, the funds must come from the government to try to induce private companies to put their know-how to find these solutions.”

7.4.3 Longevity of Impacts

The project coordinators had the following comments on the probable persistence of the project effects. One coordinator felt that dissemination is generally very good but commercialisation can be more 'hit and miss,' depending on participation and how the outputs link to the business of the industrial partners. Another coordinator was more critical stating that he thought “ the programme will not really produce long-lasting effects for a number of reasons including the fact that SME participation is not as strong as it could be and also because there is a tendency to avoid high risk projects (which are likely to have a greater potential impact)". A third coordinator thought that dissemination was good and that it was easy to find information on projects if you look for it.

7.4.4 Improving Dissemination

The project coordinators were asked for their opinions and suggestions on how project dissemination and long term impacts could be improved. One suggestion that was tested out with a number of the coordinators was the idea of increased inter project networking. There were a variety of contrasting views on the benefits that such an approach could bring. One coordinator did not see any benefits from inter-project networking, citing his own experience of trying to network four laser projects which failed due to a combination of competition between the projects and a lack of synergy. He felt that existing networking via academic channels and conferences was sufficient which was an opinion echoed by other consultees.


In contrast another participant thought there should be more inter-project networking and had never been involved in any previous attempts. Another coordinator also thought that the idea of joint conferences was a good one and would offer the benefit of increased networking between projects. A third coordinator suggested a two-day workshop where all relevant FP7 energy projects were presented. A coordinator who had been involved in a joint conference involving CCS researchers felt that it had been useful and was a good idea for certain research areas. With regard to conferences another coordinator stated that discussion was limited to published results due to confidentiality and IPR protection, though this was the accepted norm and he thought that any requirement on FP projects to network with each other would be a somewhat artificial approach. The retention of some IPR by project partners was seen as a very valuable benefit from participation and one that should not be lost in efforts to more widely publicise results. However another project participant commented on a reluctance of industrial partners in his project to share knowledge and learning with other partners, noting that some experiments within the project had been done which support a particular company's agenda rather than the project as a whole and details of the particular application were not being shared. He felt that there was a need stronger guidelines at application stage. The issue of disclosure was described as complex, as if FP7 requirements are more towards full disclosure, it would put partners off participating from fear of losing their IPR. However, he thought that the balance doesn't seem to be quite right at present. In contrast a participant in another project reported that dissemination was proceeding well and IPR protection had not been an issue. One participant raised the idea of additional follow on funding to assist in dissemination and commercialisation. The amount he thought would be required need not be large (may be only a few thousand Euros) but it could make the difference between a project stopping or actually achieving market entry and as such it would be an investment worth making. Another coordinator was less sure on the idea of follow up funding , stating that: “There may be good reasons why a project shouldn't continue or why it should be considered as a 'test-bed' and should be taken forwards outside of EC FP activity. They also pointed out that there is already some flexibility to secure funding extensions, though they believed that this was not consistently applied to all projects. One current participant felt that a separate contract for post project activity would be a complex arrangement One coordinator suggested that the EU should develop a dissemination strategy when products are mature and use highly recognised international events like Inter Solar or Solar Energy to present their FP7 projects. He felt that this would improve the credibility of the projects and help commercialise them. Such a collective approach was thought to be more efficient, in terms of impact per € spent than lots of individual project actions. One coordinator stated that his project has agreed to keep the project website alive for 1-2 years following project end to assist dissemination, he felt that this was good and it may be a good idea to oblige other projects to do this. One project coordinator mentioned security concerns regarding publicising the results of his project as it concerned the electricity infrastructure which is seen as an area of concern.


7.4.5 SET Plan

The project coordinators interviewed were asked their opinions on the future links between the FP and the SET plan. This revealed some interesting and varied perceptions. A coordinator from a very large research institute raised concerns that the rumours were that the EC plans to focus their research support solely on large multinational companies and move away from supporting SMEs. This approach was not popular as it was felt that FP would lose the ability to support growing but resource constrained SMEs. They believed that continuing to work with SMEs should be central to the SET plan and the FP. Another coordinator was also under the impression that the SET plan objectives were very focussed on demonstrations and as such he felt that funding and focus should also remain for fundamental R&D. There was also concern that too high of a reliance on private funding would put too much emphasis on cost effectiveness which could be detrimental. One coordinator felt that the EC technology platforms were a good mechanism for influencing the future direction of FP and SET and that the current process offers plenty of chances to influence/contribute to the policy debate. One coordinator felt that the FP funds should be used as part of the SET plan and should focus support on the best technological solutions independently from their proposers (SMEs or Large Enterprise). The involvement of MS funding was considered necessary in order to create a “fund association” with the European Commission and others creating Public Private Partnerships in order to support large scale demonstration projects as well as smaller projects. They referenced the Marguerite21 programme as being a good example of such an approach.

21 The Marguerite fund is a pan-European infrastructure fund for long term institutional investors to finance the implementation of

strategic European policy objectives and projects in the Transport, Energy/Climate and Renewables sectors. The main objective of the fund is to contribute to the European Economic Recovery Plan by financing the implementation of strategically important European policy objectives in Energy/Climate, Renewable and Transport sector infrastructures.


8 Network Analysis of FP6 and FP7

• Separate networks are presented for FP6 and FP7 • Networks are the result of the analysis of collaboration intensity among countries and

participants • Data have been cleaned prior to the analysis to avoid doubles (as a result of homonyms and

synonyms) – however, this cleanup mainly on the level of organisations/participants cannot be considered to be exhaustive (due to the large number of observations)

• Methodological note: the counting scheme applies is based on so-called ‘normal’ counting scheme; this implies that in the cases one projects contain participants of NL, BE, DE, DE, this would result in collaboration between NL, BE and DE; the two occurrences of DE are not treated as ‘2’

• Countries are noted by 2 digit code level (as appearing in the source data) • Participants are noted by their short name (as appearing in the source data) in order to keep

the maps readable • The titles of the figures presented below contain information on the approach used; e.g.

several network diagrams are the result of limiting the network to collaboration above a certain threshold (this is needed in order to keep figures readable)

• The thickness of the lines reflect the intensity of the collaboration – the position of countries/organisations are the result of relative positioning and indicate the ‘role’; the more central the position is the more ‘involved’ an actor is

• On the specific figures below: o Fig 1 - contains the interactions among participating countries for FP6 (all countries) o Fig 2 - contains the interactions among participating countries for FP6 (all countries)

where there are more than 5 links between countries o Fig 3 - contains the interactions among participating countries for FP6 (all countries)

where there are more than 10 links between countries o Fig 4 - contains the interactions among individual participants for FP6 (all

participants) where there is more than 1 link o Fig 5 - contains the interactions among individual participants for FP6 (all

participants) clustered around ‘CRES’ which is the most active participant in FP6 energy.

o Fig 6 - contains the interactions among participating countries for FP7 (all countries) o Fig 7 - contains the interactions among participating countries for FP7 (all countries)

where there are more than 5 links between countries o Fig 8 - contains the interactions among participating countries for FP7 (all countries)

where there are more than 10 links between countries o Fig 9 - contains the interactions among individual participants for FP7 (all

participants) where there is more than 1 link o Fig 10 - contains the interactions among individual participants for FP7 (all

participants) clustered around ‘VTT’ (Finland) which is the most active participant in FP7 energy.

8.1 Energy FP6

Figuur 1: Collaboration visualization on country level (FP6, all links)

July 2010 152

Figuur 2: Collaboration visualization on country level (FP6, > 5 links)

July 2010 154

Figuur 4: Collaboration visualization on participant/organization level (FP6, > 1 link)

July 2010 156

8.2 Energy FP7

Figure 6: Collaboration visualization on country level (FP7, all links)

July 2010 158


Figuur 9: Collaboration visualization on participant level (FP7, > 1 link)

July 2010 160


FP7 Energy Mid-Term Evaluation report

Documents

Transcript of FP7 Energy Mid-Term Evaluation report