Horizon 2020

Future and Emerging Technologies FET-Open 2016-2017

Cut-off 17 January, 2017

Roumen Borissov, REA, Unit A5

3 March, 2017

Briefing for remote evaluators

Content

• FET in Horizon 2020

• Evaluation Process – your role

• How to write an Individual Evaluation Report

(IER)

• Additional issues to be checked

• Approval of the Evaluation Summary Report

(ESR)

• Additional Information

2

FET in Horizon 2020

3

FET in Horizon 2020

Excellent Science in Horizon2020

INDUSTRIAL

LEADERSHIP

€ 16,5 billion

SOCIETAL

CHALLANGES

€ 28,6 billion

EXCELLENT

SCIENCE

€ 24,2 billion

OTHER *

€ 6 billion

H2020 budget € 74,8 billion

*OTHER: • Spreading excellence & widening participation • Science with and for society • JRC • EIT

ERC 54%

FET 11%

MSCA 25%

Research infrastructures

10%

4

FET in Horizon 2020

- collaborative research

- extend Europe’s capacity for advanced and paradigm-changing innovation.

- foster scientific collaboration across disciplines on radically new, high-risk ideas

FET: Novel ideas for radically new technologies

€ 2.6 billion to initiate radically new lines of technologies

visionary thinking … but very concrete mission

5

FET in Horizon 2020

To promote and support the emergence of radically new technology areas that will renew the basis for future European competitiveness and growth and will make a difference for society in the decades to come.

To initiate and shape the development of European research and innovation eco-systems around such future and emerging technologies, as seeds of future industrial leadership and potential solutions for societal challenges.

To turn Europe into the best environment for responsible and dynamic multi-disciplinary collaborations on such future and emerging technologies.

6

FET in Horizon 2020

FET mission

The power of FET complementary schemes

Individual research projects

FET-Open

Early Ideas

Critical mass making a case

FET Proactive

Exploration and

Incubation

Common research agenda

FET Flagships

Large-Scale

Partnering Initiatives

Exploring Developing Addressing

novel ideas topics & communities grand challenges

Roadmap based research Open, light and agile

FET: three complementary lines of action

7

FET in Horizon 2020

8

FET-Open - RIA Specific Challenge

The successful exploration of new foundations for radically new future technologies requires supporting a large set of early stage, high risk visionary science and technology projects to investigate new ideas.

Need of collaborative, agile, risk-friendly and highly interdisciplinary research approaches.

Therefore, FETOPEN encourages the driving role of new high-potential actors in research and innovation, such as excellent young, both female and male, researchers and high-tech SMEs that may become the scientific and industrial leaders of the future.

FET-Open Research and Innovation Actions

Scope:

This topic supports the early stages of research to

establish a new technological possibility. […]

9

New

knowledge

New technologies and

their applications

Dream

'Vision'

S&T Breakthrough

as Proof-of-Concept

Establish possibility

FET-Open - RIA

Scope:

[…] Proposals are sought for collaborative

research with all of the following characteristics ('FET gatekeepers'):

• Long-term vision

• Breakthrough scientific and technological target

• Novelty

• Foundational

• High-risk

• Interdisciplinary

Future and Emerging

Technologies

Interdisciplinary

Novelty

S&T targeted

Foundational

High-Risk

Long-term vision

FET-Open Research and Innovation Actions

10

FET-Open - RIA

Proposals must aim at one of the following impacts:

• Initiating or consolidating a baseline of feasibility or a radically new line of technology and its future uses by establishing the essential proofs-of-principle and their foundational scientific underpinnings, or

• Strengthening European leadership in the early exploration of visionary, new and emerging technologies, beyond academic excellence and with global recognition. This impact can be reinforced by involving also new high-potential actors such as young, both female and male, researchers and high-tech SMEs that may become the European scientific and technological leaders and innovators of the future.

• Impact is also sought in terms of the take up of new research and innovation practices for making leading-edge science and technology research more open, collaborative, creative and closer to society

11

FET-Open - RIA Expected Impact

FET-Open RIA: supporting early-stages of research to establish a new technological possibility. The evaluation is based on FET Workprogramme 2016-17

Collaborative projects up to € 3 Mio funding (indicative)

Single step submission, '1+15' pages

Early stages of R&I on any new technological possibility

Scope defined by FET gatekeepers

Proposals evaluated and ranked in one single interdisciplinary

Panel

12

FET-Open - RIA

FET-Open research and Innovation Actions (RIA)

Cut-off date: 30/09/2014 31/03/2015 30/09/2015 11/05/2016 17/01/2017

Budget: 77 M€ 38,5 M€ 38,5 M€ 84 M€ 84 M€

Proposals

received

643 670 805 545 369

Proposals

selected for

funding

24 11 11 22 ?

Proposals received

13

FET-Open - RIA

Cut-off date 17 January, 2017 details:

• 369 submitted proposals transferred to the evaluation tool SEP, checked for admissibility and eligibility.

• 523 experts, after checking the potential conflict of interest allocated to proposals. Maximum number of assignments is 7, minimum is 1.

• 45 vice chairs (quality controllers) selected with 6-8 proposals allocated to each vice chair.

• 61 cross readers identified and allocated to proposals.

14

FET-Open - RIA

Evaluation Process

15

Evaluation process

16

FET-Open - RIA

Feedback in 5 months

Ethics screening/ assessment

Panel review Cross-reading

Quality check

of IER with support from Vice Chairs

Remote evaluations write Individual

Evaluation Report

Eligibility check

REA

Proposal submission

ESR approval

Expert Assignment

Collate IER comments, median score calculated

Conflict of interest check at any stage of evaluation

Iterations if necessary

Each proposal is allocated to 4 remote evaluators

Each evaluators drafts comments and assigns scores

There are three main roles the external experts that support the evaluation process play: • Evaluators: ~520

Write remotely good quality individual evaluation reports and assign corresponding scores

Submit individual evaluation reports within deadline

• Vice-Chairs - Quality Controllers (QC): ~45 Fine tuning of proposals' descriptors Check the quality of individual evaluation reports

• Vice-Chairs - Cross Readers (CR): ~60

Cross-read proposals Take part in the panel review in Brussels

• In addition - Independent Observer

Observe the evaluation process Provide advice/suggestions to REA on the evaluation process

Evaluation process Main actors

17

Independent Observer

Appointed by REA, the Independent Observer may attend any meetings or monitor remote evaluation to ensure high quality evaluation.

Checks the functioning and running of the overall process.

Advises, in the observer report, on the conduct and fairness of the evaluation sessions and, if necessary, suggest possible improvements.

Does not evaluate proposals and, therefore, does not express any opinion on their quality.

The Independent Observer may raise any questions - please give him your full support.

Evaluation process

18

Your Role (1)

19

Remote evaluators Evaluator

Independence: You are evaluating in a personal capacity. You represent neither your employer, nor your country! Impartiality: You must treat all proposals equally and evaluate them impartially on their merits, irrespective of their origin or the identity of the applicants Objectivity: You evaluate each proposal as submitted; meaning on its own merit, not its potential if certain changes were to be made Accuracy: You make your judgment against the official evaluation criteria and the call or topic the proposal addresses, and nothing else Consistency: You apply the same standard of judgment to all proposals

Your Role (2)

20

Remote evaluators Evaluator

• You must submit your Individual Evaluation Reports (IERs) and the requested modifications, resulting from the quality check, in the electronic system SEP within the specified deadlines.

• In your work you will be supported by REA officials. Each cluster is

overseen by a Chair and a Deputy Chair – Project officers (POs) from the FET-Open unit, who will communicate with you in relation to your tasks and the quality of your work.

Confidentiality (1)

The expert:

a) must not use confidential information or documents for any purpose other than fulfilling their obligations under the Contract without prior written approval of the contracting party

b) must not disclose, directly or indirectly, confidential information or documents relating to proposals or applicants, without prior written approval of the contracting party.

In particular, the expert:

i. must not discuss any proposal with others, including other experts or contracting party or relevant service staff not directly involved in evaluating the proposal, except during the formal discussion at the meetings moderated by or with the knowledge and approval of the responsible contracting party or relevant service staff

Social media: Take great care not to post pictures or comments on

evaluation matters through social media

Evaluation process

21

Confidentiality (2)

ii. must not disclose:

• any detail of the evaluation process and its outcomes or of any proposal submitted for evaluation for any purpose other than fulfilling their obligations under the Contract without prior written approval of the contracting party

• their advice to the contracting party or relevant service on any proposal to the applicants or to any other person (including colleagues, students, etc.)

• the names of other experts participating in the evaluation.

iii. must not communicate with applicants on any proposal:

• during the evaluation , except in panel hearings between experts and the applicants organised by the contracting party or relevant service as part of the evaluation process;

• after the evaluation.

Evaluation process

22

An expert has a conflict of interest if he/she:

• was involved in the preparation of the proposal

• stands to benefit directly or indirectly if the proposal is accepted

• has a close family or personal relationship with any person representing an applicant legal entity

• is a director, trustee or partner or is in any way involved in the management of an applicant legal entity

• is employed or contracted by one of the applicant legal entities8 or any named subcontractors

• is a member of an Advisory Group set up by the Commission to advise on the preparation of EU or Euratom Horizon 2020 work programmes, or work programmes in an area related to the call for proposals in question

• is a National Contact Point, or is directly working for the Enterprise Europe Network

• is a member of a Programme Committee

Conflict of interest

Evaluation process

23

In the following situations the contracting party or relevant service will decide whether a conflict of interest exists, taking account of the objective circumstances, available information and related risks. when an expert: • was employed by one of the applicant legal entities in the last

three years

• is involved in a contract or grant agreement, grant decision or membership of management structures (e.g. member of management or advisory board etc.) or research collaboration with an applicant legal entity or the fellow researcher, or had been so in the last three years

• is in any other situation that could cast doubt on their ability to participate in the evaluation of the proposal impartially, or that could reasonably appear to do so in the eyes of an external third party.

Conflict of interest

Evaluation process

24

Consequences of a situation of conflict of interest:

(a) If a conflict of interest is reported by the expert or established by the contracting party or relevant service, the

expert must not evaluate the proposal concerned (consensus group), or take part in any panel review (including

possible hearings) where the proposal is discussed

(b) If a conflict becomes apparent at any stage of the evaluation, the expert must immediately inform the

contracting party or relevant service staff. If a conflict is confirmed, the expert must stop evaluating the proposal concerned. Any comments and scores already given by the expert will be discounted. If necessary, the expert will be replaced.

Conflict of interest

Evaluation process

25

If at any stage of the evaluation process you discover conflict of

interest, please inform us immediately!

Conflict of interest

Evaluation process

26

How to start?

27

Remote evaluators Evaluator

When you receive a task to evaluate a proposal, by clicking to the link in the email you will be lead to the electronic evaluation system (SEP). You will have to first login with your ECAS login and password (the credentials you use to login e.g. on the Participant portal). You will see a screen like the following:

What will you see?

28

Remote evaluators Evaluator

In SEP you will see all proposals assigned to you.

• Under "Panel" you will see which cluster a given proposal belongs to.

• The task will be Write IER. • Next you see the proposal numbers and acronyms. By clicking on

any one of these you will go to the proposal page, where you will find all its sections.

• The status could be "Assigned" (in the beginning), "Open", "Finished", "Declined", or "Cancelled"

• You are the Owner of the task. • The deadline listed is the final deadline for the whole remote

evaluation process. You will have to meet shorter deadlines, explained below.

• The score will be calculated, using the appropriate weights, after you assign scores to each criterion.

• The button View is the main button to go to the evaluation page where you can enter your comments and scores and where you will find comments on the quality of the evaluation.

Time schedule

Milestones Deadlines

Call cut-off: 17 January, 2017

Experts online briefing: 3 March, 2017

Remote evaluation starts: 4 March, 2017

At least 2 or all IERs submitted 10 March, 2017

At least 4 or all IERs submitted 20 March, 2017

All IERs submitted: 27 March, 2017

All quality recommendations implemented: 3 April, 2017

All corrections implemented: 10 April, 2017

Remote cross-readings completed: 8 May, 2017

Panel review in Brussels: 15-19 May, 2017

Result letters sent to applicants: 17 June, 2017

Grant Agreements signed: 17 September, 2017

Evaluation process

29

High level tasks Evaluation process

30

• If you have more than two proposals to evaluate, select two of them at your preference, accept the task and work on them within the timeframe specified above or indicated by the Chair or Deputy Chair.

• Read the proposals carefully. Make sure you understand well the criteria and sub-criteria for the evaluation, as well as the scoring system.

• Check to what degree the proposal is in the scope of the Call. • Evaluate the proposals against the evaluation criteria. For each proposal

complete an Individual Evaluation Report (IER). • Sign and submit your report in the electronic system SEP. After submission

every IER will be checked for its adherence to certain quality conditions.

• In case it is decided by the assigned quality controller that your report needs modifications, you will receive comments in the electronic system. Implement the improvements suggested by the quality controllers. This is a very important part of your work – any report you have submitted may need revisions and corrections. No IER will be accepted unless it respects the quality requirements.

Admissibility and eligibility of proposals have been checked by REA – however, if you spot an issue relating to eligibility, please contact us!

Admissibility & Eligibility

31

Evaluation process

Admissibility

1. Be submitted in the electronic submission system before the deadline given in the call conditions or rules of contest.

2. Be readable, accessible and printable.

3. Be complete

4. Include a series of supporting documents to determine the operational capacity (General Annexes - section B, para 3).

5. Include a draft plan for the exploitation and dissemination

of the results.

6. Respect page limits 32

Evaluation process

Part A: Administrative part of the proposal

Part B: Technical Annex : Section 1-3 (16 pages maximum)

Cover page

Section 1: S&T Excellence

Section 2: Impact

Section 3: Implementation

Section 4-5

Section 4: Members of the consortium

E.g. legal entity, CV, subcontract, third party

Section 5: Ethics and Security

Ethics self-assessment & supporting documents

Security checklist

Section 4 & 5 are not

covered by the page

limit.

Section 1,2 & 3 are strictly

limited to 15 pages!

WATERMARK!!!

Cover page strictly limited to 1 page!

Admissibility

33

Evaluation process

Eligibility

A proposal/application will only be considered eligible if: (a) Its content corresponds, wholly or in part, to the topic

description for which it is submitted

(b) At least three legal entities are present in the consortium. And :

• Each of the three must be established in a different EU Member State or Horizon 2020 associated country.

• All three legal entities must be independent of each other.

34

Evaluation process

Remote phase Individual Evaluation

Report (IER)

35

How to write an Individual Evaluation Report (IER)

Remote phase FET Gatekeepers

36

Long-term vision: a new, original or radical long-term vision of technology-enabled possibilities going far beyond the state of the art Breakthrough S&T target: scientifically ambitious and technologically concrete breakthroughs plausibly attainable within the life-time of the project. Foundational: the breakthroughs must be foundational in the sense that they can establish a basis for a new line of technology not currently anticipated. Novelty: new ideas and concepts, rather than the application or incremental refinement of existing ones. High-risk: the potential of a new technological direction depends on a whole range of factors that cannot be apprehended from a single disciplinary viewpoint. Interdisciplinary: the proposed collaborations must go beyond current mainstream collaboration configurations in joint S&T research, and must aim to advance different scientific and technological disciplines together and in synergy towards a breakthrough.

Remote phase In-out of scope

37

• Check if the content of the proposal you are evaluating corresponds, wholly or in part, to the description of the Call.

• The "Out of scope" should be ticked for proposals clearly outside the scope of the Call.

• If you consider that the proposal is out of scope, substantiate this in the dedicated section.

• Even if you consider the proposal "Out of scope", the IER must be completed and your assessment should be reflected in the scores for Criteria 1 & 2

Remote phase Write IER task (1)

38

• Assess only the proposal's explicit content and merits do not compare with other proposals and do not infer or deduct what is not written in the proposal.

• Do not make any comments on operational capacity, participation of third countries or human embryonic stem cells (hESC) in any sub-criterion (there are specific sections for these items).

• Assess the weaknesses, shortcomings and strengths of the proposal for each sub-criterion but do not repeat the same shortcoming or weakness in a different criterion and sub-criterion.

• Cover every sub-criterion and address all the relevant aspects of the evaluation sub-criteria.

• Comments should be well substantiated and sufficiently detailed. The comment is expected to have a reasonable length up to 1000 characters.

Remote phase Write IER task (2)

39

• Do not summarize the proposal in the IER. Only one short introductory sentence may be acceptable.

• Be specific in the comments: avoid too general comments giving impression you have not read the proposal. Comments should be explicit, factual, concrete, objective and respectful.

• Use proper and rich English language and make the comments understandable to the applicants.

• Do not cross reference (e.g. page number) to the proposal. • Do not cross reference in the evaluation report (e.g. "notwithstanding

the lack of interdisciplinarity described above, the research methods are appropriate").

• Do not make any recommendations to the consortium to improve the proposal.

Remote phase Write IER task (3)

40

• Avoid naming researchers but refer to the institution instead. • Avoid subjective statements (e.g. "in my opinion"). • Avoid colloquial expressions (e.g. "so-so", "hot topic", "there is not much

doubt"). • Do not express uncertainty (e.g. "It is hard to evaluate if this is

justified", "It seems...", "it looks like ..."). Instead use "... is not well explained" or similar expressions. Use "the proposal will" instead of "the project would". Do not formulate questions in the assessment.

• Avoid citations. • Avoid too rigid comments, categorical statements (e.g. "Info about XYZ

is absent").

41

Excellence Impact Implementation

• Clarity and novelty of long-term vision, and ambition and concreteness of the targeted breakthrough towards that vision. • Novelty, non-incrementality and plausibility of the proposed research for achieving the targeted breakthrough and its foundational character. • Appropriateness of the research methodology and its suitability to address high scientific and technological risks. • Range and added value from interdisciplinarity, including measures for exchange, cross-fertilisation and synergy.

• Importance of the new technological outcome with regards to its transformational impact on technology and/or society. • Impact on future European scientific and industrial leadership, notably from involvement of new and high potential actors. • Quality of methods and measures for achieving impact beyond the research world and for establishing European though leadership, as perceived by industry and society.

• Soundness of the workplan and clarity of intermediate targets. • Relevance of expertise in the consortium. • Appropriate allocation and justification of resources (person-months, equipment).

Threshold: 4/5 Weight: 60%

Threshold: 3,5/5 Weight: 20%

Threshold: 3/5 Weight: 20%

Evaluation criteria

42

Excellence

1. Clarity and novelty of long-term vision, and ambition and concreteness of the targeted breakthrough towards that vision.

• Is there a clearly defined novel long-term vision? • Is there a clear description of the proposed breakthrough

research? Is the research well targeted towards an attainable goal or represents continuation of research along various parallel paths already being followed?

• Relevance of the targeted breakthrough for the long-term vision – it should be outlined how the breakthrough would lead to the long-term vision.

• Are the objectives well described in the proposal? Are they specific, measurable, achievable, relevant, and time-bound within the duration of the project?

Evaluation criteria RIA

43

2. Novelty, non-incrementality and plausibility of the proposed research for achieving the targeted breakthrough and its foundational character.

• Is the analysis of the state-of-the-art comprehensive and

complete? • Is the proposed research novel? – It should be more than

continuation of research being already done or pursued currently.

• Is the research proposed ambitious and far-reaching, not just another small incremental step along a path already followed?

• Is the research plausible? • Does the research proposed have a potential to start a new

line of investigation, which can be followed for years and can lead to completely new technologies?

Excellence

Evaluation criteria RIA

44

3. Appropriateness of the research methodology and its

suitability to address high scientific and technological

risks.

What is the relevance of the chosen methods for the

achieving the proposal objectives?

Is the methodology "risk-friendly"? The details of risk

analysis and mitigation plan should be addressed under

Implementation.

Evaluation criteria RIA

Excellence

45

4. Range and added value from interdisciplinarity, including measures for exchange, cross-fertilisation and synergy. • To what extent the main idea requires involvement of

knowledge and methodology from different disciplines? • How "distant" are these disciplines involved? (There are

disciplines, which traditionally work together, while others speak different languages and use different methodologies. The expectation is that putting together distant disciplines may bring qualitatively new solutions).

• How are these different disciplines intertwined and how the ideas from different disciplines support the scientific breakthrough?

• The interdisciplinarity of the ideas should be addressed here, while the appropriateness of the composition of the consortium should be treated under Implementation.

Excellence

Evaluation criteria RIA

46

Impact

1. Importance of the new technological outcome with regards to its transformational impact on technology and/or society.

• Is the long-term technological outcome clearly defined? • Is the transformational impact on technology, on society or on

both convincingly argued? • It is not sufficient to discuss only the impact based on peer

reviewed publications!

Evaluation criteria RIA

47

Impact

Evaluation criteria RIA

2. Impact on future European scientific and industrial leadership, notably from involvement of new and high potential actors. • Is the general impact on future European scientific and industrial

leadership properly discussed? • Below are some examples of involvement of new and high

potential actors. Keep in mind that this list is neither exhaustive nor mandatory. It is not expected that the applicants will address all of these and may address none of them but may find another way to satisfy this sub-criterion. o Are there researchers at the start of their career, especially principle

investigators, who would keep up the research in a long run? or o Are there SMEs in the consortium with sufficient budget, which would

take up the results from the project and would eventually carry them over towards innovation and market realization? or

o Are there new actors, who are usually underrepresented in the specific tasks to be performed by the project? or something else…

48

3. Quality of methods and measures for achieving impact beyond the research world and for establishing European through leadership, as perceived by industry and society. • Are the "standard" measures (scientific publications, website)

properly described and planned? • Does the proposal go sufficiently far beyond the "standard"

measures? For example, are there additional measures, based on new social media included?

• Is it clearly explained how the dissemination measures would support achieving the expected impact?

Impact

Evaluation criteria RIA

49

Implementation

1. Soundness of the workplan and clarity of intermediate targets.

• Is there a clear description of the tasks that will be performed,

with specific deliverables allowing the verification of the execution of the tasks? Are there well defined work packages with relevant interdependencies among them? Is the time-dependence of the different tasks meaningful and convincing?

• Are the intermediate targets (milestones) properly defined and timely? These should be major achievements serving as verification points if the work should continue according to the original plan or alternative paths should be followed.

• It is very important to remember that the work plan, described in the proposal will become Description of action (DoA) in case the proposal is selected for funding!

Evaluation criteria RIA

50

Implementation

Evaluation criteria RIA

2. Relevance of expertise in the consortium. • Is the expertise in the consortium broad enough in

order to tackle all proposed tasks and is it of high enough quality?

• Is the interdisciplinarity of the ideas in the proposal (if present) properly reflected in the composition of the consortium?

• Is the consortium well-balanced, without redundancies?

51

3. Appropriate allocation and justification of resources (person-months, equipment). • Do the person-months allocated correspond to the proposed

tasks to be performed? • Is the necessary equipment present or properly described and

budgeted? • Is the overall budget comprehensive, well-balanced, and

convincing?

Implementation

Evaluation criteria RIA

Consistency between comments and scores:

Remote phase Individual Evaluation

Report (IER)

52

• The score must objectively reflect the aggregated comments under the respective criterion.

• Employ the whole range of scores. Half-marks can be used except for 0.5. Please do not use 0.5 since it does not have meaning according to our scoring table.

• Proposals should not be rewarded/penalised for the same issue twice .

• Do not penalise applicants that did not provide detailed breakdown of costs – they are not required.

• The score should not be influenced by Ethics or by (non-) participation in the Open Research Data!

Remote phase Scoring

53

Additional issues to be checked

• Give your view on whether each applicant in the proposal being assessed has the necessary basic operational capacity to carry on their proposed activities based on the information provided in curriculum vitae or description of the profile of the applicant, relevant publications or achievements, relevant previous projects or activities, description of any significant infrastructure or any major items of technical equipment (consider also the sections 4 and 5 of the proposal).

• You should choose "NO" only if you conclude, after checking information in the proposal, that the applicants are not capable of running the project, or they cannot have access to the facilities they describe.

• Otherwise tick "YES" and introduce a comment in Criterion 3 - Quality and efficiency of the implementation and Expertise of the consortium.

Remote phase Operational capacity

55

• If applicable, comment on the use of human embryonic stem cells (hESC) under the dedicated section (do not introduce comments regarding hESC or other ETHICS issues under the evaluation sub-criteria!)

Remote phase hESC

56

• The 28 EU Member States, the associated countries, countries listed in Section A of the General Annexes of the H2020 work programme, as well as International European interest organisations are eligible for funding.

• IMPORTANT: Associated countries such as Switzerland, Iceland, Israel, or Norway are participating in Horizon 2020 under the same conditions as EU Member states. Participants from these countries must be treated equally with the EU Member states.

• Participants, which do not belong to any of the eligible categories mentioned above (including international organisations not working in European interest) may receive funding only exceptionally, when their participation is essential to carry out the action. In such a case you have to comment in the dedicated section "Exceptional funding of third country participants/international organisations" of the IER whether funding should be granted or not.

• Non-exhaustive list of third countries to be discussed in this section: Australia, Brazil, China, India, Japan, Mexico, Russian Federation, United States, …

Remote phase EU funding to third

countries (1)

57

58

Partners belonging to this categories may receive funding only in exceptional cases – when their participation is essential to carry out the action e.g.: • unique expertise; • access to unique know-how; • access to unique research infrastructure; • access to a particular geographical environment; • access to unique data. In those exceptional cases, the IER should clearly mention whether funding should be granted or not. NB: when EU funding is NOT granted, this entity must be considered to participate with its own funding, regardless of whether own funding is mentioned in the proposal or not.

Exceptional funding of third country participants / International organisations not in European interest

Remote phase EU funding to third

countries (2)

59

• Resubmissions are evaluated as if they are submitted for the first time, i.e. avoid comparison with previous versions of the proposal.

• If the proposal's Technical Annex Sections 1-3 exceed 15 pages, please contact your cluster Chair or Deputy Chair (REA POs) for further instructions.

• ETHICS ASSESSMENT is not part of the remote evaluation!

• Do not evaluate proposals more favourably because they participate in the Open Research Data Pilot and do not penalise them for opting out of the Pilot.

Remote phase Other issues

Quality check of IERs:

Each IER will be quality-checked by a Quality Controller (QC) for:

• relevance and completeness of the comments for all sub-criteria,

• consistency between comments and scores

• inappropriate language

Immediately after an evaluator submits an IER, the quality control will start.

The QCs provides feedback to REA in case an IER needs to be re-opened for further improvement.

If applicable, each evaluator should revise a given IER based on REA request, following input from QC.

Remote phase Individual Evaluation

Report (IER)

60

Some typical problems, requiring the intervention from the QCs (1)

• Comments start with a too long summary of the proposal

• Particular sub-criterion not addressed at all • Comments relevant for one criterion entered

under another criterion • Scores not aligned with the comments • Comments too short or comment too long • Comments involve emotional statements,

personalized statements, or abusive language

QC intervention Remote phase

61

Some typical problems, requiring the intervention from the QCs (2)

• Too generic comments, giving the impression that the evaluator has not read the proposal

• Too rigid and concrete comments – "there is no description of…", "there is complete lack of…"

• Comments with examples of research being conducted elsewhere (and citations)

• Suggestions for improvement of the proposal

QC intervention Remote phase

62

The QC comments can be seen in SEP, in the left side of the IER's screen under the Task comments field (please press on 'Expand comments')

How to implement changes? (1)

Remote phase

63

Check the latest comments per each criterion

How to implement changes? (2)

Remote phase

64

65

Approval of the Evaluation Summary Report (ESR)

• Once all four IERs have been drafted, quality checked, and submitted, you will receive access to the consolidated report built from the four IERs (the Evaluation Summary Report or ESR). In the ESR all the comments are collated per sub-criterion in an anonymous way.

• At this time you will receive an invitation from the responsible PO to review the ESR. In order to do this, please go to SEP

• From the top row menu select "Proposals" then in the "Call" filter enter

the indicative of the current call H2020-FETOPEN-1-2016-2017-RIA_17-01-2017. You may search the proposals either by acronym or number, using the filters "Acronym" or respectively "Proposal".

ESR approval (1) Remote phase

66

• Click on the "R" icon under the column ESR • Click on the "View Tasks" button in the bottom side of the

window. This will bring the screen with the ESR. • In the ESR window press the "Expand" button. This will allow

you to see the comments provided by all four remote evaluators.

• At this stage you will have the opportunity to check the

complete set of comments for the proposal under consideration and to confirm your comments and scores..

ESR approval (2) Remote phase

67

• In exceptional cases, when you discover that you have overlooked a crucial specific point or you have made a substantial mistake, please inform the POs responsible for the cluster, the proposal under consideration belongs to. This will mean that you are requesting reopening of the IER and an opportunity to implement corrections in your own comments. The PO will analyse the case and if it is decided that a reopening is warranted you will be given a short period of time to implement your corrections.

• Do not comment in your revision on other evaluators' IERs or on your own opinion from before the revision!

• Please do not ask for reopening for minor points or for errors you believe have been made by the other evaluators!

• NOTE: Should this operation be required in case of one or several of your IERs, you will receive detailed instructions from the Call Coordinator and the POs.

ESR approval (3) Remote phase

68

Additional Information

69

Feedback to proposers – Evaluation Summary Report (ESR)

• Collation of all evaluators' comments , per sub-criterion, which may

be mutually contradicting (no consensus) - full transparency

• Proposal score calculation (per criterion) - median of the scores

from individual evaluators

• Final score per criteria is decided by the final panel review

• Total final score for the proposal is calculated as the weighted sum

of the final scores from the 3 evaluation criteria

• Final panel review adds also some additional comments

Additional Info Remote phase

70

Cluster Chair Deputy chair

1 Antonio LOREDAN

Antonio.LOREDAN@ec.europa.eu

Adriana GODEANU METZ

Adriana.GODEANU-

METZ@ec.europa.eu

2 Elisabet SALAS

Elisabet.SALAS-IBANEZ@ec.europa.eu

Maksym TSYTLONOK

Maksym.TSYTLONOK@ec.europa.eu

3 Salvatore SPINELLO

Salvatore.SPINELLO@ec.europa.eu

Iliana KOSTOVA

Iliana.KOSTOVA@ec.europa.eu

4 Guadalupe SEPULCRE

Guadalupe.SEPULCRE@ec.europa.eu

Olivier DAHON

Olivier.DAHON@ec.europa.eu

5 Maciej LOPATKA

Maciej.LOPATKA@ec.europa.eu

Bart VAN CAENEGEM

Bart.VAN-CAENEGEM@ec.europa.eu

Roumen BORISSOV (Call Coordinator) Roumen.BORISSOV@ec.europa.eu

Adelina NICOLAIE Adelina.NICOLAIE@ec.europa.eu

71

Functional mailbox: REA-FET-OPEN-2017-1-EVALUATORS@ec.europa.eu

• List of countries and applicable rules for funding: described in part A of the General Annexes of the General Work Programme.

• Eligibility and admissibility conditions: described in part B and C of the General Annexes of the General Work Programme.

• Provisions, proposal templates and evaluation forms for the type(s) of action(s) under this topic: Specific provisions and funding rates Specific proposal template for FET-Open research and innovation actions Specific evaluation form for FET-Open research and innovation actions H2020 General MGA -Multi-Beneficiary Annotated Grant Agreement

• Work Programme H2020 Work Programme 2016-17: Future and Emerging Technologies (FETs)

Useful links (1)

72

H2020 website: http://ec.europa.eu/programmes/horizon2020/

Participant portal:

http://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/index.html

FET Work Programme call text :

http://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-fet_en.pdf

73

Useful links (2)

74

Thank you for your attention!