Space business, entrepreneurship, outreach and education · 5/6/2018 · † Attract the interest...

Gisèle Van Bunnen & Diana Mjaschkova-Pascual

B1 – Space Research Unit, REA

Space business, entrepreneurship, outreach and education

DT-SPACE-08-BIZ-2018 Space outreach and education

Purpose of this briefing

• To ensure common understanding of the WP's key elements in order to guarantee a single and harmonized approach in the evaluation

• To review the evaluation criteria and provide guidelines, including what is addressed under which sub-criterion

General info

• During the briefing and remote evaluation, all evaluators must remain anonymous in order to avoid any bias

• Evaluators are invited to ask questions by email ([email protected]) Please specify the topic in the e-mail subject: 08-BIZ. We will reply to these questions after the briefing by email.

• An independent observer is attending the topic briefing to monitor the process

• The slides and the FAQs will also be available at the Space 2018 Evaluation website at https://ec.europa.eu/info/h2020-space-2018_en

• You have received by email the IER Guidelines document that summarises the information on the evaluation criteria presented here

BIZ-8 topic

overview

DT SPACE-08-BIZ-2018: Space outreach and education

What?

Coordination and support actions (CSA): Description: Actions consisting primarily of accompanying measures such as standardisation, dissemination, awareness-raising and communication, networking, coordination or support services, policy dialogues and mutual learning exercises and studies, including design studies for new infrastructure and may also include complementary activities of strategic planning, networking and coordination between programmes in different countries.

How much?

- Total budget: 2 million EUR

- Estimated amount per proposal: 1-2 million EUR

- Duration typically: 2-3 years

- Funding rate: 100 %

- Focus area: Digitising and transforming European industry and services

- 20 proposals to evaluate

DT SPACE-08-BIZ-018: Space outreach and education

• Initiative attracting the interest of young people towards space, motivating them to choose space related careers and education in STEM

• Create impact on a significant number of students of different ages, their families and the general public

• Engage academia and educators at different levels of education targeting young children, teenagers, women students and underprivileged communities

• Run sustainable education activities, inside and outside the classroom using online resources, public spaces, contests

• Take into account similar activities of ESA and national education programmes

• Explore options for space activities within the existing KICs (Knowledge and Innovation Communities) of the EIT (European Institute of Technology) and a specific space KIC

7

Some reference policy action from DG GROW

concept and methodology

the concept• Design and run sustainable education and outreach activities

• Attract the interest of a significant number of students for space, space-related subjects and steer them towards education paths in the fields of science, technology, engineering and mathematics.

• Create a relevant impact on student’s families and the general public in terms of News coverage, social-media interest, stakeholders’ involvement?

• Do key advancements of the European space programmes have a privileged position in the proposal without being the main objective of the proposal?

the methodology• Engage academia and educators involved in different education levels, targeting

different demographics including young children and teenagers

• Pay particular attention to stimulating interest amongst female students and reaching children in underprivileged communities?

• Focus in the context of the classroom or outside the normal classroom environment, making use of space educational centres or online resources, including contests and public exhibitions (for instance in science museums)?

• Promote the network European space education and outreach actors and reach out to a significant number of students, their families and the general public.

• Achieve a significant coverage by media and attention by stakeholders and help increase the political support for European space programmes and initiatives within the EU and national Parliaments.

• Increase the number of students that opt for a technical career related to space when compared to the general population of students in their cohorts;

• Promote research in collaboration with universities.

• Reinforce links between space and the EIT (European Innovation Institute of Technology ) KICs (Knowledge and Innovation Communities )and explore options for a dedicated space KIC.

Impacts WP topic 08-BIZ

Topic on the Participant Portal

10

https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/dt-space-08-biz-2018.html

Individual Evaluat ion

Report ( IER)

A w a r d c r i t e r i a , s c o r e s a n d w e i g h t i n g

AW A R D C R I T E R I A

Excellence Impact Quality and efficiency of the implementation

Scoring: 0-5, weakness / shortcoming

0 — The proposal fails to address the criterion or cannot be assessed due to missing or incomplete information.

1 — Poor. The criterion is inadequately addressed, or there are serious inherent weaknesses.

2 — Fair. The proposal broadly addresses the criterion, but there are significant weaknesses.

3 — Good. The proposal addresses the criterion well, but a number of shortcomings are present.

4 — Very Good. The proposal addresses the criterion very well, but a small number of shortcomings are present.

5 — Excellent. The proposal successfully addresses all relevant aspects of the criterion. Any shortcomings are minor.

Thresholds

Threshold for individual criteria = 3 + Overall threshold, applying to the sum of the three individual scores, =10.

Criter ion 1

- Excel lence

E X C E L L E N C E

• How does the proposal address the specific challenge and scope ofthe work programme topic?

• Are the objectives clear, measurable, realistic and achievablewithin project duration?

1.1 Clarity and pertinence of the objectives

EXAMPLES – for each sub criteria in the hide slides

STRENGHTS

+ The objectives are clearly stated and are (fully) within the scope of the BIZ-8-2018 topic.

+ The objectives are described clearly and appropriate quantifiable Key Performance Indicators have been identified to monitor xxxx.

WEAKNESSES/SHORTCOMINGS

- The specific objectives have not been presented with sufficient precision and have not been convincingly demonstrated as fully pertinent to the Biz-8-2018 topic.

- The objectives are formulated in too generic terms and as general problems to be solved rather than as solutions to be provided. They are also not focused adequately on business cases.

1.2 Soundness of the concept, and credibility of the proposed methodology

Is the concept of the proposal sound (i.e. based on models or assumptions which are valid and convincing)?

• Design and run sustainable education and outreach activities

• Attract the interest of a significant number of students for space, space-related subjects and steer them towards education paths in the fields of science, technology, engineering and mathematics.

• Create a relevant impact on student’s families and the general public in terms of News coverage, social-media interest, stakeholders’ involvement?

• Do key advancements of the European space programmes have a privileged position in the proposal without being the main objective of the proposal?

E X C E L L E N C E

1.2 Soundness of the concept, and credibility of the proposed methodology

Does the proposal show that the proposed methodology is appropriate and effective for achieving the stated objectives?

• Engage academia and educators involved in different education levels, targeting different demographics including young children and teenagers

• Pay particular attention to stimulating interest amongst female students and reaching children in underprivileged communities?

• Focus in the context of the classroom or outside the normal classroom environment, making use of space educational centres or online resources, including contests and public exhibitions (for instance in science museums)?

E X C E L L E N C E

S o u n d n e s s o f t h e c o n c e p t , a n d c r e d i b i l i t y o f t h e p r o p o s e d m e t h o d o l o g ySTRENGHTS

+ The concept is clear, achievable and sound, using meteorological and chemical transport models to determine air pollution using Copernicus data.

+ The proposed methodology is credible, appropriate and coherent. How the specific challenges will be addressed is demonstrated very well, for example by making a clear distinction between customers and users of the proposed service platform for a broad range of users in xxx, yyy and zzz.


- The concept is sound in general terms as it builds on a current service concept delivered by xxx to zzz and aims to include Sentinel data and UAV in the service. However, the requirements of xxx in terms of the xxx information have not been considered in sufficient detail and therefore the added value of the outcomes has not been demonstrated convincingly.

- The methodology is presented briefly, sufficient detail is not presented, which affects the overall credibility. For example, the xxxx is not described in sufficient detail.

E X C E L L E N C E

E X C E L L E N C E

1.3 Quality of the proposed coordination and/or support measures – 08-BIZ

Are any national or international research and innovation activities which will be linked with the project, especially where the outputs from these will feed into the project, described?

In particular, does the proposal:

• Take into account similar activities of ESA and national education programmes ?

• Identify links with the Knowledge and Innovation Communities (KICs) of the European Innovation Institute of Technology (EIT) and possible scope for dedicated activities for space ?

Q u a l i t y o f t h e p r o p o s e d c o o r d i n a t i o n a n d / o r s u p p o r t m e a s u r e s

STRENGHTS

+ The proposal is able to demonstrate an easy access to a large network of xxxx. This networking capabilities a very strong asset of this proposal.


- Since several xxx already offer acceleration programmes, it is not presented clearly how this proposal will complement and bring an added value to the services already available.

E X C E L L E N C E

Criter ion 2

- Impact

• How much would the proposal contribute to each of the expectedimpacts mentioned in the work programme?

(Here you should base your evaluation on the assumption that the proposed concept and methodology would work in order to avoid penalising the proposal twice for the same shortcomings.)

I M P A C T

2.1 The extent to which the outputs of the project would contribute to each of the expected impacts mentioned in the work programme under topic DT SPACE-08-BIZ-2018: Space outreach and education

I M P A C T

• Promote the network European space education and outreach actors and reach out to a significant number of students, their families and the general public.

• Achieve a significant coverage by media and attention by stakeholders and help increase the political support for European space programmes and initiatives within the EU and national Parliaments.

• Increase the number of students that opt for a technical career related to space when compared to the general population of students in their cohorts;

• Promote research in collaboration with universities.

• Reinforce links between space and the EIT (European Innovation Institute of Technology ) KICs (Knowledge and Innovation Communities )and explore options for a dedicated space KIC.

Impacts WP topic 08-BIZ

T h e e x t e n t t o w h i c h t h e o u t p u t s o f t h e p r o j e c t w o u l d c o n t r i b u t e t o e a c h o f t h e e x p e c t e d i m p a c t s m e n t i o n e d i n t h e w o r k p r o g r a m m e u n d e r t h e r e l e va n t t o p i c

STRENGHTS

+ The proposal demonstrates clearly that the outcomes will contribute significantly to the expected impacts mentioned in the topic description, for example…


- The proposal only partly addresses the enabling of value-added services without the need to deploy storage and processing facilities. Moreover, it has not been demonstrated convincingly that the outcomes will foster the establishment of interoperable access facilities to all EU Member States.

I M P A C T

I M P A C T

2.2 Quality of the proposed measures to:- exploit and disseminate the project results (including management of

IPR), and to manage research data where relevant - communicate the project activities to different target audiences

• Please assess the quality of the "draft Plan for the Exploitation and Dissemination of the project's Results (PEDR)" with respect to the specific measures to be implemented both during and after the end of the project.

• Is the plan to disseminate project results to audiences who might use the results in their own work (e.g. Presentations in conferences, publications in peer review journals etc.) appropriate?

• Are activities to communicate project activities to multiple audiences beyond the project's own community (media, public etc.) included? How are these tailored to the needs of various audiences?

• Is the strategy for the management of IPR appropriate? Does it describe the main background IPR needed to carry out the project, its ownership and accessibility, authorisation to use Third Party rights, and the ownership principles that will govern the project results?

STRENGHTS

+ The envisaged dissemination and communication measures are very good and are tailored to maximise customer participation, for example by visiting customers and using targeted webinars.

+ Proposal also presents an effective communication plan that clearly defines the target audiences and activities foreseen, with specific strategies devised for each of them.

+ Key aspects of IPR, in particular licensing and the potential for patenting, have been considered with responsibilities for activities in the area being clearly defined. There is clear consideration of both background and foreground IP in the proposal.


- Plans for exploitation are not convincing and are not clearly presented. In particular, the path to exploitation at the partner level is not adequately outlined.

While it is stated that IPR will be addressed in the consortium agreement, the management of IPR has not been sufficiently considered in the proposal.

- Dissemination and communication activities are treated together with an insufficiently clear distinction between them. The proposed activities are in general appropriate but explained too superficially.

- The communication plan has not been presented in sufficient detail as it is described in terms of generic goals rather than specific activities.

I M P A C T

Q u a l i t y o f t h e p r o p o s e d m e a s u r e s t o …

Criter ion 3 - Qual i ty

and eff ic iency of the

implementat ion

3.1 Quality and effectiveness of the work plan, including extent to which the resources assigned to work packages are in line with their objectives and deliverables

• How good is the overall structure of the work plan?

• Have the different work packages been scheduled and linked appropriately? Are their inter-dependencies clear?

• Are milestones and deliverables well defined and appropriately timed?

• Are the work packages and associated tasks sufficiently described to provide a good understanding of the work involved and justify the proposed resources to be allocated?

Q u a l i t y a n d e f f i c i e n c y o f t h e i m p l e m e n t a t i o n

Q u a l i t y a n d e f f e c t i v e n e s s o f t h e w o r k p l a n , i n c l u d i n g e x t e n t t o w h i c h t h e r e s o u r c e s a s s i g n e d t o w o r k p a c k a g e s a r e i n l i n e w i t h t h e i r o b j e c t i v e s a n d d e l i v e r a b l e s

STRENGHTS

+ The overall structure of the work plan is described well and is appropriate for the achievement of the objectives. The work package descriptions are thorough and appropriate resources have been allocated. The timing of work packages, tasks and activities is also appropriate and the deliverables match the objectives well.


- Some of the work package descriptions are defined in terms that are too generic. For example, a survey will be used for collecting user needs, but the nature of the survey or its target subjects have not been defined with enough precision.

- The allocation of resources across tasks is not balanced well. For example, WP5 is heavily over-estimated.

- How the tasks and work packages relate to the presented work breakdown structure is not sufficiently described i.e. it is not sufficiently clear how nodes in figure xx relate to the tasks in the work package descriptions.


3.2 Appropriateness of the management structures and procedures, including risk and innovation management


• Are the described organisational structure and decision making mechanisms appropriate to the scale and complexity of the project?

• Are the management procedures, including quality management and conflict resolution, appropriate to the scale and complexity of the project?

• Are critical internal and external risks related to project implementation described with appropriate mitigation measures and severity/probability estimates?

Q u a l i t y a n d e f f e c t i v e n e s s o f t h e w o r k p l a n , i n c l u d i n g e x t e n t t o w h i c h t h e r e s o u r c e s a s s i g n e d t o w o r k p a c k a g e s a r e i n l i n e w i t h t h e i r o b j e c t i v e s a n d d e l i v e r a b l e s

STRENGHTS + Management structures are appropriately addressed with roles and conflict resolution procedures being clearly defined.+ Risk management approaches are well described by reference to standards based approaches, an appropriate initial risk matrix is also presented.+ Appropriate risks have been identified with generally satisfactory mitigation measures.


- Project management principles are described, however the detailed management structures and procedures (for example the overall management board structures and the decision making and conflict resolution) are not adequately described.

- While risk management is, in general, adequate, risks pertaining to the xxxx and yyyy are not sufficiently considered.


3.3 Complementarity of the participants and extent to which the consortium as a whole brings together the necessary expertise


• How is the complementarity of the participants described?

• Does the consortium bring together the necessary expertise for the project?

C o m p l e m e n t a r i t y o f t h e p a r t i c i p a n t s a n d e x t e n t t o w h i c h t h e c o n s o r t i u m a s a w h o l e b r i n g s t o g e t h e r t h e n e c e s s a r y e x p e r t i s e

STRENGHTS

+ The consortium is balanced well and shows good complementarity in terms of the required technical and market expertise, and the presence of a large xxx solution provider is very valuable in providing world-wide market access.

+ The participants have the complementary skills required for the development of the service platform and of xxx, including extensive experience with xxx models and xxx in non-European countries where a large market for the outcomes is envisaged. The consortium covers all the required expertise.


- The consortium consists of x participants with complementary skills, however the high level of subcontracting required indicates that not all the required expertise is present in the consortium.


3.4 Appropriateness of the allocation of tasks, ensuring that all participants have a valid role and adequate resources in the project to fulfil that role


• Does each partner have a valid role and adequate resources (both effort and budget) to fulfil that role?

• If "Other Direct Costs" exceed 15% of personnel costs:

have they been explained adequately?

Are these costs "reasonable, justified and comply with the principle of sound financial management, in particular regarding economy and efficiency"(Art. 6.1.)?

Please particular attention should be given for proposal with very high Other Direct Costs.

• If any of the work will be subcontracted to third parties, has this been justified convincingly?

Ap p r o p r i a t e n e s s o f t h e a l l o c a t i o n o f t a s k s , e n s u r i n g t h a t a l l p a r t i c i p a n t s h a v e a v a l i d r o l e a n d a d e q u a t e r e s o u r c e s i n t h e p r o j e c t t o f u l f i l t h a t r o l e

STRENGHTS

+ Tasks have been allocated appropriately and in accordance with the skills and expertise of each partner.

+ Each participant has a valid role with adequate resources.

WEAKNESSES/SHORTCOMINGS- The level of other direct costs in the proposal is not sufficiently justified, for example, the travel budgets proposed are not substantiated by a number of trips foreseen, similarly the materials budgets for partners 2 & 3 do not have sufficient explanation.- The large allocation of effort to partner 2. is inadequately justified given the fact that its experience and expertise in the domain is insufficiently demonstrated, in particular in the…- The rationale for the subcontracting costs required is not adequately explained.


Exceptional funding for 3rd countries or international organisations

Eligible for funding only exceptionally if the Commission deems participation of the entity essential for carrying out the action

• International organisations (UNESCO etc.)

• Other third countries (USA, Canada, Japan…)

• (Switzerland IS an Associated country in H2020 Space in 2018)

Automatically eligible for funding:

• EU Member States

• Associated Countries: Iceland, Norway, Albania, Bosnia and Herzegovina, the former Yugoslav Republic of Macedonia, Montenegro, Serbia, Turkey, Israel, Moldova, Switzerland, Faroe Islands, Ukraine, Tunisia, Georgia, Armenia

• International organisations of European interest: the majority of members are Member States or H2020 Associated Countries & principal objective is to promote scientific and technological cooperation in Europe (ECMWF etc.)

• Certain developing countries (see list in Annex A, General Annexes)

Need opinion from experts in

IER!

moderators will assist the experts in Consensus

Meetings

This section is not covered by the page limit. Discard information

that should be in Part 1-3!

Operational Capacity

PART B – 4. Members of the consortium

4.2. Third parties involved in the project (including use of third party resources)• Does the participant plan to subcontract certain tasks

(please note that core tasks of the project should not be sub-contracted)

• Does the participant envisage that part of its work is performed by linked third parties

• Does the participant envisage the use of contributions in kind provided by third parties (Articles 11 and 12 of the General Model Grant Agreement)

4.1. Participants (applicants)

• a description of the legal entity and its main tasks• a curriculum vitae + profile of the persons• a list of up to 5 relevant publications, and/or products,

services • a list of up to 5 relevant previous projects or activities• a description of any significant infrastructure and/or any

major items of technical equipment

OPERATIONAL CAPACITY = a participant's experience, expertise, availability of infrastructure, equipment, human resources etc. to carry out proposed activity based on the information provided

1. Flagging the issue as part of the Individual Evaluation: give your view on whether each applicant has the necessary basic operational capacity

2. Dealing with the issue at the Consensus Meeting

36

Quali ty of

the reports

‘If this was my proposal,would I find this report fair, accurate, clear and

complete?’

… even if it brings bad news …

Quality of the reports

Avoid:Comments not related to the criterion in question.

Double counting of same shortcoming/weakness under different criteria.

Contradicting statements relative to strengths and weaknesses.

Comments too short / too long / inappropriate.

Categorical statements not properly verified and quoting texte.g. “The proposal doesn’t mention user requirements” – when there is a short reference…

Scores that don’t match the comments.

Making recommendations.

Evaluating on potential, proposals need to be evaluated as they are.

Discrimination/political incorrectness.


As Evaluator you have to ensure that…

Comments in IERs and CRs are:• Specific to the relevant criterion• Not double counting of same shortcoming/weakness under

different criteria• Clear and substantial• Facts, not opinions: "We think that…" "This proposal is… "• Consistent with the score, balancing strengths and

weaknesses• Of adequate length: not just one sentence, not a booklet!

Verify your statements.

Comments judge the proposal, they do not summarise it.

Do not provide advice on improving the proposal.

Good vs. poor comments


Poor comments merely echo the score

“The innovative aspects of the research programme are poor”.

Good comments explain it

“This proposal is not innovative in X or Y, and it does not take Z into account”.

Good comments are clear

“The resources in WP 4 and 6 are seriously underestimated given the complexity of the activity proposed”.

Poor comments are ambiguous

“The resources for the project are unrealistic”


Poor comments are vague, subject to interpretation

“We think the management plan is probably inadequate given the duration of the project and the number of partners”.

Good comments are precise and final

“The management plan is inadequate. It does not include clear risk assessment; it doesn't foresee a problem-solving mechanism in the event of disputes between partners”.


Poor comments are inaccurate and provide an opening for a complaint

“There is no discussion of a dissemination strategy.”

“There is only one industrial partner in the consortium.”

“The coordinator is not adequately experienced."

Good comments close the question

“Dissemination activities are listed but the proposal doesn't have a clear dissemination strategy”.

“The consortium doesn't include a sufficient industrial participation.”

“The coordinator does not demonstrate in the proposal an adequate level of experience in this field.”


Poor comments include words like…

PerhapsThinkSeems

AssumeProbably

Good comments include words like…

BecausePercent

SpecificallyFor example


Useful vocabulary

Insufficient, minimal, fails to describe, unacceptable, inadequate, very generic, not evident, unfocused, very weak, bad, does not meet the requirements, inappropriate, limited, unclear, not sound enough, not specified, no significant impact, unjustified, overestimated …

Extremely relevant, credible, very clear, precisely specified, realistic, very innovative, extremely well suited, timely, convincing, comprehensive, high quality, justified, very well identified, strong, highly effective, thoughtful, very promising, evidence, well-formulated, carefully prepared, very professionally prepared, fully in line, very profound, sound, very convincingly integrated, clearly articulated, coherent, well balanced, very plausible, ambitious, clear advances, well above average …


Frequently Asked Questions DT-SPACE-08-BIZ-2018

Q: Under call H2020-SPACE-2018, topic DT-SPACE-08-BIZ-2018, is there any particular Knowledge and Innovation Community (KIC) among the existing six KICs that applicants should address?

A: All six KICs may be addressed. You should define in your proposal the KICs you want to consider and the rationale behind it. Climate KIC to be relevant for Copernicus for example.

Process reminders

• Evaluators: accept the proposals assigned in SEP (by EOB Monday 23.4.)

• In case of non-acceptance provide a justification (e.g. CoI)

• Finalise and submit one individual evaluation report by Thursday 26.4.

• So that quality controllers can check the completeness of your report and give you feedback on whether issues were commented under appropriate subcriteria

• You will receive comments in your IER reports in SEP

• In some cases your report may be re-opened by REA staff so that you may complete it

• Complete the Remote Evaluation by Friday May 11th

• Necessary to allow the rapporteur to prepare a draft consensus report (CR) prior to the consensus meeting

• The draft CR will be available in the system a few days before the consensus meeting

• Read the draft CR prior to the meeting to prepare for the consensus discussion

About the central evaluation 28/5/2018- 5/6/2018

• The General Briefing starts at 9.30 on Monday morning. You should be in the COVE building (Place Rogier 16, 1210 Brussels) at 8.30 in order to register and pass the security.

• The consensus meetings will take place between Monday 28/5 and Friday 1/6. You should be prepared to stay in the evaluation premises from 9.00 until 18.00 every day. During this time you will have scheduled consensus meetings, but you also need to prepare for the meetings, and finalise and comment the consensus report draft with your colleagues.

• We will do the cross-reading and panel meeting on Monday 4/6 and Tuesday 5/6. A reduced number of evaluators will participate.

48

Thank you! In case of questions, do not hesitate to contact the

secretariat with tag 08-BIZ:REA SPACE CALLS <[email protected]>

(for technical issues: IT Helpdesk)EC HELPDESK<[email protected]>

Disclaimer:This presentation is intended for information purposes only within the framework of the evaluation of the call H2020-SPACE-2018.Please refrain from sharing it with other parties not involved in this process or from making it publicly available.

Space business, entrepreneurship, outreach and education · 5/6/2018 · † Attract the interest...

Documents

Transcript of Space business, entrepreneurship, outreach and education · 5/6/2018 · † Attract the interest...