Practicalities and Technical Implementation

of a Synchronised Call

Christian Listabarth

INSO 12-2014

• Elements (scope)

• New instrument/pilot within ERA (no precedence)

• Impact: call text & to be defined by consortium

• Implementation: technical & practical solutions

Summary of INSO 12-2014• Objectives / Elements – 1 call, synchronized, based on bundled national calls– peer review evaluationminimum implications: agreed topic, one language, joint evaluation

• Background/input– your national call, peer review evaluation, joint calls

• Foreground/outcome– practice of implementation (feasibility and practicality)– technical modalities (solutions for implementation)– proof of concept (efforts v. added value)

Mechanics

• national call v. synchronized call => bundle nat. calls• joint call v. synchronized call => no new set-up– no transnational dimension of projects– no fresh money needed – BUT: consider experience & good practice of joint calls

• national tasks & joint tasks => coordinate – timeline – evaluation

• practical implications along a standard timeline

Hypothetical timeline (flexible)

joint activities: the coordinator + participating agenciesnational level activities

Call cyclenational & joint calls have same cycle (in principle)synchronized call has very same cyclecall cycle is a generic matter and involves certain essential aspects (ERA-LEARN cycle)

use it as a standarduse it as a draft, but not literally

Joint activtiesmajor joint actions in synchronized call:

planning, evaluation, (submission)agreed call timeline

partly joint actions (dependent on settings):call preparation(submission)

1 planning2 preparation3 submission

4 evaluation

5 decision6 aftercall

http://netwatch.jrc.ec.europa.eu/web/lp/learning-platform/toolbox/call-implementation

Phase 1 – 3 (example)

Phase 4 – 6 (example)

Synchronization of national calls

Call planning Call preparation Submission Evaluation Funding decision After the Call

National timelines: same principle, but individual characteristics (position and length of phases)

Submission deadline: eligibility check and start of evaluation

Deadline and eligibility: could be planned as a (short) phase

Approach: optimized (negotiated)

•date of funding decision fixed: evaluation must be ready at that date (minimum requirement)•national timelines flexible: => optimize for joint evaluation

Planning I (for proposal)

• Consortium building – compatible participating nat. programs?

• Management/coordination– who/how/which tasks– plan meetings

• Call topic (apt for all?)• Timeline for the action period (feasibility)• Number of expected proposals (cost estimate) • Evaluation scheme (basics at least) • Confidentiality of proposals/results (is there a problem?)

Planning II (for proposal)

• Evaluation procedure– centralized v. decentralized (but identical within

consortium)– 1-step v. 2-step – with/without panel– selection of evaluators (EC database)– costs for evaluation (data from practice)

Evaluation: 1 or 2-step

Synchronized Call Office National programs

Pros:•legal constraints, tradition•less in depth evaluation effortCons:•longer timeline•double administrative effort

Evaluation outcome:•proposals assessed •project quality compared•above threshold

Planning III (for proposal)

• Electronic Evaluation and Submission System– national v. central– which systems, what can they do?– availability (within / outside consortium)– interaction with provider, desired complexity

Workflow of a call & role of electronicEvaluation and Submission System (ESS)

Call announcement Submission Proposal Evaluation Result Decision

Start………………………..…….…Call life cycle………………………………………End

Start Project life cycle…

ESS: Interface between actors/activities and deliverables (documents)

ESS functionality: Interface establishedData administrated and managedDatabase / documentation generated

ESS: Information hub for: proposal up- and download (actors: applicants, agencies and evaluators)evaluation results up- and download (actors: agencies and evaluators)

Participating agencies ESS Provider

Outcome: tailor-made ESS and explanation of use

• Using ESS for submission• Provide list of evaluators for access distribution

• Define call specifications• Develop evaluation procedure and timeline• Define access rights accordingly

(general and specific to each phase)• Formulate order for ESS

• Provide consult and experience to client(feasibility of procedure and timeline, constraints, additional functions)

• Design ESS from existing system for specified variables, explain operation to client

• Return access rights• Prepare database• Perform evaluation

• download of projects by respective referees• upload of reviews by referees• administrate data for consensus meeting

Outcome: evaluation report (with scores)

Workflow of interaction with ESS provider

Everything you expect from operation must be specified here

Budget• Evaluation budget– Variable costs (fees for experts, panel)– Fixed costs (secretariat)

• Coordination budget– Preparation and coordination (PM costs)– Call promotion (PM, material)– Meetings (travel)– IT Tool / ESS (subcontracting)

• Dependent on: – Number of proposals expected, fees, in-kind– Consider Call settings: 0.2-0.5 Mio €

Planning IV (for proposal)

Competition for excellence Quality and efficiency of implementation consider/resolve

•state of the art•as many aspects as possible•impact on costs

Implementation I (start of project)

• Install management and task forces• Refine all previous issues, particularly:– topic (agreed range of eligible topics, definitions)– timeline, fix dates– eligibility: national and joint criteria – evaluation criteria (identical)– submission and evaluation process

Implementation II (start of project)

• Call preparation (promotion and documents)– intrinsically dependent on national v. central setting– usually, applicants are attracted mainly by national promotion

• national elements– inform applicants on your purpose for synchronized call– inform on validity of national standards– argue for positioning of national science community within ERA

• central elements– announce call and focus on joint image– explain the European dimension of the call– refer to national program owners for details (national rules apply)

– Documents may differ, but must contain the same elements

Summary

• no precedence – much room for creativity• take professional help (ERA-LEARN)• main joint task (general): manage call operatively• main challenge (specific): agree and accept

evaluation procedure (and results)• main prerequisites: trust and commitment• main factors (technical): sound concept and

compliant coordinator

Thank you for your attention

http://netwatch.jrc.ec.europa.eu/web/lp/learning-platform/toolbox/call-implementation