ESFRI WORKSHOP MONITORING OF RIs, PERIODIC UPDATE OF ... · ESFRI WORKSHOP – MONITORING OF RIs,...

ESFRI WORKSHOP MONITORING OF RIs, PERIODIC UPDATE OF

LANDMARKS, USE OF KPIs

19-20 NOVEMBER 2018 Palazzo delle Stelline - Milano

Giorgio Rossi

We are convened at the former Orfanotrofio delle Stelline “Orphanage of the Starlets”

The “Spedale dei Poveri Mendicanti e Vergognosi” (Hospice for the Poor Beggars and Shameful) was created in 1578 on input by Carlo Borromeo (one of the leaders of the Catholic Counter-Reform of XVI century, with Ignazio da Loyola and Filippo Neri) within the premises of the monastery “Santa Maria della Stella”. The Hospice was open to all who where forced to beg for lack of subsistence. In 1534 the order of Padri Somaschi had built a dedicated orphanage for women. In 1583 Borromeo gave a strict rule to both the Hospice and the Orphanage.

In 1752 Maria Teresa d’Austria established the Spedale as Royal Female Orphanage, with new rules issued directly from Vienna.

The education of the girls was basic end less intense than for the corresponding male institution “I Martinitt”. Aim of the “Stelline” was to train the girls to be good mothers and house-wives. The more gifted ones were trained as house managers in noble family homes and were educated in art, music, dance, French or German language, history and galateo. In 1807 the Congregazione di Carità Napoleonica coordinated the administration of the Stelline and Martinitt with the Pio Albergo Trivulzio (old people hospital) and later, after Unità d’Italia, they became a unique administration. At the beginning of XX century the Stelline started to join public school and training outside the orphanage. As of 1970 the system has transformed in family-style housing communities the Palazzo delle Stelline was sold to the Comune di Milano that transformed it into a conference center.

Just Across the Street you see the church Santa Maria delle Grazie is a BASILICA and SANTUARIO belonging to the ORDINE DOMENICANO, built between 1492 and 1493 by the architect GUIINIFORTE SOLARI (construction master of DUOMO, OSPEDALE MAGGIONE –now University – and CERTOSA DI PAVIA) and BRUNELLESCHI upon patronage of the Duke of Milano LUDOVICO IL MORO to become a mausoleum for his family.

1943 bombing

Leonardo da Vinci, 1494-1498 fat tempera on plaster 460×880 cm In the REFETTORIO of the CONVENTO owned by COMUNE DI MILANO

THIS WORKSHOP ROADMAP 2016-2018 + new mandate of COUNCIL May 2018

• THE FIRST THREE ESFRI ROADMAPS (yellow period 2006-2010) led to a portfolio of 40 Projects

• In December 2012 the Competitiveness Council: “...EMPHASISES the need for renewing and adapting the mandate of ESFRI to adequately address the existing challenges and also to ensure the follow up of implementation of already ongoing ESFRI projects afte a comprehensive assessment, as well as the prioritisation of the infrastructure projects listed in the ESFRI Roadmap

• In 2012 ESFRI created an ad-hoc WG on “INDICATORS OF PAN-EUROPEAN RELEVANCE OF RIs” that reported to Forum in 2013

• In 2013 ESFRI and the EC run an ASSESSMENT of the ESFRI Projects in view of the INNOVTION EUROPE objective of IMPLEMENTING 60% of the Roadmap by 2016. It was carried out by the ad-hoc HIGH LEVEL EXPERT ASSESSMENT GROUP and led to conclusions on RIs being at different stages of implementation.

• IN 2014 ESFRI formulated the PRIORITIZATION OF SUPPORT TO ESFRI PROJECTS FOR IMPLEMENTATION that led to the recommendation of SPECIAL FUNDING to 12 RIs that needed extra support to meet the 2016 implementation deadline

• IN 2016 the NEW ESFRI ROADMAP (inaugurating the blu period) introduced the “eligibility maturity threshold” and the concept of 10-year cycle for PROJECTS and defined the LANDMARKS (29 assessed)

• The ROADMAP 2016 accordingly introduced the regular MONITORING of the ESFRI RIs, in praticular in view of 10-year expiration, and addressed the need of regular review of LANDMARKS

ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs 19-20 November 2018, Milano

GIORGIO ROSSI

THIS WORKSHOP ROADMAP 2016-2018 + new mandate of COUNCIL May 2018

• In 2017-2018 ESFRI monitored the Projects and started addressing the development of the PERIODIC REVIEW of Landmarks

• In 2017 ESFRI engaged 4 Landmarks into a PILOT EXERCISE in view of formulating the REVIEW

• In 2018 the ESFRI FORUM discussed at length the concept of PERIODIC REVIEW, with different views expressed and a final

synthesis that involved a long non-session in Corfu when the formula of PERIODIC UPDATE of LANDMARKS was agreed

• The ROADMAP 2018 assessed for 8 Projects the new status of LANDMARKS

• In May 2018 the Competitiveness Council: “…INVITES Member States and the Commission within the framework of ESFRI to

develop a common approach for monitoring of their performance and… INVITES the Pan-European Research Infrastructures, on

a voluntary basis, to include it in their governance and explore options to support this through the use of Key Performance

Indicators”.

• ESFRI consequently decided to create and ad-hoc WG on MONITORING and approved its ToR in September 2018

• THIS WORKSHOP is therefore meant to address: LANDMARK PERIODIC UPDATE method and the MANDATE ON MONITORING


GIORGIO ROSSI

ABOUT ESFRI LANDMARKS ROADMAP 2016

1. FIRST INTRODUCED IN ROADMAP 2016 AS FOLLOWS

2. THE ROADMAP 2016 COMPRISED 29 LANDMARKS that have reached the

implementation phase are pan-European hubs of scientific excellence, and will require

continuous support to fulfil their mission and ensure their long-term sustainability

ESFRI WORKSHOP – MONITORING OF RIs, PERIODIC UPDATE OF LANDMARKS, USE OF KPIs

19-20 November 2018, Milano

GIORGIO ROSSI


3. LANDMARKS AS COMPLEMENTARY TO ESFRI PROJECTS

4. TEN-YEAR TIME WINDOW for completing their incubation and reaching

implementation


GIORGIO ROSSI



GIORGIO ROSSI

1. THE DEFINITION HAS BEEN REFINED AS FOLLOWS

2. INTRODUCTION OF THE LYFECYLE APPROACH AND FURTHER REFINEMENTS

1. THE EVOLUTION IN TIME OF ESFRI PROJECTS AND

ESFRI LANDMARK

2. THE NEEDS AND TARGETS OF THE IMPLEMENTATION

PHASE ARE UNDERSTOOD AS A SEQUENCE OF PHASES

FROM THE CONCEPT TO OPERATION AND TO

TERMINATION

THE LYFECYCLE APPROACH FROM PROJECTS TO LANDMARKS

THE ROADMAP DYNAMICS MONITORING OF PROJECTS

GIORGIO ROSSI

1. PROJECTS HAVE TEN YEARS OF RESIDENCY ON THE ROADMAP TO REACH IMPLEMENTATION: THOSE THAT DO NOT MEET THIS GOAL ARE REMOVED FROM THE ROADMAP

2. CHECK OF THE OVERALL PROGRESS TOWARDS IMPLEMENTATION ACCORDING TO THE FULFILMENT OF MINIMAL KEY REQUIREMENTS AS DEFINED IN THE ESFRI ROADMAP GUIDE

3. ADVICES TO THE PROJECTS AND RECOMMENDATIONS TO THE FORUM, INCLUDING THE POSSIBLE PROMOTION OF PROJECTS TO THE STATUS OF LANDMARK


GIORGIO ROSSI

THE PILOT PERIODIC REVIEW OF ESFRI LANDMARKS has been carried out to test the methodology for assessing the state of play

of the ESFRI Landmarks. Four Landmarks volunteered to be part of the pilot exercise, upon invitation. They were:

• ICOS ERIC (ENV) AS A RI WITH GLOBAL OUTREACH

• ELIXIR (H&F) AS A DISTRIBUTED RI EXAMPLE DEALING WITH HIGH DATA-INTENSIVENESS AND E-NEEDS

• SPIRAL2 (PSE) AS A SINGLE SITE RI WITH SPECIAL IMPLEMENTATION CONDITIONS

• ESS ERIC (SCI) AS A DISTRIBUTED RI WITH ISSUES CONNECTED TO THE MANAGEMENT OF SOCIAL SCIENCES DATA AND E-NEEDS

The lessons learnt from the pilot exercise were presented to the EB and to the Forum.

The discussion at all FORUM meetings in 2018 addressed the very NATURE and SCOPE of the MONITORING in the case of

LANDMARKS

THE ROADMAP DYNAMICS PILOT PERIODIC REVIEW

THE LANDMARK DYNAMICS UPDATING OF LANDMARKS

GIORGIO ROSSI

1. LANDMARKS HAVE LONG TERM RESIDENCY ON THE ROADMAP TO PERFORM OPERATION AT THE TOP LEVEL IN THEIR DOMAIN AND AS REFERENCE RESEARCH INFRASTRUCTURES THAT NEED CONTINUOUS SUPPORT

2. PERIODICALLY THE EVIDENCE OF THE LANDMARK STATE OF PLAY (PERFORMANCE) MUST BE UPDATED IN A NON-INVASIVE, BUT FULLY TRANSPARENT WAY

3. A SUSTAINABLE CYCLE (FOR RIS AND FOR ESFRI) OF UPDATE CYCLE (5-7 YEARS, 5-6 LANDMARKS/YEAR) MUST BE ESTABLISHED, INDEPENDENT OF THE PERIODICITY OF THE ROADMAP UPDATES

MONITORING OF RIs NEW MANDATE


GIORGIO ROSSI

EU COUNCIL CONCLUSIONS OF 29 MAY 2018 ON ACCELERATING KNOWLEDGE CIRCULATION IN THE EU

“Stresses the importance of human resources and training skills as key factors in the success for Research

Infrastructures and ACKNOWLEDGES the need for Research Infrastructures to strengthen a service-driven

approach; INVITES Member States and the Commission within the framework of ESFRI to develop a common

approach for monitoring of their performance and INVITES the Pan-European Research Infrastructures, on a

voluntary basis, to include it in their governance and explore options to support this through the use of Key

Performance Indicators”.

THESE RECOMMENDATIONS INSPIRE THE FORMULATION OF THE

PERIODIC UPDATES OF LANDMARKS, and the work TOWARDS A FEW GENERIC KEY PERFORMANCE

INDICATORS.

PERIODIC UPDATES OF LANDMARKS THE WAY FORWARD


GIORGIO ROSSI

The concept of ESFRI Landmarks implies PERIODIC UPDATING of their state of play throughout ALL THE

LIFECYCLE PHASES.

ESFRI is aware that THE INCLUSIONS OF LANDMARKS in its Roadmap often ARE CONDITIONAL ON (ELIGIBILITY

FOR) REGIONAL, NATIONAL AND EUROPEAN FUNDING and underlines that these inclusions are a result of

the robust and transparent methodology.

ESFRI will be responsive to the possible requests for guidance indicated by the Research

Infrastructures, and will TAKE NOTE OF ALL THE INTERNAL AND EXTERNAL EVALUATIONS performed by their

stakeholders and users communities, in order to be effectively updated on the state of play of the

Landmarks.



GIORGIO ROSSI

The GOAL of Periodic Updates of ESFRI of ESFRI Landmarks is TO UNDERSTAND THE SCIENTIFIC STATUS OF

EACH LANDMARK, considering its specific position in its lifecycle, and enabling ESFRI to fully deploy

its strategic role and support the Landmarks.

Moreover, periodic updates on ESFRI Landmarks will facilitate meaningful interaction between

ESFRI and its Delegations to ADDRESS THE GAPS AND AREAS OF STRATEGIC IMPORTANCE highlighted in the

Landscape analyses.

The periodic update of ESFRI Landmarks differs fundamentally from the evaluation and

assessment of proposals and from the monitoring of Projects along the following lines.



GIORGIO ROSSI

• It will be independent from the ESFRI Roadmap updates and each Landmark is addressed for update periodically, taking into

account the specific context of each RI. The updates will be “reflected” in the Annual Reports and Roadmaps.

• The periodic updates primarily take stock of external and internal evaluations provided by the Landmarks themselves and

carried out by their internal bodies, funders and stakeholders. Importantly, the periodic review is directed towards providing

a basis for dialogue between ESFRI, Landmarks and stakeholders, taking into account also national roadmap processes.

• ESFRI will identify a combination of generic KPIs and specific KPIs – suggested by the RI itself – for updating the state of play

of the Landmarks. It is important that each KPI is set against certain targets or metrics in terms of self-assessment and

quality assurance.

• The updates of the Landmarks are meant to establish a lasting dialogue between ESFRI and the ESFRI Landmarks – and their

research communities – and with the involved countries.

• A non-bureaucratic, evidence based, periodic update of Landmarks will reinforce the strategic role and position of ESFRI in

supporting these RIs when referring to the EU Council, as well as in allowing for the maintenance of the Landmarks’ list.

PERIODIC UPDATES OF LANDMARKS TOWARDS A FEW GENERIC KEY PERFORMANCE INDICATORS


GIORGIO ROSSI

To help address the unique aspects of each RI, Landmarks will develop specific KPIs and report on the few generic ones chosen

by ESFRI, therefore updating their profile and capturing their own peculiarities.

Avoiding one-size-fits-all shortcomings, indicative elements for developing flexible KPIs include:

• scientific excellence outcomes, output and delivery of talent;

• reference role in the disciplinary field (uniqueness in capabilities, in capacity) at European and international level;

• progress in achieving the RI´s milestones along its lifecycle;

• impact, innovation, entrepreneurship;

• establishment and development of the RIs’ user community;

• scientific data management policy, metadata catalogue interfacing

• EOSC and similar open science initiatives, advanced data services for scientific analysis and for innovation developments;

• enforcement of quality control of access (peer review), data (FAIR and reproducible), and services to research and

innovation.

MONITORING MANDATE METHODS AND USE OF KEY PERFORMANCE INDICATORS


GIORGIO ROSSI

• European RIs, not developed within ESFRI, have often a long record of excellent performance and constitute

the majority of operational RIs in Europe. They are usually assessed by the OWNERS, often through

international ad-hoc committees;

• National Roadmaps are often based on national assessment/monitoring of performance. Good practices must

be captured and understood in order to develop a model that integrates and generalize them

• KPIs must help the monitoring, but SHOULD NOT become a SHORTCUT METROLOGY;

• The WG-MONITORING will work on it and report to ESFRI next summer;

• ESFRI will prepare a “PAPER ON MONITORING” by the end of 2019

Let’s have a fruitful ESFRI WORKSHOP on MONITORING OF RIs,

PERIODIC UPDATE OF LANDMARKS, USE OF KPIs

19-20 NOVEMBER 2018 Palazzo delle Stelline - Milano

ESFRI WORKSHOP MONITORING OF RIs, PERIODIC UPDATE OF ... · ESFRI WORKSHOP – MONITORING OF RIs,...

Documents

Transcript of ESFRI WORKSHOP MONITORING OF RIs, PERIODIC UPDATE OF ... · ESFRI WORKSHOP – MONITORING OF RIs,...