07-08/6/2011

Methodological and IT innovation mechanisms in the ESS- a review

E. di Meglio – P. Jacques – J.M. Museux

Unit B2 : Methodology & Research

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

The Vision

Integration and standardisation of ESS processes : data warehouses

Incorporation of ICT development (SOA, distributed architecture, Web 3.0)

New methods for producing statistics (reuse, data matching and linking, mixed modes approach, visualisation and analysis of big datasets, …)

New generation of indicators (Stiglitz …)

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Legal framework : from stove pipe legislation to cross cutting framework legislation

Methodological developments /standardisation – Best practices – guidelines – rules– Common tools

Modernisation of IT infrastructure (SOA, web services, distributed and cloud computing, use of standardised IT solutions)

Standardisation and industrialisation– Common description and semantic– Common data exchange standard– Common metadata– Common tools

ESS actions required

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Reference ESS Enterprise architecture

BACK

International standards

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Role of Research Framework programme in Official Statistics– FP5 (35 seminal research projects) – FP7 (new indicators, data

infrastructure, ) and upcoming FP8

Preparing and addressing future challenge– New data sources (i.e. Social networking, Internet of Things, Data

Streams, mobile info/GI, ..)– Data analysis / data mining / visualisation of big datasets (Data Deluge)– Web 3.0, semantics– New modes of data collection and related estimation issues– Understanding non response– Set up e-infrastructures– Critical events forecasting

Collaboration with academics (statistical watch - think tank, adviser for innovation, training of Official statisticians)

Research

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

ESSnets

A collaborative network of several ESS organisations aimed at providing results beneficial to the whole ESS

26 ESSnet projects since 2008

www.essnet-portal.eu

ESSnet review on going – ESSnet coordination - ESSnet planning

ESSnet characterisation according to their contribution to innovation process

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

ESSnets typology

1. ESSnet research consolidation and incorporation of research knowledge and testing of prototype

tools Ex. SDC I - ISAD

2. ESSnet pole of expertise specialisation in process steps – cover wider range of activities (best practices

identification, guidelines, tool development, dissemination of knowledge-know how and training

Ex SDC II, DI, SAE, Mixed mode approach

3. ESSnet infrastructure facilitation to incorporation of international standards, recommendations, cases

studies, taylorisation, improvement of standards, setting up of common infrastructure

Ex. EGR, SDMX I & II, MEETS Datawarehouse, Geostat, CORA & CORE, Decentralised access to EU micro data

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

ESSnets typology

1. ESSnet standardisation contribute to knowledge and process standardisation (concept,

tools, guidelines, ….) Ex SDC harmonisation, MEETS Memobust, Preparing

Standardisation

2. ESSnet strategy : support to strategic action like Sponsorship on Standardisation

3. ESSnet piloting domain dependent, testing in common, practical implementation

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

ESSnet – contribution to a reference architecture

CORE

DI

ESSnet

SDMX

DW

Consistency

MemobustSDC Harm

Stand Prep

DecentralisedAccess

EGR

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Contribution to ESS business process

BACK

Administrative Data

SDC

SAE

DI

Profiling

Net SILC

ICTLinking

Culture

PHGeostat

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Common tool development

Tool developed in common for implementing ESS guidelines

Need for specific governance and development strategy– Open source– Modular – web service – portability (JAVA)– ESS Steering Group – User group

Best practice– Demetra+ :

• Tool for implementing ESS seasonal adjustment guidelines• Demetra+ user group – prototype development• Java toolkit (API, GUI, Data cruncher to be run in batch, library

of core functions, structured objects)• Re-used for Ecotrim, NAPS developments

– Argus+ : sustainability of T-Argus – proof of concept

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Eurostat modernisation - VIPs

Strategic project for Eurostat and ESS modernisation Joint venture between business area units (relevance) and

horizontal units (methodology, IT) 6 projects (still on going) launched in 2010-11

BACK

1. VIP data validation

2. VIP data matching

3. VIP new dissemination

4. VIP National account production

5. VIP Secure infrastructure for confidential data

6. VIP on PILLAR approach for household data collection

ESTAT process /

infrastructure

ESS design

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Eurostat modernisation - VIPs

Common characteristics– Cross domains development– Mainly ESTAT focus - IT or process oriented– Prepare for vertical (MS- Eurostat) integration

Need to maintain global coherence– Planning – yearly work programme– Spin off of ESS(net) project– ESS relevance - coordination

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

VIP, ESSnet – elements of coherence

NAPS

Data Matching

Pillar

CORE

DI

SICON

ESSnet

SDMX

VIP

New Diss

DW

Consistency

Memobust SDC Harm

Stand Prep

Data Validation

DecentralisedAccess

DW

Common tools

Demetra+

Argus+

EGR

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Strategic level – sponsorship

Sponsorship on standardisation

Sponsorship group : general directors or deputies of NSOs, co chaired by Eurostat and a NSO) to build mechanism and steer competence on key challenge for the ESS over a relatively short period (2 years)

Main objectives– to investigate ways to organise and strengthen the sharing of methods

and tools and the creation of common infrastructures– to contribute to the establishment of an ESS Enterprise Architecture for

standardisation (key principles, …)– to establish a ESS work programme towards a more integrated way of

producing European statistics building on existing instrument

BACK

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

VIP, sponsorship and ESSnet – global picture

Data Matching

Pillar

CORE

DI

Quality

Standardisation

Communication

ESSnet

SDMX

Sponsorship

VIP

DW

Consistency

MemobustSDC Harm

Stand Prep

NAPS

New Diss

Data Validation

DecentralisedAccess

DW

Common tools

Demetra+Argus+

SICON

EGR

7-8 June 2011 SISAE – Methodological and IT innovation in the ESS – a review

Conclusions

Many initiative on going – bottom up approach initially

Hard to maintain coherence between different layers, scope

Big hope with the Sponsorship on standardisation

Need for an Enterprise Architecture at ESS level

BACK