ESSnet DataWareHousing Stocktaking Pieter Vlag, Viviana di Giorgi, Sonia Queresma.
European Modernisation Workshop on... · European Modernisation Workshop ESSNET on SDMX II - Use of...
Transcript of European Modernisation Workshop on... · European Modernisation Workshop ESSNET on SDMX II - Use of...
European Modernisation Workshop ESSNET on SDMX II - Use of the Deliverables Laura Vignola
23-24/01/2014 - Valencia
Valencia 23-24, January, 2014
• Follow up of the ESSNet on SDMX (phase I).
• The countries involved were: Italy (coordinator), Bulgaria,
Slovak Republic and Sweden.
• Duration was of 18 Months.
• 4 Work packages were proposed with different objectives.
• The planning project period was, from the end of September
2010 (Invitation letter) to the begin of November 2010
(MBGA submitted to Eurostat).
General information
Statistical Data and Metadata eXchange
Valencia 23-24, January, 2014
• Develop software to facilitate the dissemination of statistical data in SDMX format.
• Provide capacity building action on SDMX through manuals, handbook and workshops.
Main Aims of the ESSNet on SDMX II
All the WPs were based, partially or as whole, on the SDMX Reference Infrastructure (SDMX-RI) intermediate version: a set of tools released by Eurostat to support NSIs in the dissemination of data in SDMX format.
Software Capacity building
Valencia 23-24, January, 2014
• From co-partners to coordinator: based on monthly reports.
• From coordinator to Eurostat: based on quarterly reports + intermediary deliverables.
• Physical meetings:
• kick-off meeting,
• three coordinator meetings.
Communication aspects
Valencia 23-24, January, 2014
Templates for the administrative tasks: were based on the templates used in the previous ESSNet. Templates for the documentation to be delivered
• Analysis documentation
• Documentation of the test phase
• Installation guide • Step by Step tutorial
Produced on some examples provided by Eurostat
Homogeneity of documentation
Valencia 23-24, January, 2014
WP2 - SDMX – PC-Axis integration (SCB)
Valencia 23-24, January, 2014
WP2: SDMX – PC-Axis integration (SCB)
Objectives:
Finalize the work started in the ESSNet on SDMX (phase I): disseminate data in SDMX format starting from PC-Axis environment.
Deliverables:
An new architecture in which the PC-Axis tools have been integrated with the SDMX-RI for data reporting in SDMX.
Practical Workshop on how to use the released architecture.
Valencia 23-24, January, 2014
A family of software for the management and dissemination of statistical data
The architecture has been released and it is actually usable by the PC-Axis community.
Further improvement are foreseen by Eurostat on the
SDMX-RI to include a dedicated PC-Axis driver (Autumn 2014).
The workshop was held back to back with the Istat
workshop.
Use of deliverables – WP2
Valencia 23-24, January, 2014
WP3 - SDMX for microdata handling (INFOSTAT)
Valencia 23-24, January, 2014
WP3: SDMX for microdata handling (INFOSTAT)
Objectives:
Finalize the work started in the ESSNet on SDMX (phase I): analyze the feasibility to describe microdata in SDMX and develop a software for data collection in SDMX.
Deliverables:
eCollect-X application, a software that allows the creation of electronic questionnaires and allows the description of microdata using the SDMX artefacts. Document on a comparison between DDI vs SDMX standards.
Valencia 23-24, January, 2014
Data Documentation Initiative: an initiative to define a standard for describing data from the social, behavioral, and economic sciences
The software eCollect-X released at the end of project, was tested from a functional point of view but was not completely tested to see if it covers all the NSI’s expectations. Eurostat, during the SISAI meeting of 2013, proposed NSIs to test it.
The potential use of eCollectX must be seen in the light of
the on-going international developments in particular related to the use of DDI and SDMX.
The comparison between DDI and SDMX standards
has been used in context in which the two standards are compared (i.e. BLUE-External Trade Statistic).
Use of deliverables – WP3
Valencia 23-24, January, 2014
WP4 - Harmonization of existing tools (ISTAT, BNSI)
Valencia 23-24, January, 2014
WP4-Harmonization of existing tools (ISTAT, BNSI)
Objectives:
Complete the SDMX-RI with a software for the data loading.
Provide instruments for IT people on how to reuse the source code of the SDMX-RI for their own needs.
Deliverables:
Loader application, a software to create and populate a “SDMX compliant” database for data dissemination. It is integrated with the SDMX-RI.
Analysis of source code on some modules of SDMX-RI representing the core of the infrastructure.
Two “Step by Step” tutorials based on real examples on how to reuse the source code of SDMX-RI.
Valencia 23-24, January, 2014
The Loader will be used in an internal Istat project (SISTAN-HUB) to support Italian regions not having a dissemination database.
Istat is in contact with Eurostat on the possibility to use the
Loader for the ESA VIP project. Istat continued to work on the Loader application (Loader ver. 2.0
released October 2013). http://bms.istat.it/sodidownload/download.aspx
The analysis of the source code has been used by Eurostat to
complete its analysis documentation. The Step by Step tutorials were used internally in Istat SDMX
technical training courses.
Use of deliverables – WP4
Valencia 23-24, January, 2014
WP5 - Capacity Building (ISTAT)
Valencia 23-24, January, 2014
WP5: Capacity Building (Istat)
Objectives:
Provide capacity building actions on SDMX both for statisticians then for IT staff
Deliverables:
Handbook on how to model statistical data in SDMX
Workshop on SDMX based on the model of UNESWA (United Nation’s Economic and Social Commission for Western Asia) for IT staff and statisticians on how to overcome, jointly, some issues related on SDMX
Valencia 23-24, January, 2014
Capacity building deliverables
The handbook has been promoted by Istat and Eurostat during the training courses and in SDMX implementation when relevant.
It was requested by some international working groups (i.e.
Statistical Network - Innovation in Dissemination). Eurostat is finalizing the updating (on January 2014 should be available through our SDMX Info Space site). The workshop (two sessions) represented an important
practical experience of knowledge sharing and discussion on SDMX between IT people and statisticians.
Valencia 23-24, January, 2014
The software delivered is not already in production but part of it is planned to be used in the future. The deliverables related on the “Capacity Building” domain have been reused in several contexts
Why??
There is an objective difficulty to use software produced by third parts:
• when used as it is, it must cover the NSI expectations,
• when used as open source, it must be developed under strict programming rules.
General consideration
Valencia 23-24, January, 2014
Strengths
• Frequent contacts among co-partners, coordinator and
Eurostat.
• Release of intermediary deliverables.
• Definition of templates for the documentation.
• Strict cooperation with Eurostat.
Valencia 23-24, January, 2014
• Too short planning period.
• Share common objectives among co-partners (more
complex but with greater chance of success).
• The SDMX standard version was evolving from 2.0 to 2.1
and the infrastructure used (SDMX-RI) was changing to the
ultimate version.
• No strict rules were defined for the software
development (who can require their use?
The coordinator or Eurostat?).
Weakness
Valencia 23-24, January, 2014
What should be planned for the future
• Re-adaptation to the new version of the standard
(SDMX 2.1).
• Re-adaptation to the SDMX-RI ultimate version.
• Test of the software in different environments and
using different Use-Cases
on a voluntary basis? as part of a user group, task force ...?
How? Who?
Valencia 23-24, January, 2014
Thank You
Valencia 23-24, January, 2014