ESSnet Workshop Cologne 2011 ESSnet on measuring global value chains.
CORE ESSnet Report @MSIS 2011
-
Upload
carlo-vaccari -
Category
Technology
-
view
764 -
download
1
description
Transcript of CORE ESSnet Report @MSIS 2011
MSIS Meeting - Luxembourg May 23-25 2011 1
CORE ESSnet(COmmon Reference Environment)
progress report
Carlo VaccariIstat - Italy
MSIS Meeting - Luxembourg May 23-25 2011 2
Outline
Introduction and history
CORE objectives
CORE where we are
Architecture implementation
Information model
CORE and SDMX
CORE and GSIM
MSIS Meeting - Luxembourg May 23-25 2011 3
CORA ESSnet
Financed by Eurostat under 2009
Statistical Workprogramme
Countries involved: it (coordinator),
ch, dk, lv, nl, no, se
Duration: October 2009 - October
2010
MSIS Meeting - Luxembourg May 23-25 2011 4
CORA Technical
Architecture
CORA Model: two dimensions
Functional dimension
Construction dimension
Functional dimension
Adoption of GSBPM 4.0
9 subprocesses of level 2
1Specify Needs
7Disseminate
2Design
3Build
4Collect
5Process
6Analyse
8Archive
9Evaluate
MSIS Meeting - Luxembourg May 23-25 2011 5
Construction Dimension:
Layers
Figures
Time Series
Statistic
Population
Unit
Variable
Value
A domain of interest documented by statistical products
Statistical series over time
Integrated or simple statistical product for a given time
A statistical unit at a given time
A population at a given time
A statistical variable at a given time
A logical representation of the value of a variable
MSIS Meeting - Luxembourg May 23-25 2011 6
CORA Model Grid
Statistical processes compliant to CORA model are
intended to be designed by statisticians
MSIS Meeting - Luxembourg May 23-25 2011 7
After CORA…CORE!
COmmon Reference Environment (CORE),
financed by Eurostat under 2010
Statistical Workprogramme
Countries involved: it (coordinator), fr, nl,
no, pt, se
Duration: December 2010 - January 2012
MSIS Meeting - Luxembourg May 23-25 2011 8
CORE Principal
Outcomes
Environment for the definition and
execution of statistical processes
Definition of a process in terms of
services selected from an available
repository
Execution of the composed workflow
MSIS Meeting - Luxembourg May 23-25 2011 9
CORE Outcomes:
Design
CORA model → CORA information
model
Design of CORE services and
processes
MSIS Meeting - Luxembourg May 23-25 2011 10
CORE Outcomes:
Implementation
Selection of available middleware solutions for
process execution
Realization of an environment able to permit the
execution of processes:
Interfaces (GUIs) for defining CORE processes for
statistical users
Integration APIs
Repository of integration layers
MSIS Meeting - Luxembourg May 23-25 2011 11
CORE Outcomes:
Testing
Realization of processes starting from
services implementing some GSBPM phase
Evaluation of costs related to integration
Prototype implementation (to be
engineered)
MSIS Meeting - Luxembourg May 23-25 2011 12
CORE Architecture (1)
GUIs to support modelling of CORE processes
according to the CORA grid
Modeling & control flow constructs
Drag & drop facilities for process design
Global schema
Implementation: we are evaluating the usage of
an open process editor tool (Oryx -
http://bpt.hpi.uni-potsdam.de/Oryx/WebHome )
MSIS Meeting - Luxembourg May 23-25 2011 13
Process Runtime
Controlled execution of services
Implementation: integration of existing workflow solutions, currently in evaluation phase
Service Runtime
Integration APIs (in-out data transformation)
Service execution
Implementation: CSV and SQL data transformations are currently being implemented
Service Repository
Deployment of services
CORE Architecture (2)
MSIS Meeting - Luxembourg May 23-25 2011 14
CORE Information Model (1)
First draft of CORE information model
Del. 2.1 released: requirements for the
model of the interface through which
statistical services will communicate
Information Model to be released,
currently in discussion phase
MSIS Meeting - Luxembourg May 23-25 2011 15
Design Principles (in discussion):
Rectangular data sets (rows & columns)
Strong typing (data, rules, parameters)
Dataset kinds (eg micro/aggreg)
Free-style arguments (eg scripts tool
dependent)
Other (service arguments and infos)
CORE Information Model (2)
MSIS Meeting - Luxembourg May 23-25 2011 16
CORE and SDMX
Both initiatives foster standardization
CORE
Focus on standardization of processes and data
exchanges (mainly) intra-NSI
SDMX
Focus on standardization of processes and data
exchanges (mainly) inter-NSIs (or between NSIs
and international organizations)
MSIS Meeting - Luxembourg May 23-25 2011 17
CORE
Focus on all phases of statistical processes
Both micro and macro data considered
SDMX
Focus (mainly) on dissemination phase
Mainly macro data considered
CORE and SDMX - 2
MSIS Meeting - Luxembourg May 23-25 2011 18
Information Model
Both propose an information model
CORE information model
Takes explicitly process dimension into account
through GSBPM
Data dimension
SDMX information model
Mainly focused on data dimension
MSIS Meeting - Luxembourg May 23-25 2011 19
CORA Information
Model
Object
Figure Time series Statistic Population Variable
Constructorprescript
Service
n
1
n +implements
1
Layerlevel+contains
+belongs_to
Construct
Unit
Element
+has
+belongs_to
n
n
+inputn
+output n
1
n
+represented_by
1
noutput.belongs_to.level = input.belongs_to.level + 1
MSIS Meeting - Luxembourg May 23-25 2011 20
SDMX Data & Metadata Information
Model
Category Scheme
Data or Metadata Structure Definition
Category
can have child categories
comprised of subject or reporting categories
Data or Metadata
Flow
Data Provider
Provision Agreement
uses specific data or metadata structure
can be linked with categories from multiple category schemes
conforms to business rules of the data or metadata flow
can get data from multiple data providers
Data or Metadata
Set
MSIS Meeting - Luxembourg May 23-25 2011 21
On Information Models
Different abstraction levels
CORE
“Higher” modelling level
E.g.: statistics as tabular data
SDMX
“Lower” modeling level
E.g.: aggregated data set with dimensions, attributes
and measures
MSIS Meeting - Luxembourg May 23-25 2011 22
Open Issues - 1
Can we use SDMX for micro and macro
data exchanges in a CORE process?
Need for mapping of information
models
MSIS Meeting - Luxembourg May 23-25 2011 23
On-Going Work - 1
CORE implementation scenario within
Istat
Main phases: Sample selection and
allocation
CORE wrapping of available SAS and
R procedures
MSIS Meeting - Luxembourg May 23-25 2011 24
Design and implementation of CORE
Integration APIs
Possible in/out SDMX translations
IAPIIAPI TOOLSDMX
CORE TOOL
SDMX
On-Going Work - 2
MSIS Meeting - Luxembourg May 23-25 2011 25
Open Issues - 2
What about metadata?
CORE: Data and metadata managed in the
same way
SDMX:
Distinction between structural metadata and
reference metadata
Dedicated effort for metadata management
MSIS Meeting - Luxembourg May 23-25 2011 26
Collaboration between
CORE/SDMX ESSnets
CORE planned deliverable on “Feedbacks
on SDMX Usage in CORE”
Periodical meetings inside Istat between
coordinators of the two ESSnets
Exchanges of resources between the two
ESSnets
MSIS Meeting - Luxembourg May 23-25 2011 27
CORE and GSIM
GSIM: Generic Statistical Information Model
deliverable from OCMIMF Operationalising a
Common Metadata/Information Management
Framework activity inside Statistical Network
Ambiguity on the acronym: reference to
“generic statistical information model” in CORE
ESSnet proposal
In March started activity to clarify relationships
(thanks to J.P.Kent and A.Hamilton)
MSIS Meeting - Luxembourg May 23-25 2011 28
First analysis and discussions: the deliverables
from the two initiatives are complementary in
intent and do not overlap in concept
Necessary to avoid gaps and/or duplications
and ensure the complementary relationship
CORE and GSIM
MSIS Meeting - Luxembourg May 23-25 2011 29
CORE Information
ModelCORE will define a very generic information model
(CORE-IM) for the interface through which statistical
services will communicate with each other within
the framework of the CORA model
As a communication protocol, CORE-IM focuses on
the “postal envelope” used when passing
information between services, rather than focusing
in detail on the information being communicated (ie
what is inside)
MSIS Meeting - Luxembourg May 23-25 2011 30
CORE-IM current hypothesis: to support a flag to
indicate if the information being communicated is
described within GSIM
→ without claiming to align the semantics of the
content (eg, “classification”), but only to alert a
consuming service which “understands” GSIM that
it can relate the content to GSIM
CORE and GSIM
MSIS Meeting - Luxembourg May 23-25 2011 31
Complementary nature
CORE-IM supporting semantic interoperability at a very high, abstract level (“here is an information object, along with the ‘envelope’ information about it”) where GSIM can provide greater semantic precision to a subset of information objects communicated using CORE
CORE supporting communication between services
→ substantial interoperability benefits
Information aligned with GSIM semantics
→ further level of interoperability
MSIS Meeting - Luxembourg May 23-25 2011 32
GSBPM reference model for statistical business processes
GSIM reference model for information input to, used by and produced by those processes
Models are independent → it's possible to use one without the other
CORE-IM recognizes and uses GSBPM and (hopefully) will do the same with regard to GSIM, giving them a potential contact point
Complementary nature
MSIS Meeting - Luxembourg May 23-25 2011 33
Coordination in
practiceNeed to maximize the extent to which these synergies are achieved in practice: Members in common (no, se) ABS leader for the OCMIMF and observer in CORE CORE members external reviewers for GSIM material CORE WP2 “co-ordination input” from the OCMIMF collaboration team in regard to deliverables Half day session at METIS workshop (October) presenting CORE and OCMIMF works to external metadata specialists Common documents in preparation ...