Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives...
Transcript of Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives...
![Page 1: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/1.jpg)
INTEROPERABILITY +
SEMANTICS = CHECK!
Smart and Cost Effective Data
Modelling and Tools of the Future
Miika Alonen, CSC – IT Center for Science
Suvi Remes, CSC – IT Center for Science
Patrik Maltusch, Aalto University
Mikael af Hällström, Finnish Tax Administration
1
![Page 2: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/2.jpg)
Summary
• Interoperability challenges
• Interoperability case: Student Transcript
• Framework for Semantic Interoperability
• Implementing the Framework:
– Interoperability workbench
2
![Page 3: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/3.jpg)
Organisation and application specific documentation
Point to point Integration
Redefining data models
No change management over organisation borders
Present – Information chaos
VIRTA
Opintopolku
Oodi X
Vipunen
Oili VTJ
Y
EMREX
![Page 4: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/4.jpg)
Interoperability CASE: Student Transcript
4 Your logo here
![Page 5: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/5.jpg)
1000 h 60.000 €
80 h 4800 €
20
Making data sources semantic interoperable for a DW
~80.000 €
~1 milj. €
Cost for the community
![Page 6: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/6.jpg)
Equation for Interoperability
ICC∗ESB∗EIM∗DW = 𝑏𝑒𝑛𝑒𝑓𝑖𝑡𝑠
𝑐𝑜𝑠𝑡 >1
ICC = Integration Competence Center ESB = Enterprise Service Bus EIM = Enterprise Information Management DW = DataWareHouse
by Patrik Maltusch @aalto.fi
SEMANTIC IMPACT
![Page 7: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/7.jpg)
Data warehouses
RDB SQL Reference data
Enumerations
XML schema
JSON
Codeservers
Application interfaces XML WSDL SOAP
Programming
Java
Javascript Swagger
Python
Terminological dictionaries
Glossary Wikis
Concept models
Classifications
SKOS
Data modelling
UML
XML
Wiki
Excel
RDF
ER
Documents
Used methods for documenting APIs?
![Page 8: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/8.jpg)
Application profiles
Uniform documentation for data
APIs and Applications
Domain vocabularies
Reusable core
data models Reusable classifications
Vision – Data model reuse and uniform documentation
VIRTA
Oili Opintopolku
X Y
UNI Ö
EMREX
Application profiles
Shared semantics
Use of shared semantics
![Page 9: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/9.jpg)
2000 - Idea of Application Profiles evolved from DESIRE Registry project: “mixing and matching” metadata elements
2003 - Guidelines for creating application profiles (CEN/CWA 14855)
2005 – Guidelines for machine-readable representation .. (CEN/CWA 15248)
2006 - Guidelines building application profiles in e-learning (CEN/CWA 15555)
2008 - Guidelines for Dublin Core Application Profiles
2011 - 2016 – MLR Framework (ISO/IEC 19788-1)
An Application profile seeks to address the interoperability requirements between systems by:
• retaining conformance with a base standard
• defining new requirements in an open and interoperable manner
Application profiles
Examples of standard Application profiles: CEN/CWA 15903: Metadata for Learning Opportunities CEN/CWA 16132: European Learner Mobility Achievement Information EMREX AP (2016): Report for the recognition of external studies
![Page 10: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/10.jpg)
Need for common framework
Shared concepts with the business and IT: 1. Well defined concepts 2. Unique identifiers 3. Machine readable format
Service innovation and data modeling based on business needs:
1. Reuse terms and definitions 2. Create reusable components 3. Focus on the interfaces and
integration
Framework for semantic interoperability: • How to publish core vocabularies • and application profiles? • How to reuse standards? • How to reuse core vocabularies in
the implementations? • How to document the metadata
reuse? • How to document application
interfaces?
![Page 11: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/11.jpg)
Terminology
• Concepts
• Definitions
• Classifications
Core vocabularies
• Classes
• Attributes
• Associations
Application Profiles
• Context
• Constraints
• Extensions
Common metadata architecture for data modelling:
• Machine readable terminology • Reusable core vocabularies and core components • Documented reuse of core components
Semantic interoperability framework
![Page 12: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/12.jpg)
Interoperability objectives
• Shared metadata
models for education
and research
• Modular and reusable
metadata definitions
• Improve readability
and understandability
of data models
• Interoperability with
international
standards
• Promote standard
reuse
• Formal and semantic
mappings to CEN and
ISO standards
12
![Page 13: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/13.jpg)
Interoperability benefits
• Collaborative tool for
metadata publishing
– Benefit from external
expertice
• Avoid redefinition of
data models
– Lower integration
costs
• Shared terminology
– Less confusion
• Same terms for many
communications
needs
– Study administration
– User interfaces
13
![Page 14: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/14.jpg)
14
![Page 15: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/15.jpg)
15
Domain specific vocabularies are administered by different agencies and published in the standard SKOS model in the Finnish Thesaurus and Ontology Service
![Page 16: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/16.jpg)
16
Core vocabularies are published as Linked Data that defines re-usable classes and properties based on shared concepts and links to standards and best practices.
![Page 17: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/17.jpg)
17
Domain specific data models and interfaces are documented as Application profiles that re-use the classes and properties from the Core Vocabularies
![Page 18: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/18.jpg)
18
Data models are implemented with languages that best suit the given architecture by implementing the application profile.
![Page 19: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/19.jpg)
Simplified process for describing Core
vocabularies and Application profiles
19
![Page 20: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/20.jpg)
Piloting Interoperability Framework
Core vocabularies Application profiles
iow.csc.fi
VIRTA
Oili
Opintopolku
University Information
Systems
Enables Universities and Consortium Partners to start documenting local implementations using free & open source tools Documenting
Open APIs
Documenting Open APIs
EduCloud
EMREX
![Page 21: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/21.jpg)
Interoperability workbench
21
• Collaborative online tool for creating Core Vocabularies and Application Profiles:
![Page 22: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/22.jpg)
22
Interoperability workbnech • Tool for defining resolvable and machine readable data models • Document the use of data models, standards and best practices
![Page 23: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/23.jpg)
Integration to controlled vocabularies
23
• Link controlled vocabularies to created model • Create classes and properties based on existing concepts
![Page 24: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/24.jpg)
Integration to classification schemes
24
• Link to existing reference data from code service • Restrict allowed values by using existing reference data
![Page 25: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/25.jpg)
25
Interoperability workbench • Import existing models from local models and external namespaces
![Page 26: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/26.jpg)
Interoperability workbench
26
• Include and search metadata from imported standards • Create new domain models as highly reusable metadata
![Page 27: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/27.jpg)
Interoperability workbench
27
• Map new classes and class usage to relevant standards
![Page 28: Smart and Cost Effective Data Modelling and Tools of the ... · Interoperability objectives •Shared metadata models for education and research •Modular and reusable metadata definitions](https://reader033.fdocuments.net/reader033/viewer/2022042412/5f2b5f0ed025e04e00729cc5/html5/thumbnails/28.jpg)
Interoperability workbench
28
RDF
XML Schema tbd …
JSON Schema
• Export schemas in multiple formats • Enforces Naming practices