DAWG-RegRep-1OJN/KC 8/5/2008
MIT Lincoln Laboratory
NextGen Network-Enabled Weather (NNEW)
Registry/Repository
Oliver Newell, Kajal Claypool
5 August 2008
MIT Lincoln LaboratoryDAWG-RegRep2OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep3OJN/KC 8/5/2008
Key Roles of a NNEW Registy/Repositry
• Build-time– Storage/discovery of service interface descriptions (WSDL and
associated schemas)– Storage of dictionary/vocabulary information (e.g. coordinate reference
system dictionary, upper-level weather ontology)
• Run-time– Discovery of datasets and their
associated data access services using high-level metadata
– Dataset metadata within registry includes weather cube domain ‘membership’ (e.g. SAS)
– Dataset can be a member of more than one domain
MIT Lincoln LaboratoryDAWG-RegRep4OJN/KC 8/5/2008
NNEW Registry/Repository
• Universal Description, Discovery, and Integration (UDDI) Registry
– Underpowered with respect to NNEW use cases Limited query capabilities (especially geospatial queries)
– Industry support on the wane… IBM, Microsoft (co-creators of UDDI spec) closed public registries
in 2006 IBM now offers more capable (but proprietary) registry/repository
solution OASIS UDDI Technical Committee now inactive
• ebXML Registry/Repository– More flexible, extensible solution– Freely-available version (freebXML) used in variety of domains
DOD Metadata Repository uses freebXML ‘under the hood’
• NNEW Approach– Adopt ebXML registry/repository – extend as needed– Partner with ebXML registry/repository vendor (Wellfleet)– Formalize extensions with OASIS ebXML technical committee
MIT Lincoln LaboratoryDAWG-RegRep5OJN/KC 8/5/2008
ebXMLRegRep
Standard
Metadata
Fede
rate
d
Info
rmat
ion
Man
age-
men
t
SO
A
Gov
erna
nce
Events
Secure
Architecture
Cataloging, Validation, Version Control,Lifecycle Support,Extensible Info Model
Taxonomies, Classifications, Relationships
Federated Queries, Inter-registry links
Digital Signatures,Audit Trail,Access Control,SAML SSO
SOA Registry
Repository
Registration, Discovery, Queries
Content-Based Event Notification
Key features of ebXML RegRep
MIT Lincoln LaboratoryDAWG-RegRep6OJN/KC 8/5/2008
DOD Metadata Repository
Source: <https://metadata.dod.mil/mdr/about.htm>
Web GUI
• Based upon freebXML Registry OS project
• Customized web based GUI client
• Manages schemas, data elements, attributes, document type definitions, style-sheets, data structures
• Not currently used for registration of services for run-time discovery purposes– ebXML reg/rep does support the run-time use case
MIT Lincoln LaboratoryDAWG-RegRep7OJN/KC 8/5/2008
NNEW Registry-Repository/Catalog
• For 2007/2008 demos, using early release of ebXML RegRep 4.0 – compliant regrep (OASIS draft standard expected ~4th Qtr 2009)
MIT Lincoln LaboratoryDAWG-RegRep8OJN/KC 8/5/2008
Data Discovery and Access using OGC Services and ebXML Registry (Dec ‘07)
OGC service-enableddisplay Client
FAA Technical CenterNational Center for Atmospheric Research (NCAR)
Web CoverageService
(OGC WCS)
IcingTurbulence
Data
NOAA
SurfaceObservations
DataWeb Coverage
Service(OGC WCS)
MIT Lincoln Laboratory
CIWSForecast
Data
Catalog(ebxml)
Web CoverageService
(OGC WCS)
Virtual 4-D Weather Cube
“Discover Available Data Sets”
“Get data from time T1 to T2in spatial region A and return in NetCDF formatusing a Lambert ConformalProjection”
MIT/LL
NCAR
NOAA
RequestedData
MIT Lincoln LaboratoryDAWG-RegRep9OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep10OJN/KC 8/5/2008
Creation of 4-D Cube Single Authoritative Source Domain
“The designated administrator for the SAS domain accesses a 4-D Wx Data Cube registry and determines the identifiers of all datasets categorized as the SAS for their respective data type. A list of all the datasets is passed to the Enterprise service manager responsible for the top-level information registry. The service manager verifies that all the selected datasets have the appropriate access rights and services for the SAS domain, and registers the domain in the registry for subsequent use by all consumers of SAS information.”
Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3
MIT Lincoln LaboratoryDAWG-RegRep11OJN/KC 8/5/2008
Creation of 4-D Cube Experimental SAS Domain
“NCAR, NASA, NOAA, and MIT Lincoln Laboratory are collaborating on a research project to produce a new weather data product. The 4-D Wx Data Cube data access mechanisms are used by the project due to its distributed nature. The majority of inputs that are used to create the new product are those provided by the SAS, with the exception of a higher-quality surface winds product. A new ‘Experimental SAS’ is generated for the duration of the research project, and made available in a registry. Following the completion of the project, the ‘Experimental SAS’ designation is removed from the product’s metadata registry .”
Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3
MIT Lincoln LaboratoryDAWG-RegRep12OJN/KC 8/5/2008
Build-Time Dataset/Service Discovery
“A small flight services company serving the state of Alaska wishes to incorporate a summary of winds information into an existing ‘quick-look’ Web page that they provide to general aviation users. The company browses the NextGen registry to discover what types of wind data are available, and what services exist to access them. An appropriate data access service is located, and members of the company’s software team download the service and data format schema needed to construct a data access client. A thin layer of additional logic is used to convert the raw data to the desired ‘synopsis’ form, and the updated Web application is made available to pilots over the open Internet for flight planning purposes. The data access client software, which conforms to a service standard used to disseminate many types of weather data, provides the company with easy access to numerous other types of weather data in the future.”
Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3
MIT Lincoln LaboratoryDAWG-RegRep13OJN/KC 8/5/2008
Run-Time Dataset/Service Discovery
“A client application contacts a registry containing dataset information for the weather cube, and requests information about ‘air temperature’ datasets available in the SAS domain. Metadata for the single, primary, ‘air temperature’ data set are returned, along with metadata for zero or more backup datasets, each with a priority indication describing the ordering in which the backups should be used. For each dataset, metadata describing one or more data access services, along with the data formats supported by each service, are returned. The client application selects the most appropriate service for the primary dataset, and begins accessing data. ”
Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3
MIT Lincoln LaboratoryDAWG-RegRep14OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep15OJN/KC 8/5/2008
4D Weather Data Cube Domains
1. Single Authoritative Source (SAS)
2. Regulatory2a – Government2b – Commercial
3. Both 1 & 2a4. Intermediate 4a – Government 4b – Commercial
4D Weather Cube Domains
• SAS data (supports ATM decisions) to be available on open & unrestricted conditions• Opportunities for commercial entities to provide weather information still exist• Flexible support for different domains needs to be incorporated into architecture
• SAS data (supports ATM decisions) to be available on open & unrestricted conditions• Opportunities for commercial entities to provide weather information still exist• Flexible support for different domains needs to be incorporated into architecture
MIT Lincoln LaboratoryDAWG-RegRep16OJN/KC 8/5/2008
Importing the Cube Domain Classification Scheme
MIT Lincoln LaboratoryDAWG-RegRep17OJN/KC 8/5/2008
Weather Cube Domain TaxonomyebRIM Encoding (some details ommitted)<RegistryObjectList xmlns="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0" xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0">
<rim:RegistryObject xsi:type="rim:ClassificationSchemeType" id="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" isInternal="true" nodeType="urn:oasis:names:tc:ebxml-regrep:NodeType:UniqueCode"> <rim:Name> <rim:LocalizedString xml:lang="en" value="FAA/NOAA/DOD Data Cube Domain Taxonomy" /> </rim:Name>
<rim:ClassificationNode code="DataCube" parent="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube"> <rim:Name> <rim:LocalizedString xml:lang="en" value="DataCube" /> </rim:Name>
<rim:ClassificationNode code="Restricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Restricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Restricted" /> </rim:Name> <rim:ClassificationNode code="Government"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Commercial"> ... </rim:ClassificationNode> </rim:ClassificationNode>
<rim:ClassificationNode code="Unrestricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Unrestricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Unrestricted" /> </rim:Name> <rim:ClassificationNode code="SAS"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Regulatory"> ... </rim:ClassificationNode> </rim:ClassificationNode> </rim:ClassificationNode> </rim:RegistryObject></RegistryObjectList
MIT Lincoln LaboratoryDAWG-RegRep18OJN/KC 8/5/2008
Viewing the Cube Domain Taxonomy via the RegRep UI
MIT Lincoln LaboratoryDAWG-RegRep19OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep20OJN/KC 8/5/2008
Datasets and Metadata
Metadata
Metadata
DatasetMetadata
DatasetSeries
Metadata
Dataset Series
e.g. Precip Data for most recent one year period
Registry/Repository
ServiceMetadata
Data Access Service(e.g. JMBL, WCS, WFS)
Operates On
Data AccessService
OperatesOn dataset association(s) can be added at RegistrationTime via registry UI if not included in Service Metadata
MIT Lincoln LaboratoryDAWG-RegRep21OJN/KC 8/5/2008
ISO 19115/19139 Metadata
MIT Lincoln LaboratoryDAWG-RegRep22OJN/KC 8/5/2008
ISO 19119 Service Metadata Class Diagram (from specification)
• ISO 19119 (Conceptual Model) does not yet have a standardized schema (ala the ISO 19115/19139 pair)• ISO 19119 (Conceptual Model) does not yet have a standardized schema (ala the ISO 19115/19139 pair)
MIT Lincoln LaboratoryDAWG-RegRep23OJN/KC 8/5/2008
OGC Cat-ebRIM Dataset/Service Registry Information Model Mapping
• Using ebRIM mapping for Service Metadata temporarily until standard 19119 schema is available• Using ebRIM mapping for Service Metadata temporarily until standard 19119 schema is available
MIT Lincoln LaboratoryDAWG-RegRep24OJN/KC 8/5/2008
Registry/Repository Basic Profile Supports ISO 19119 Service Taxonomy
• JMBL, WFS, WCS services can all be classified under this scheme• Flexibility to declare datasets available via JMBL and/or OGC WFS/WCS
• JMBL, WFS, WCS services can all be classified under this scheme• Flexibility to declare datasets available via JMBL and/or OGC WFS/WCS
MIT Lincoln LaboratoryDAWG-RegRep25OJN/KC 8/5/2008
INSPIRE Geoportal Dataset/Service Discovery (ISO 19115/ 19139)
MIT Lincoln LaboratoryDAWG-RegRep26OJN/KC 8/5/2008
INSPIRE Geoportal Metadata EditorISO 19115/19139
• INSPIRE portal does not (apparently) support a service interface• ebXML regrep UI being designed to support the functionality of the
INSPIRE portal, plus the ebXML registry service interface (ebRS)
• INSPIRE portal does not (apparently) support a service interface• ebXML regrep UI being designed to support the functionality of the
INSPIRE portal, plus the ebXML registry service interface (ebRS)
MIT Lincoln LaboratoryDAWG-RegRep27OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep28OJN/KC 8/5/2008
Dataset Discovery: All Datasets in Cube
MIT Lincoln LaboratoryDAWG-RegRep29OJN/KC 8/5/2008
Dataset Discovery: All Restricted Datasets in Cube
• Lightning is only dataset series classified under DataCube/Restricted – all other datasets not returned by query
• Lightning is only dataset series classified under DataCube/Restricted – all other datasets not returned by query
MIT Lincoln LaboratoryDAWG-RegRep30OJN/KC 8/5/2008
Dataset Discovery: All Unrestricted Datasets in Cube
• All datasets with the exception of lightning returned by query• Follow-up queries allow determination of services that operate on each
returned data set (may be zero or more)
• All datasets with the exception of lightning returned by query• Follow-up queries allow determination of services that operate on each
returned data set (may be zero or more)
MIT Lincoln LaboratoryDAWG-RegRep31OJN/KC 8/5/2008
Overview
• Registry/repository background
• NNEW registry/repository use case overview
• Registration of 4-D cube data domain taxonomy
• Registration of a dataset
• Basic dataset and service provider discovery
• Enhanced discovery using Ontologies
MIT Lincoln LaboratoryDAWG-RegRep32OJN/KC 8/5/2008
Intelligent Discovery of Datasets
• Intelligent discovery– Consults knowledge base/ontology to find alternative
meanings Clustered by: synonyms, parent, children
– Enables discovery of resources without exact keyword match
MIT Lincoln LaboratoryDAWG-RegRep33OJN/KC 8/5/2008
Intelligent Discovery of Datasets
SPARQLQueryEngine
SPARQLQueryEngine
OntologyEditor
OntologyEditor
OntologyMapper
OntologyMapper
Query EditorQuery Editor
Service Discovery UI/Client
Service Discovery UI/Client
WCSWCS
air_temperatureair_temperature
ebXML Registry/Repository
Architect
Domain Expert
Service Consumer
Service Provider
Service Provider
Service Provider
WCSDataset: air_temperature
JMBLDataset: temperatureAir
WCSDataset: surface_air_temperature
JMBLJMBL
temperatureAirtemperatureAir
WCSWCS
surface_air_temperaturesurface_air_temperature
• Supports discovery of datasets and associated datasets using CF standard names, JMBL parameter names, or potentially other terms
• Flexibility to support JMBL and/or OGC service access to the same dataset
• Supports discovery of datasets and associated datasets using CF standard names, JMBL parameter names, or potentially other terms
• Flexibility to support JMBL and/or OGC service access to the same dataset
MIT Lincoln LaboratoryDAWG-RegRep34OJN/KC 8/5/2008
Registry/Repository and Infrastructure Support
DomainOntologyDomain
Ontology
MappingOntologyMappingOntology
SPARQLQueryEngine
SPARQLQueryEngine
OntologyEditor
OntologyEditor
OntologyAlignmentOntologyAlignment
Query Editor
Query Editor
OWL-SService Description
Generator
OWL-SService Description
Generator
Service Discovery UI/Client
Service Discovery UI/Client
ServiceAnnotator UI
ServiceAnnotator UI
Service DescriptionService Description
Service AnnotationService Annotation
ebXML Registry/Repository
Architect
Domain Expert
Service Consumer
Service Producer
MIT Lincoln LaboratoryDAWG-RegRep35OJN/KC 8/5/2008
Discussion
• Is an ebXML registry/repository potentially useful in the Joint METOC context?
• Would it be complementary to JMCAT ?
• Would it largely replace JMCAT functionality?
• What is DOD’s view of ISO 19115/19139 metadata vs. DDMS?
Top Related