DWD GISC Node Current Status
description
Transcript of DWD GISC Node Current Status
Deutscher Wetterdienst
DWD GISC NodeCurrent Status
DWD GISC Node - Current Status
Deutscher Wetterdienst
Agenda
Welcome Current status: Overview based on functional blocks
High Level Architecture Web Interface DAR Component
Catalogue systems, Metadata and identified issues GISC-to-GISC communication
Experience with Blog solution and AFDSchema for GISC backup scenario
Data ComponentsMSS, AFD and subscription management
Live Demo Challenges during the Implementation Q&A
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Functional blocks
Subscription Management
Datapool(s)• controller/manager• access adapter• data sources
DAR component• controller/manager• catalogue• editor
DAR-Generator/Importer
Input Management
Logging, Monitoring and Reporting Authorisation/Authentication services
Web functions• Frontend• Web services (interfaces)• Web services (core)• Business logic
Gray List Container
WMOGISC-GISCInterface
Receiver
Delivery tools• Moving Weather• AFD• controller
ExternalDCPC
DWD GISC Node - Current Status
Deutscher Wetterdienst
HLA: Web Interface (User/Admin/Customer)
Subscription Management
Datapool(s)• controller/manager• access adapter• data sources
DAR-Generator/Importer
Input Management
Logging, Monitoring and Reporting Authorisation/Authentication services
Gray List Container
WMOGISC-GISCInterface
Receiver
Delivery tools• Moving Weather• AFD• controller
ExternalDCPC
DAR component• controller/manager• catalogue• editor
Web functions• Frontend• Web services (interfaces)• Web services (core)• Business logic
DWD GISC Node - Current Status
Deutscher Wetterdienst
HLA: Web Interface (User/Admin/Customer)
WebWerdis
Redesign of WebWerdis
Navigation structure / search functions
Internationalization
Integrate logging framework (necessary for monitoring)
Code clean-up
Integration with Geonetwork
Geonetwork
Customization of GUI
DWD GISC Node - Current Status
Deutscher Wetterdienst
HLA: Web Interface (User/Admin/Customer)
Search by URL – ISO 23950
Timo (WMO Secretariat) did a SRU implementation in Geonetwork
Solution is running in DWD DMZ
3 day implementation workshop with Timo about SRU and jzkit at DWD
Direct integration into DWD GISC node
DWD GISC Node - Current Status
Deutscher Wetterdienst
ExternalDCPC
High level architecture: Metadata Management
Input Management
Logging, Monitoring and Reporting Authorisation/Authentication services
Web functions• Frontend• Web services (interfaces)• Web services (core)• Business logic
WMOGISC-GISCInterfaceDelivery tools
• Moving Weather• AFD• controller
DAR-Generator/Importer
Subscription Management
Datapool(s)• controller/manager• access adapter• data sources
DAR component• controller/manager• catalogue• editor
Gray List Container
Receiver
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Metadata Management
DAR Catalogue Did extensively Performance Tests (Geonetwork 2.2, terraCatalog 2.3)
DWD formulated test casesContracted external partner for tests
Functional Tests DAR Generator / Importer
Developed DAR metadata generator (template driven architecture) External review of DAR metadata Developed DAR importer
Issues we identified during our work Tested DAR catalogues fulfilled most, but not all INSPIRE and WMO requirements DAR Metadata
Paper with Request for Clarification (JMA, CMA and DWD)Schematron rules for validating WMO profile would be very helpful
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Metadata Management
Metadata Harvesting
CMA and DWD did several functional tests with OAI-PMH in 09/09
Different implementations of OAI-PMH could not harvest each other
Deleted metadata records: different implementations
Standard vanilla implementations modify harvested metadata
Ongoing tests between CMA and DWD
Started January 2010
Initial tests showed 1 record ~ 1 second space for improvement
Backup strategy
Exchange of metadata files as products
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Metadata Management
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: G2G communication
Subscription Management
Datapool(s)• controller/manager• access adapter• data sources
DAR component• controller/manager• catalogue• editor
DAR-Generator/Importer
Input Management
Logging, Monitoring and Reporting Authorisation/Authentication services
Web functions• Frontend• Web services (interfaces)• Web services (core)• Business logic
Gray List Container
Receiver
Delivery tools• Moving Weather• AFD• controller
ExternalDCPC
WMOGISC-GISCInterface
DWD GISC Node - Current Status
Deutscher Wetterdienst
HLA: GISC-to-GISC communication Collaboration CMA and DWD
Memorandum of Understanding signed end of last year Two Workshops held
GISC-to-GISC communication tests with CMA CMA and DWD installed Blog solution and AFD solution Data sets with appropriated time profile (issue warning during model transfer)
Blog solution CMA and DWD did several improvements on publishing and polling components Usage of tar minimize latency of transfer Necessary to include size of “metadata” (AtomFeed) into calculation of transfer volume Identified several further improvements like prioritization, AtomFeed handling, logging JMA has provided a new improved solution in 10/2009
AFD solution Average transport time between 4-5 sec during 24hour continuously test, warnings between 3-4 sec In case of unsecured connection sometimes corrupted data occur
DWD GISC Node - Current Status
Deutscher Wetterdienst
HLA: GISC-to-GISC communication backup
Technical specification requires a bilateral agreement from 2 GISC for a backup scenario
CMA and DWD prepared proposal for priorities in DAR metadata
First common draft of dissemination metadata (XML schema)
Need to prove in practice
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Delivery components
DAR component• controller/manager• catalogue• editor
DAR-Generator/Importer
Input Management
Logging, Monitoring and Reporting Authorisation/Authentication services
Web functions• Frontend• Web services (interfaces)• Web services (core)• Business logic
Gray List Container
WMOGISC-GISCInterface
Receiver
ExternalDCPC
Subscription Management
Datapool(s)• controller/manager• access adapter• data sources
Delivery tools• Moving Weather• AFD• controller
DWD GISC Node - Current Status
Deutscher Wetterdienst
High level architecture: Delivery components
Message Switching System
REST Web-Service to manage and control Moving Weather
First proposal for generic Interface
Integration into WebWerdis (still under work)
Subscription Management
Controller: time and event triggered management (still under work)
Customization according to GISC-to-GISC backup necessary
DWD GISC Node - Current Status
Deutscher Wetterdienst
Live Demonstration
DWD GISC Node - Current Status
Deutscher Wetterdienst
TC 5 - Maintenance of Users, Roles, Authorization and Authentication
Search metadata
Access data
Show difference between authorized and unauthorized User
User and Customer management
Subscription
View historic transactions
DWD GISC Node - Current Status
Deutscher Wetterdienst
Metadata Management – TC 1
Show metadata editor
Fully integration open because of separation between Intranet and DMZ
Authorization required to add/modify/delete metadata
DWD GISC Node - Current Status
Deutscher Wetterdienst
Metadata Management – TC 2, TC 6
A WIS Centre provides new metadata records
Show that GISC Geonetwork doesn’t contains new metadata records
Show that GISC_PRODUCTS is empty
Copy new metadata records to remote provider, reindex
GISC harvests new metadata records
Update of GISC Geonetwork and jOAI Provider
Show update of Geonetwork with SRU query
Show update of GISC_PRODUCTS
DWD GISC Node - Current Status
Deutscher Wetterdienst
Challenges during the Implementation
Metadata generation
See paper “Questions on Metadata” from CMA, JMA and DWD from 11.2009
WIS Specification is open – what is good!
DWD GISC Node - Current Status
Deutscher Wetterdienst
Q&A