NATO ARW September 2007 SeaDataNet A Pan-European Infrastructure for Ocean and Marine Data...
-
Upload
lora-wilcox -
Category
Documents
-
view
218 -
download
1
Transcript of NATO ARW September 2007 SeaDataNet A Pan-European Infrastructure for Ocean and Marine Data...
NATO ARW September 2007
SeaDataNetA Pan-European Infrastructure
for Ocean and Marine Data Management
by Dick M.A. Schaap (MARIS)
Technical coordinator
NATO ARW September 2007
Background
• Data availability of vital importance for marine research
• European data set is fragmented
• Not always quality controlled
• Not always easily accessible
• In 35 countries bordering European seas• more than 1.000 scientific laboratories from government
organizations and private industry collect data
• use various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites
• measure physical, geophysical, geological, biological and chemical parameters, biological species, etc.
NATO ARW September 2007
What is SeaDataNet?
• EU Sixth Framework Programme • Integrated Infrastructure Initiative (I3)• 1 April 2006 to 31 March 2011• Objectives
• develop an efficient distributed pan-European marine data management Infrastructure for managing large and diverse data sets
• network existing professional data centres of 35 countries, active in data collection
• provide integrated databases of standardised quality on-line
NATO ARW September 2007
SeaDataNet partners
49 partners including:• Major oceanographic
institutes of 35 countries
• National Oceanographic Data Centres (NODCs)
• Divisions of major national marine research institutes
• 3 satellite data centres
• 3 scientific modelling centres
• 2 SMEs experts in software development
• 3 International organisations (IOC, ICES,
JRC)
NATO ARW September 2007
Building on experience
• Leaders of Data Management Concerted Actions• MEDAR/MEDATLAS II - Mediterranean Data Archaeology
and Rescue, aiming to make available a comprehensive data product of temperature, salinity and bio-chemical data in the Mediterranean and Black Sea through a wide international cooperation
• Sea-Search – The gateway to Oceanographic and Marine Data & Information in Europe, to , to provide an effective navigation tool to oceanographic and marine data and information sources for European marine waters and to centres in Europe with expertise and skills in oceanographic and marine data & information management.
• Data management of scientific projects:• MTPII-MATER, CANIGO, OMEX, MERSEA, etc.
• Bringing in new partners – satellite data centres
NATO ARW September 2007
Infrastructure, versions & planning
• Version 0Continuation, maintenance and further population of existing Sea-Search systems with minor modifications
• Version 1All data centres involved in upgraded meta-databases (ISO compliant XML metadata-profiles and web services)
Technical Task Team partners involved in transparent data access, security, and monitoring of use = trans-national access via downloading of datasets in standard formats (ODV ASCII; NetCDF; MedAtlas)
Launch by March 2008
• Version 2All data centres involved in trans-national access and upgrading of data access to include spatial data infrastructure features (adopting OGC; ISO and INSPIRE standards) and using XML data profiles
Operational by 2010
NATO ARW September 2007
SeaDataNet Data Policy
• Metadata• free and open access, no registration required• each data centre should provide the meta-data in standardized
format to populate the discovery services (metadata directories)
• Data and products services • in general free and open access, but mandatory registration• a “SeaDataNet role” (partner, academic, commercial etc.) is
attributed to each user • Each NODC attributes the roles to the users of its of country• Out of the partnership, the roles are assigned by SeaDataNet user-
desk
• each data centre node delivers data according to the user’s role and its local regulation
• each data centre should provide freely the data sets necessary to develop the common products
NATO ARW September 2007
Discovery services
• Marine environmental data sets (EDMED)
• Marine environmental research projects (EDMERP)
• Observing systems (EDIOS)• Research Cruise Summary
Reports (CSR) • Common Data Index (CDI)• Directory of marine
organisations (EDMO)
NATO ARW September 2007
Cruise Summary Reports
• Conceived by IOC• ICES acted as focal point • Currently over than 37000
cruises• Documenting data
collected on scientific research cruises operated by European vessels
• Cruise track charts included
• Online entry form• Online searchable
database
NATO ARW September 2007
European Directory of the Ocean-observing System (EDIOS)
• Initiative of the European Global Ocean Observing System (EuroGOOS)
• Regular repeated measurements
• On-line searchable directory of observing, measuring, and monitoring systems
• Over 10,000 observation entries
• Links to on-line real-time and archive data
• Tool to aid design of observing systems
NATO ARW September 2007
European Directory of Marine Organisations (EDMO)
• Over 1.000 organisations: addresses, profiles and relations in other directories
• Online maintenance per country by CMS
• On-line user interface with combined alphanumeric – geographical searching
• Export in .csv files• Also available as Web
service (SOAP based)
NATO ARW September 2007
Common Data Index
• Index to individual data sets
• Now >250000 entries from 20 partners
• Mid Oct 2007 entries from 35 countries
• ISO19115 metadata standard has been adopted
• On-line searchable directory
• Direct access to on-line data sets, on-line order facility or e-mail
request
NATO ARW September 2007
Technical developments
• Streamlining of metadata formats for interrelationships between the directories
• Establishment and governance of common vocabularies• ISO 19115 content model as basis for the XML formats and
exchange schemas (XSD) with SeaDataNet XML approach• Innovating maintenance modalities:
• XML exchange using local software component (MIKADO) or own application
• XML validation services• Online Content Management System (CMS)
• Introducing Web services• Vocabularies and Directory of Marine Organisations (EDMO)• Coming: other metadata directories
NATO ARW September 2007
Interrelationships between the directories, using vocabularies
Country
Param
Platf
…
…
…EDMED
EDIOS
EDMERP
CDI
CSR
EDMO
XML
NATO ARW September 2007
Common Vocabularies
• Controlled vocabularies containing keys, terms, abbreviations and essential (but often missing) definitions
• Lists for platforms, countries, sea areas, etc…
• International content governance by SeaVox list server
• Web service • On-line user client• Export in .csv files
NATO ARW September 2007
SeaDataNet standards
• Interoperability – key to distributed data management system success• Common understanding of terms used via common
vocabularies• Standard data exchange formats (ODV ASCII,
NetCDF, MedAtlas)• Common quality control protocols and flag scale• Standard Web services • Schemes for handling duplicate data• Ocean Data View software (ODV) adapted for
SeaDataNet
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Primary function• Users will be able to
download selected data sets
CDI User Interface• Searching and
browsing of metadata of data sets
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Shopping basket• Preparing a user request
of multiple data sets Central User Register• Validation of users and
their rolesBusiness matrix• Combining user role and
data access restriction(s) to decide on modality for processing each data request
A B C D
U
V
W
NATO ARW September 2007
Central User Register
• Single Sign On system required for access to distributed system
• User’s authentication information based on personal login / password
• User must register in order to get login• Web form to provide necessary information • Online user agreement “SeaDataNet General Licence”• Registration processed per country via online CMS • After processing, login/password sent by e-mail (e-mail check)• User gets ‘Role’
• SeaDataNet Data Policy • Data access based upon combination of User role -
Access restriction(s) as indicated in CDI record and business logic
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Request Status Manager• Processing of all requests
and data deliveries • Enables users to follow
progress• Enables DC’s to oversee
transactionsDownload manager• Local component for
communication with portal and preparing
data deliveries
NATO ARW September 2007
SeaDataNet User’s portal schema V1
IFREMER Database
BODC Database
BSH Database
...
Download managers in data centres
CSR EDIOS
EDMED
CDI
Shopping basket
Requestsstatus
manager
datametadata
CSR
CDI
Project info
Metadata & Data catalogues
Data request Status of request
General request Metadata request
Organisation +data source id
Data download
at BODCat BSH at MARIS+ EDMERP
services for metadata
Data downloading services
Visualisation services (WMS) for V2
Entry point for access hits
EDMO
EDIOS EDMED
EDMERP EDMO
Software
Vocabularies
Standards Cross search
AAA
Registr.
My transact.
User Register
Userregistration
NATO ARW September 2007
Data products
• SeaDataNet will provided added-value products to be created by regional centres• Mediterranean Sea• Black Sea• Baltic Sea• Arctic and North Sea• Atlantic and global ocean
• First products => gridded climatologies • based on already available data-sets (mostly hydrographic data)• updated during the project life
• To make products comparable, all regional data centers will use a common tool for spatial mapping
• For this a new version of DIVA (Data Inverse Variational Analysis) was developed (University of Liege)
NATO ARW September 2007
SeaDataNet wider involvement
• SeaDataNet partners are involved in many other European and international projects, contributing and safeguarding good data management. Examples:
• Services for the European marine geological institutes
• EDIOS Services for EuroGOOS
• Partners in the MERSEA project (GMES) for MIM
• Partners in the HUMBOLDT project (GMES) for developing a framework for a European Spatial Data Infrastructure
• Partners in SIMORC for oil & gas industry metocean data
• Partners in ECOOP for data provision and management
• Component of the GMES Marine Core Services
NATO ARW September 2007
Marine geology data
Catalogue of > 300.000 seafloor samples and >2.5 million line kilometres of marine seismic and sonar survey from the ocean basins and continental shelves, held at European institutions
www.eu-seased.net
NATO ARW September 2007
Industry metocean datasets - SIMORC
• System of Industry Metocean data for the Offshore and Research Communities• Metadatabase and Database of datasets from major oil & gas companies
www.simorc.orgAt present ca 870 data sets (Shell, BP, Total), more
underway
NATO ARW September 2007
SeaDataNet - BlackSeaScene
• BSS is an Integrating Activity of EU-Research Infrastructures programme and runs from 1 Dec 2005 – 30 Nov 2008
• BSS has 32 partners with 25 from Black Sea region and is coordinated by MARIS
• BSS is a Black Sea Scientific Network• analysing the present quality of Black Sea datasets and applied DQC procedures
• developing a Black Sea regional portal for metadata and data access
• BSS is adopting the SeaDataNet standards and building upon the SeaDataNet basic services
• BSS engages 25 Black Sea institutes, strengthening and expanding the compilation of metadata and access to data for the Black Sea region
NATO ARW September 2007
SeaDataNet capacity building
The training program has two main objectives:
•to ensure that the data and metadata to be integrated in the SeaDataNet system will be formatted, checked for quality and disseminated according to the common protocols developed
•to transfer expertise and to train IT experts of the data centers in the basics, installation and operation of the SeaDataNet technical components
NATO ARW September 2007
International Conference IMDIS 2008
• Second international conference “International Marine Data Information Systems” (IMDIS)
• March 31 - April 2, 2008, Athens, Greece • Joint organisers: SeaDataNet, Hellenic Centre for
Marine Research, IOC/IODE • Four sessions:
• Marine environmental data bases: infrastructures and data access systems
• Data Services in ocean science: standards and interoperability
• Visualisation tools• Services for Users and Education