IDN TASK TEAM
95
IDN TASK TEAM Presented by Mr. Ken McDonald
description
IDN TASK TEAM. Presented by Mr. Ken McDonald. Notes from the Cordoba meeting. MD9.4 Software Release Demonstrated Primary action items assigned to the IDN Task Team Completed tasks: set up GEOSS portal for WGISS to review - PowerPoint PPT Presentation
Transcript of IDN TASK TEAM
IDN TASK TEAM
Presented by Mr. Ken McDonald
Notes from the Cordoba meeting
MD9.4 Software Release Demonstrated Primary action items assigned to the IDN Task
Team Completed tasks:
(1)set up GEOSS portal for WGISS to review
(2)prepared related service/tool (SERF) descriptions for GEOSS-related services.
(3)sent the URL to all of WGISS for the Conference Calendar (provided on IDN website-see “calendar” tab).
(4)updated lyrics of ‘We WGISS’ and captured into an audio file.
IDN Website – Current Homepage
New IDN Map
New IDN Web Site Header
New tabs that link Data Sets, Data Servicesand the Conference Calendar. Action (3) from Cordoba
Status of IDN Site
• IDN Website continues to use Zope/Plone.
• Discussion List: Please use Interop to provide feedback on:– DIF extension for Models– Renaming of Keyword Fields– Geographic locations Hierarchy– Platform/ Instrument Restructure
Numerical Model Fields Proposed (2002/03)• Anne Sophie Archambeau from IPSL, France proposed an addition
to the DIF to handle numerical output data sets:
Group: Numerical_Experiment Model_Name: Model_Version: Model_URL: Model_Configuration: Model_Resolution: Model_Calendar: Group: Model_Integration_Period Start_Date: Stop_Date: End_Group Simulation_Name: Initial_Conditions: Perturbation: Imposed_Boundary_Conditions: End_Group
• Interop discussions on the proposed new fields are summarized in the CEOS IDN Interop Newsletter #15 for April 2003:
First proposed in 2002First proposed in 2002
http://gcmd.gsfc.nasa.gov/pipermail/interop/2003-April/000011.html
Global Organization for Earth System Science Portal (GO-ESSP) – June 2005 Meeting
New Proposal for numerical extensions to the DIF
• DIF describes datasets at discovery level.
• Where simulations are involved, discovery metadata needs more information than is available in the existing schema.
• New DIF extension schema for numerical model output proposed by the British Atmospheric Data Center (BADC) can be found at http://ndg.badc.rl.ac.uk/kwiki/ndg/index.cgi?DIFExtensionV005
• Additional information and draft examples can be found at http://ndg.badc.rl.ac.uk/kwiki/ndg/index.cgi?ProposedChangesToDIF
Comments are welcome!Comments are welcome!
A new model keyword listA new model keyword listGlobal Coupled Climate System ModelsGlobal General Circulation Models (GCM)Earth System Component Models (Global or Regional)Process Models: Clouds and PrecipitationPhenomenological/Decision Making ModelsGeophysical Models (as in Earth Geophysics)Solar-Atmospheric Space-Weather ModelsEnergy Balance ModelsRadiative-Convective Models
A higher level of hierarchy?
Diagnostic and Simulation ModelsData Assimilation ModelsForecast, Prediction, Projection (Scenario’s) ModelsMonitoring and Early Warning Systems
New DIF/SERF Changes
• Parameters: will be renamed to Science Keywords:• Service Parameters: will be renamed to Service
Keywords:• Keyword: will be renamed to Ancillary Keyword:
• Add field to Temporal Coverage for Relative Time:Group: Temporal Coverage
Start_Date: [yyyy-mm-dd] Stop_Date: [yyyy-mm-dd] Relative_Start_Date: -x (where x= # days ago)End_Group
Relative time will allow for:– Auto updating of temporal coverage for near-real time data. – Specifying number of days data available.
Instrument and Platform Clean-up
• Implement an option to list all instruments and platforms alphabetically for searching.
• Expanded database table structures for Instrument and Platform information and categories.
• Extend Supplemental Information to include expanded Instrument and Platform information.
• Feedback Welcome!
Instrument and Platform Clean-up
• Instrument database information can include:– Instrument Name (currently exists as short name > long name– Categories: types, subtypes (can be used in web interface for
browsing. For example:• Earth Viewing vs. Space Viewing vs. in situ, etc.• Wavelength region (microwave vs. visible, etc.)
– Channels (if appropriate)– Spectral information; instrument information.
• Platform database information can include:– Platform name (currently exists as short name > long name– Categories: type of platform (e.g. Landsat Series)– Instrument names (if appropriate): link to instrument
database table.– Platform orbital and launch information.
Instrument ExamplesGroup: Instrument Instrument_Category: Earth Remote Sensing Instruments Instrument_Type: Passive Remote Sensing Instrument_Subtype: Sounders Short_Name: AIRS Long Name: Atmospheric Infrared Sounder Group: Spectral Information Wavelength_Keyword: Infrared Wavelength_Keyword: Visible Number_Channels: 21 Spectral_Coverage: 0.41-15.4 micrometers Spectral_Resolution: 1200 IFOV: 0.185 - 1.1 degrees Radiometric_Resolution: 2.3-13.5 km End_Group Group: DescriptionThe AIRS instrument …. End_Group Online_Resource: http://www-airs.jpl.nasa.gov/ Creation_Date: 2005-08-25 Revision_Date: 2005-08-25 IDN_Node: USA/NASAEnd_Group
Group: Instrument Instrument_Category: In Situ Instruments Instrument_Type: Trace Gas Detectors Instrument_Subtype: Chromatographs (?) Short_Name: ACATS Long Name: Airborne Chromatograph for Atmospheric Trace Species Group: Spectral Information Wavelength_Keyword: (r) Number_Channels: Spectral_Coverage: Spectral_Resolution: IFOV: Radiometric_Resolution End_Group Group: Description
ACATS is....
End_Group Online_Resource: http://www.cmdl.noaa.gov/hats/airborne/acats.html Creation_Date: Revision_Date: IDN_Node: USA/NOAAEnd_Group
Instrument Examples
Platform ExampleGroup: Platform Platform_Category: Earth Observing
Satellite Platform_Series: EOS Short_Name: Aqua Long_Name: Group: Instruments Aboard Instrument_Short_Name: MODIS Instrument_Short_Name: AMSR-E Instrument_Short_Name: AMSU-A Instrument_Short_Name: CERES Instrument_Short_Name: AIRS Instrument_Short_Name: HSB End_Group Group: Launch Launch_Date: 2002-04-02 Launch_Site: Vandenberg AFB, CA End_Group Group: Orbit Orbit_Altitude: 705 km Orbit_Inclination: 98.2 degrees Equator_Crossing: Period: 99 minutes Repeat_Cycle: 16 days Perigee: 673 km Apogee: 686 km Orbit_Type: Sun Synchronous Swath_Width_Range: End_group Coverage: Global Group: DescriptionThe Aqua satellite.... End_Group Creation_Date: 2005-08-25 Revision_Date:
Node Reports
0:035
JCADM Report
Antarctic Antarctic Master Master
DirectoryDirectoryHome PageHome Page
Antarctic Master DirectoryNADC Portals
The alias format is http://gcmd.nasa.gov/portals/portal_name/Examples of alias: http://gcmd.nasa.gov/portals/amd_ar/
• Belgium Antarctic Program created January 2003 • Finnish Antarctic Program created January 2003• Centro de Datos Antárticos, Argentina created August 2003• Canadian Polar Commission/Canadian Committee for Antarctic Research created
October 2003• Swiss Committee on Polar Research created November 2003• US Antarctic Data Centers/US Antarctic Data Coordination Center created February
2004• Japan National Institute for Polar Research created February 2004 • Estonian Antarctic Data Center created July 2004• Centro Nacional de Datos Antárticos, Spain created February 2005• Instituto Antartico Uruguayo created February 2005• Italian Antarctic Program created February 2005• Institut Polaire Francais Paul Emile Victor, France created June 2005• Centro Nacional de Datos Antarticos, Chile created July 2005
Antarctic Master DirectoryContent Statistics – Growth 2002 through 2005
NU
MB
ER
OF
DIF
S
Demo Mapserver for Ukraine
00:55
GCMD Ukraine/CEOS Prototype Mapserver
Goals:
• Incorporate geospatial data from various sources: ESRI, NOAA, USDA, and NASA
• Provide users information to familiarize themselves with the Ukraine region.
• Provide direct links to metadata and geospatial data sets. GIS Layers:
• Thematic Layers: Political Boundaries, Roads, Water Bodies, Cities.
• Raster Layers: World Land Cover, World Ecoregions, MODIS Imagery.
http://mapserver.gsfc.nasa.gov/website/CEOS/
GCMD Prototype Mapserver Interoperability
• Access to GCMD’s Web Mapping Services (WMS) through the ESRI OGC Connector.
• OGC WMS compatible clients can access GCMD’s map services.
GCMD MapserverUser Friendly Desktop GIS
(WMS Client)
Direct Access to Map Service & Metadata
Layers Link to Metadata
Data Query
Ecosystems of the Kiev Region
New Keywords since Last CEOS Meeting
PART C
1:05
New/Modified Earth Science Keywords Since May 2005
• NEW Terms and Variables for Sun-Earth Interactions– Solar Energetic Particle Properties– Solar Energetic Particle Flux– Ionosphere/Magnetosphere Dynamics
• Some Existing Variables Modified
Ionosphere/Magnetosphere Dynamics:Solar-Energetic Particles: Aurorae
Energy Deposition Electric Fields/Electric CurrentsParticle Composition Geomagnetic ForecastsParticle Density Geomagnetic IndicesParticle Distribution Functions Ion Chemistry/IonizationParticle Speed Magnetic Fields/Magnetic CurrentsParticle Temperature Magnetic StormsTotal Electron Flux Plasma Waves
Solar WindSolar Energetic Particle Flux Solar Activity:
Alpha Particle Flux Corona HolesElectron Flux Coronal Mass EjectionsProton Flux Coronal PropertiesIon Flux Solar Active RegionsHeavy Nuclei Flux Solar FlaresNeutral Particle Flux Solar ImagerySub-Atomic Particle Flux Solar Irradiance
Solar OscillationsSolar Prominences/Solar FilamentsSolar Radio Wave EmissionsSolar Synoptic MapsSolar Ultraviolet EmissionsSolar Velocity FieldsSolar X-Ray EmissionsSunspots
New Sun-Earth Interactions KeywordsEarth Science > Sun-Earth Interactions
New TERM
New Variable
Space Science/Astronomy Keywords
• Added new Category > Space Science to accommodate space science/astronomy researchers in Antarctica supporting JCADM and the NSF Polar Program. – Space Science Category keywords can be selected from the docBUILDER
tools, but are not yet available for searching for data.
• Need assistance from IDN in defining Space Science/Astronomy Topics, Terms and Variables.
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Jan-02Mar-02May-02Jul-02Sep-02Nov-02Jan-03Mar-03May-03Jul-03Sep-03Nov-03Jan-04Mar-04May-04Jul-04Sep-04Nov-04Jan-05Mar-05May-05Jul-05
Month
Hits
Spectral/Engineering
Search Hits on Topic Spectral/Engineering
Drop-off was an artifactof the statistics and nota real decline in usage.
Detailed Variable Review
• Staff is currently reviewing all Detailed Variables for spelling, clarity, and duplication for consistency and validity.– Sections completed include Atmosphere,
Paleoclimate, Solid Earth, Sun-Earth, Human Dimensions, Oceans, and Land Surface.
New Data Center, Location, Instrument, and Platform
Hierarchies for Improved SearchMulti-year Effort
Comments Welcome
Data Centers and Locations
• Data Center “Bucket” clean-up has been completed and implemented!
• Location “Bucket” has been completed and implemented!– Database hierarchies rather than buckets will be
implemented for easier location navigation and maintenance.
• Data Centers and Locations can be searched hierarchically.
New Data Center Hierarchy Search
Shows long nameFor all data center namesWhen you place your mouse over the data center name
New Data Center Hierarchy Search
New Location Hierarchy Search
New search box for locationSearch or map date search
Choose from among top levellocation categories
SelectContinent
New Location Hierarchy Search
Select Europe
Breadcrumbs show you where you are in the hierarchy
Continent category is subdividedinto individual continents
New Location Hierarchy Search
Category Europe is further subdivided into individual countries and regions
Select Eastern Europe
Breadcrumbs
New Location Hierarchy Search
Category Eastern Europe is further subdivided into individual countries and regions
Select Ukraine
Breadcrumbs
New Location Hierarchy Search
Results for search on“Ukraine” are displayed
Breadcrumbs
New Location Hierarchy Search
URL Content Type Cleanup
• Provide an accurate account of related information in the DIF and SERF.
• Provide “Direct Access” to data URL to get quick access to data.
1. GET DATA
2. WEB MAPPING SERVICE (WMS)
• Provide “Direct Access” to Services
1. GET SERVICE (Online Clients/Web Services)
• Provide “Supplemental Information” that describes URL Content Types.
Objective:
URL Content Type Keywords
ASSOCIATED DATA SET(S) BDP METADATA CALIBRATION/VALIDATION DATA DIAL DATA SERVER DISC-AIRS DISC-MODIS DISC-OMI DODS_DIR DODS_FILESERVER DODS_LAS DODS_URL EML METADATA FGDC METADATA GCMD MAPSERVER GET DATA GET SERVICE NBII METADATA NOMADS DATA SERVER PROJECT HOME PAGE REFERENCE MATERIALS RELATED DATA SET DESCRIPTION RELATED DATA SET METADATA RELATED SERVICE METADATA SOAP API SOFTWARE PACKAGES UDDI REGISTRY WEB COVERAGE SERVICE (WCS) WEB FEATURE SERVICE (WFS) WEB MAPPING SERVICE (WMS)
GET RELATED DATA SET(S)
GET RELATED SERVICE(S)
GET EXTENDED METADATA RECORD
VIEW RELATED DATA SET DESCRIPTION
VIEW RELATED SERVICE DESCRIPTION
GET DIF METADATA RECORD
GET SERF METADATA RECORD
VIEW PROJECT HOME PAGE
VIEW RELATED REFERENCE MATERIAL
GET WEB FEATURE SERVICE (WFS)
GET WEB COVERAGE SERVICE (WCS)
GET WEB MAPPING SERVICE (WMS)
GET DATA
GET SERVICE
GET SOFTWARE PACKAGES
Current List New Proposed List
DIF/SERF POPULATION
6000
8000
10000
12000
14000
16000
18000
Jan-99Apr-99Jul-99Oct-99Jan-00Apr-00Jul-00Oct-00Jan-01Apr-01Jul-01Oct-01Jan-02Apr-02Jul-02Oct-02Jan-03Apr-03Jul-03Oct-03Jan-04Apr-04Jul-04Oct-04Jan-05Apr-05Jul-05
Date
# DIFs
Total DIFs January 1999 - August 2005
split of Aqua MODIS DIFs from Terra MODIS DIFs
new Antarctic DIFs
WHFC children DIFs
Cumulative DIF Population 2003-2005
11500
12000
12500
13000
13500
14000
14500
15000
15500
16000
16500
Jan-03Feb-03Mar-03Apr-03May-03Jun-03Jul-03Aug-03Sep-03Oct-03Nov-03Dec-03Jan-04Feb-04Mar-04Apr-04May-04Jun-04Jul-04Aug-04Sep-04Oct-04Nov-04Dec-04Jan-05Feb-05Mar-05Apr-05May-05Jun-05Jul-05Aug-05
Month
# DIFs
New and Revised Supplemental Descriptions
0
20
40
60
80
100
120
140
160
180
200
Jan-03Feb-03Mar-03Apr-03May-03Jun-03Jul-03Aug-03Sep-03Oct-03Nov-03Dec-03Jan-04Feb-04Mar-04Apr-04May-04Jun-04Jul-04Aug-04Sep-04Oct-04Nov-04Dec-04Jan-05Feb-05Mar-05Apr-05May-05Jun-05Jul-05
Month
# New/Revised
Data Centers New2 Data Centers Revised2 Sources New3 Sources Revised3
Sensors New4 Sensors Revised4 Campaigns New5 Campaigns Revised5
New, Revised and Deleted DIFs 2003-2005
0
200
400
600
800
1000
1200
1400
1600
1800
Jan-03Feb-03Mar-03Apr-03May-03Jun-03Jul-03Aug-03Sep-03Oct-03Nov-03Dec-03Jan-04Feb-04Mar-04Apr-04May-04Jun-04Jul-04Aug-04Sep-04Oct-04Nov-04Dec-04Jan-05Feb-05Mar-05Apr-05May-05Jun-05Jul-05Aug-05
# DIFs
Month
DIFs New DIFs Deletes Revised
DIF Population by Earth Science Topic
AGRICULTURE5%
ATMOSPHERE17%
BIOSPHERE16%
CLIMATE INDICATORS1%
CRYOSPHERE4%
HUMAN DIMENSIONS10%
HYDROSPHERE7%
LAND SURFACE12%
PALEOCLIMATE2%
SPECTRAL/ENGINEERING
6%
OCEANS13%
SOLID EARTH6%
SUN-EARTH INTERACTIONS1%
AGRICULTURE ATMOSPHERE BIOSPHERE CLIMATE INDICATORS CRYOSPHERE
HUMAN DIMENSIONS HYDROSPHERE LAND SURFACE OCEANS PALEOCLIMATE
SPECTRAL/ENGINEERING SUN-EARTH INTERACTIONS SOLID EARTH
G3OS4%
CLIVAR0% UN
7%
NCAR1%
PNRA1%
NOAA18%
CONAE1%
USDA7%
NEONET0%
CNES0%
DLR1%
USGS8%
CIESIN2%
ESA/ESRIN2%
AMD32%
UNEP/GRID6%
JST1%
CSIRO0%
CCRS7%
INPE0%
JAXA2%
RAS0%
NOAA USDA CIESIN USGS CONAE INPE CCRS ESA/ESRIN
DLR CNES NEONET PNRA RAS JAXA JST CSIRO
UNEP/GRID AMD G3OS CLIVAR UN NCAR
DIF Population by Node
NASA41%
NOAA11%
G3OS3%
CLIVAR0%
NEONET0%
CNES0%
DLR0%
ESA/ESRIN1%
USGS5%
CIESIN1%
USDA4%
AMD19%
UN4%
NCAR1%
CCRS4%
UNEP/GRID3%
CSIRO0%
JAXA1%
JST1%
INPE0%
CONAE0%
RAS0%
PNRA1%
NASA NOAA USDA CIESIN USGS CONAE INPE CCRS ESA/ESRIN DLR CNES NEONET
PNRA RAS JAXA JST CSIRO UNEP/GRID AMD G3OS CLIVAR UN NCAR
DIF Population by Node, August 2005
IDN SERF Population
Total SERFs In GCMDJan. 2001- August 2005
0
200
400
600
800
1000
1200
1400
Jan-01 May-01 Sep-01 Jan-02 May-02 Sep-02 Jan-03 May-03 Sep-03 Jan-04 May-04 Sep-04 Jan-05 May-05
Month
SERF Totals
Distribution of SERFs
Hazards Management3%
Reference and Information Services
10%
Metadata Handling3%
Data Management/ Data Handling
17%
Data Analysis & Visualization
32%
Models12%
Environmental Advisory8%
Education/Outreach15%
Data Analysis & Visualization Data Management/Data HandlingEducation/Outreach Environmental AdvisoryHazards Management Reference and Information ServicesMetadata Handling Models
Distribution of SERF Usage
Environmental Advisory12%
Education/Outreach11%
Data Handling12%
Data Analysis & Visualization21%
Models15%
Metadata Handling10%
Hazards Management9%
Information and Reference Services
10%
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Aug-02 Nov-02 Feb-03 May-03 Aug-03 Nov-03 Feb-04 May-04 Aug-04 Nov-04 Feb-05 May-05
Months
Number of Hits
Data Analysis & Visualization Data Manamgent/Data HandlingEducation/Outreach Environmental AdvisoryHazards Management Reference and Information ServicesMetadata Handling Models
User Activity (Service Parameter Hits)August 2002 – January 2005
Introduction of the new web page
Indexed by GOOGLE
Possible result of several GIS
conferences
WEB USAGE
IDN Usage by Domain Type - 2005
.gov3%
.edu7%
.org1%
.com19%
.net22%.mil
1%
numeric23%
international23%
.us1%
.gov .edu .org .com .net .mil numeric international .us
0
2000
4000
6000
8000
10000
12000
14000
Jan-02Mar-02May-02Jul-02Sep-02Nov-02Jan-03Mar-03May-03Jul-03Sep-03Nov-03Jan-04Mar-04May-04Jul-04Sep-04Nov-04Jan-05Mar-05May-05Jul-05
Month
Users by Domain
.gov .edu .org .com .net .mil numeric international .us
IDN Usage by Domain 2002-2005
Introduction of the new web page
Cache opened toInternet robots
0
1000000
2000000
3000000
4000000
5000000
6000000
Jan-03Mar-03May-03Jul-03Sep-03Nov-03Jan-04Mar-04May-04Jul-04Sep-04Nov-04Jan-05Mar-05May-05Jul-05
month
#hits
# Web Page Hits Since January 2001
Introduction of the new web page
Cache opened to InternetSearch robots
25000
45000
65000
85000
105000
125000
145000
Jan-03Feb-03Mar-03Apr-03May-03Jun-03Jul-03Aug-03Sep-03Oct-03Nov-03Dec-03Jan-04Feb-04Mar-04Apr-04May-04Jun-04Jul-04Aug-04Sep-04Oct-04Nov-04Dec-04Jan-05Feb-05Mar-05Apr-05May-05Jun-05Jul-05Aug-05Month
Sessions
# Web Sessions Since January 2003
Introduction of the new web page
Cache opened to InternetSearch robots
DIF/SERF SEARCH/RETRIEVAL
Searches by Controlled Keyword
Agriculture10%
Atmosphere15%
Biosphere8%
Climate Indicators5%
Cryosphere4%
Human Dimensions7%
Hydrosphere6%
Land Surface11%
Oceans16%
Paleoclimate4%
Spectral/Engineering5%
Sun-Earth Interactions2%
Solid Earth7%
Agriculture Atmosphere Biosphere Climate Indicators Cryosphere
Human Dimensions Hydrosphere Land Surface Oceans Paleoclimate
Spectral/Engineering Sun-Earth Interactions Solid Earth
Update on Most Current Software
Middleware Searching Architecture
User Search
Central IndexGlobal Change Master Directory
ToolsData contributed by staff, universities,
federal and local agencies, international agencies and
organizations, or commercial providers
Other Data Contributions
UNEP (4%)
JCADM (18%)
NOAA (11%)
GeoConnections (4%) Harvesting InterfacesHarvesting Interfaces
(e.g. OAI-PMH)(e.g. OAI-PMH)
Data sharing nodesData sharing nodes
CEOP , GLOBEC, GOFC, GOSIC, IAI, JAXA, NCAR, NOAA, OPeNDAP, UN, USDA, WDC,
NASA Contribution (42%)EOS DAACs, ESIP,
NASA Centers Portals API Interfaces Data contributed through various portal collaborations
GCMD Search Engine
FreetextFreetext (Lucene) Database Index Layer
SpatialSpatial Database Index Layer
Controlled VocabularyControlled Vocabulary Database Layer joinsdelegates
ControllerController
User performs query
1)
2)
3)
Middleware Search/Retrieval Component(Integration of spatial, freetext, and controlled queries)
4) Set of unique IDs Result Result ProcessorProcessor
Multi-Layer Search ComponentMulti-Layer Search Component
5) Title set information, brief summary, dataset links, etc.
6) Returns to user
SearchSearchResultsResults
Cache7) Refines search
Data provider
MD ServerStart docBUILDER
Click on Location Field
Click on Data Center Field
load
Unrecognized Personnel
Set Personnel
Unrecognized Instrument Keyword
Set instrument Keyword
Get Location Keywords
Get Data Center Keywords
Get Personnel
validate
Commit to database
Set Personnel
load
Unrecognized Personnel
Unrecognized Instrument Keyword
Set instrument Keyword
Ingest of Metadata: Sequence diagramInteraction among User, OPS/docBUILDER and MD Server
User ActionSoftware Action
OPS/
docBUILDER
Services Services APIAPI
RM
I/SO
AP
Controlled Controlled Vocabulary Vocabulary
APIAPI
Dataset Dataset APIAPI
PublishedInterfaces
Operations Operations
Tool/ Tool/ Third Party Third Party ClientsClients
(e.g.(e.g. OPeNDAP)OPeNDAP)
Database
Sec
urit
y L
ayer
MiddlewareMiddleware
GCMD Architecture
XML Loader
Data Object ModelDIF SERF Keywords
Persistent FrameworkCore Transaction Peer
Query EngineLucene RDMS Spatial
HTTP
Full TextSearch Interface
SubscriptionService Interface
KeywordInterface Search
MetadataAuthoring Tool
Spatial/TemporalSearch Interface
Portal Interfaces
OpenAPI
Web Application Server (Tomcat)Web Application Server (Tomcat)
Global Change Master Directory
search data sets
refine results OPeNDAP (DODS)
MODIS
Other
Data Access SystemsData Access Systems
Data Retrieval Flow Diagram
refined search results
search results
view full record
request data
requested data set
full data set description
Local Database Agent
Network
New Content
Local Database Agent
HTTP Server
(Jetty/Tomcat)
OAIRepository
DBMD
Server
Local Database Agent
DB
MD Server
Local Database Agent
MD Server
Local Database Agent
DB
OAI Network
FileBased
Repository
OAIHarvester
Mirrors Table
MetadataControlledVocabulary
IncomingQueue
Commit
docBUILDER(Authoring Tool)
NCAR
OAI-PMHRepository
Identify ListMetadataFormats (DIF,
FGDC, Dublin Core) ListIdentifiers
Get Record ListRecordsGCMD
Controlled Vocabulary
OAI-PMHHarvester
Metadata
Get
rec
ords
OAI/PMH (Open Archives Initiative-Protocol for Metadata Harvesting) Capabilities
Retrieves DIF Metadata identifiers, records in XML
Identifiesprotocol; supported formats
New Portals added since March 2005 (Current total: 55)
CEOS Services Portal GEOSS
Data Portal Services Portal
Arctic Portal Arctic France Portal
AMD France Portal
AMD Chile Portal
NOAA NCDC Portal
New Features added since April 2005
• New Keyword interface– richer location hierarchy
– richer data center hierarchy
• MD Server upgrade– support for data resolution
"valids”
– started finalizing DB schema with support for platform and instrument bucket changes
• docBUILDER – Standalone and HTML version released
• New collaborative portals – 7 portals added
• New Geospatial One Stop projects added– Includes NASA sun-earth projects
• Top 10 DIF/SERF based on monthly stats – “User’s Choice” box on home page
features highlights for the month
Data Resolution “Valids”• Reminder! DIF authors can now populate their DIFs by selecting
horizontal, vertical and temporal resolution in docBUILDER.
• Vertical Resolution refers to both Altitude and Depth resolution.
< 1 second
1 second - < 1 minute
1 minute - < 1 hour
Hourly - < Daily
Daily - < Weekly
Weekly - < Monthly
Monthly - < Annual
Hourly Climatology
Daily Climatology
Pentad Climatology
Weekly Climatology
Monthly climatology
Annual
Annual climatology
Decadal
Climate Normal (30-year climatology)
Temporal Resolution Range: Horizontal Resolution Range:
< 1 m1 m - < 30 m30 m - < 100 m100 m - < 250 m250 km - < 500 m500 m - < 1 km1 km - < 10 km 10 km - < 50 km 50 km - < 100 km100 km - < 250 km 250 km - < 500 km 500 km - < 1000 km > 1000 km or > 10 degrees
Vertical Resolution Range:
< 1 m1 m - < 10 m10 m - < 30 m
30 m - < 100 m100 m - < 1km> 1km
MD9.4.0 Release: Highlights
New Free-Text Search Capability powered by
New secondary refinement capabilities by keywords or free-text for data sets.
New Data Center and Location hierarchies.
New brief excerpt of record summary display with results and search term highlighting within the metadata records.
New customized portal interfaces with additional navigation features.
New spatial and temporal resolution range keywords within the data resolution fields now available for population (but not for search until a majority of records contain this information).
PostgreSQL
• Database setup scripts are being customized to work with an open source database.– Scripts are being updated to work with
PostgreSQL 7.4.7 database. – Currently testing MD Server setup with
PostgreSQL.– Install instructions will be rewritten to make
installation as easy as possible for Nodes.
New docBUILDER
Highlights of New docBUILDER Tool(Changes the users will see)
• Offers option to use personalized templates.
• Supports three-color coding for required, recommended and optional fields.
• Provides automatic population of Data Center /Service Provider URLs.
• Allows direct editing of the XML document.
• Offers more intuitive interface for Personnel and Data Center fields.
• Incorporates spell-checker for all fields and full document.
= HTML = Both = Standalone
Highlights of New docBUILDER Tool (Changes “behind the scenes”)
• Complies fully with XML Schema (previous version only supported DTD). – Provides capability of rendering custom authoring tool
according to the XML schema– Renders customized widgets based on XML element names
association.• Contains new business logic with a completely rewritten
business layer, resulting in much faster access to XML documents and valids.
• Separates business logic from presentation logic, making changes to presentation much easier.– Refactored framework using Struts and JSP allowing for
complete portal customizations.
HTML docBUILDER docBUILDER
HTML docBUILDER(Opening Screen)
HTML docBUILDER(Main Screen)
Editable text area
Menu with optionsto save, email, etc.
Legend
Checkboxes indicating progress
HTML docBUILDER(Data Resolution Field)
Add/Remove field: user can click the green (+) to add a new field, or the red (–) to remove that field.
Link to full field description: link to the DIF/SERF Writer's Guide page that fully describes the field, including characters allowed, definition, format, etc.
Field Name
Field Description: short description of the filed, including the type of information expected.
HTML docBUILDER(Science Keywords Field)
Subset list selector
Variable level filter: if checked, removes all items with Variable level
List search: filters list as you type, keeping only those items that match.
Option to add detailed variable: allows user to add detailed variables to any selected items in the list
HTML docBUILDER(Personnel Field)
Personnel Lookup link: when clicked, presents the user with all personnel currently in the GCMD DB. After the user selects a person and clicks the Accept button, the information is then populated in the form on the Personnel page.
c l i c k !
HTML docBUILDER(Data Center Field)
Data Center Lookup link: allows the user to select from a current list of data centers, then populates the Data Center form with this info, including URL.
c l i c k !
MD9.5 Plans for the future
MD9.5 Projected Release: December 2005New refinement capabilities for services by keywords or free-text.
Keyword subscription service.
Relative Time Capability for data sets.
Automated update reminders.
Platform/Instrument Upgrade and Clean-up.
Open Archives Initiative (OAI) - Protocol for Metadata Harvesting (PMH) substitution for Local Database Agents (LDA)
Central Repository for very small data sets.
Upgraded spatial search.
Integrate SOAP API.
Server software upgrade to handle search on temporal/spatial resolution. This will be dependent on adequate population of the resolution field by authors.
Keyword Subscription Service
• Receive notifications of updates, removals, and additions to keywords for:– Data Centers/Service Providers– Platforms– Instruments– Locations– Projects
Search Refinements for SERF MD9.5
• Refine by:– Service Providers– Instruments– Platforms– Projects– Science Keywords– Service Keywords– Parent SERFs– Child SERFs
Methods to refine by
Close-up of Refinement
Users can also refine by Parent and
Children SERFs
