1

ECHO and EDGStatus

May 9, 2006

Beth Weinstein, Beth.Weinstein@nasa.govYonsook Enloe, yonsook@mindspring.com

2

Modeling,Applications,

Decision SupportSystems

Collection & Granule

Catalog

Browse Images

What is ECHO?

ECHO is middleware between Data, Service, and Client Partners• Data Partners provide information about their Earth science-

related data holdings• Client Partners develop software (“clients”) to access ECHO’s

metadata using ECHO’s open APIsEnd users search ECHO's metadata using an ECHO client; ECHO is not a user interface

ECHO

ExtendedWeb

Services

DataPartner

APIs

ClientPartner

APIsMachine – to –

Machine

TailoredGraphical User

Interfaces

Client Partners Data Partners

End User

Other DataPartners

NASA DAACs

3

ECHO Capability Today

User Registration and LoginMetadata ingest, validation, and reconciliationSearch Parameters• Spatial (e.g. point, line, polygon, multipolygon, circle)• Temporal (e.g. date range, day/night/both)• Keyword (e.g. dataset id, sensor name)• Numeric (e.g. cloud cover percentage )• Boolean (e.g. Only data with browse data, Only data that is online)

Open interfaces for human-machine or machine-to-machine clientsData Access• Direct On-line Access• Brokering of Orders• Price Quotes• Subscriptions

Interoperability with other systems (OGC/NSDI Client support)Service Catalog based on web services standards

4

ECHO Data Partner Status

ECHO’s Current Holdings (May 2006) from 10 Data Partners• Collections 2,237• Granules 56 million• Browse 14 million

All NASA ECS DAACs are actively participating in ECHO (GES, LARC, LP, NSIDC)• Atmospheric Composition and Dynamics, Global Precipitation, Ocean

Biology, Ocean Dynamics, Solar Irradiance• Radiation Budget, Clouds, Aerosols and Tropospheric Chemistry• Land Processes• Snow and Ice, Cryosphere and Climate

V0 DAACs are participating (ASF, GES, JPL, ORNL, SEDAC, PO)• Synthetic Aperture Radar (SAR), Sea Ice, Polar Processes, Geophysics• Biogeochemical Dynamics, Ecological Data, Environmental Processes• Oceanic Processes, Air-Sea Interactions• Population, Sustainability

MODIS Data Processing System (MODAPS) and JAXA CEOP are in test mode

5

ECHO Client Partner Status

Current ECHO Clients• General purpose geospatial and

temporal searching• Customized user interfaces to facilitate

specific communities and tasks• Back-end harvesting tools to support

client-side caching of key information• Additional value-added processing by

clients (e.g. subset, resample, reproject, reformat)

Client Partner: 17• Operational 2• In evaluation or test 7• Active development 3• Planning/requirements 3• Proposed 2

6

EDG Transition to ECHO’s WIST Client

Warehouse Inventory Search Tool (WIST) • ECHO client being developed by NASA ESDIS• General search and order interface• Will offer all EDG functionality • Public access to current ECHO operational version

WIST is expected to be fully operational for EOS datasets by 2Q 2007ECHO must meet criteria (e.g. search performance, available, up-to-date metadata) before EDG is turned offU.S. EDG clients and servers will continue to operate until the GSFC EDG is turned off

7

ECHO Schedule

ECHO 7.0 operational•Browse Data Insert, Update, and Deletes•Multiple Collections and Groups for Access Control Rules•Spatial Query Based on Lat/Lon Point

Mar 2006

ECHO 8.0 operational•Web Services API•Asynchronous Queries

4Q 2006

WIST operational 2Q 2007

ECHO 9.0 operational•Improved performance•More Comprehensive Error Handling•Enhanced Security•Metrics

2Q 2007

ECHO 10.0 operational•EOSDIS Evolution Items

4Q 2007

8

Why Use ECHO?

Open system provides Earth science data and services to large, diverse pool of users enabling scientific community interaction and collaboration Control in the hands of the data partner• Automate mapping between your metadata and ECHO

catalog metadata • Control visibility and access to your contributed resources• Select the best spatial search approach for your data• Check the history of orders and provide status on open

ordersUsers search for collection and inventory-level data• Search and order data through a customized user interface• Directly access online data and/or order data on media

ECHO offers high system availability• 99% system availability• Even if your system is down, ECHO users can still search

your metadata

9

International Activities of Interest

CEOP (Coordinated Enhanced Observing Period) program • Plans to use ECHO and OPeNDAP enabled clients and servers• Satellite data used by CEOP will be represented in ECHO

JAXA is currently evaluating ECHO through its CEOP activityIsrael Space Agency would like to become an ECHO Data PartnerDundee (Scotland) and the IRE RAS (Russia) considering becoming ECHO Data PartnersStudying interoperability with ESA and other international partners

10

ECHO/OPeNDAP Activities

CEOP program will ingest metadata from satellite data of interest into ECHO. The satellite data is from JAXA, NASA, ESA, and EumetsatWTF-CEOP developing extensions to OPeNDAP based tools to provide access to satellite data to the CEOP science communityDirect search and access of ECHO through the web service APIs by OPeNDAP clients – prototype Matlab client will be demoed in July 2006 with operational capability expected when ECHO 8.0 is operational

11

ECHO and IDN

GCMD and ECHO are working together to share information from its registries and give users a more unified experience when interacting with the two systemsGCMD Portal to ECHO data operational in 2Q 2007

12

ECHO Project Website

Contact information and mailing listsInformation on how to get started as an ECHO Data, Client, or Service PartnerReference materials and toolsSystem access informationReal-time systems status Operations metrics updated weeklyInfo on various ECHO community meetings

http://eos.nasa.gov/echo

13

ECHO Contact Information

ECHO International contact • Yonsook Enloe (yonsook@harp.gsfc.nasa.gov)

Contact ECHO Operations (Ops)• echo@killians.gsfc.nasa.gov• +1 301 867-2071 (Weekdays, 08:00–19:00 ET)

Visit the ECHO Project Website• http://eos.nasa.gov/echo

Join ECHO Mailing List: echo-all@killians.gsfc.nasa.gov Schedule bi-lateral telecons to discuss potential collaboration!

14

Backup Slides

15

ECHO Mission/Vision Statement

ECHO Mission• ECHO’s mission is to enable a global marketplace of Earth

Observation resources that will make Earth Observation data utilization more efficient and will spark innovation. ECHO provides Earth Observation communities with the ability to publish, discover, access and integrate directory and inventory level data and services through community-developed user interfaces.

ECHO Vision  • ECHO will…

•be highly recognized, trusted and valued by the Earth Observation community

•be a critical building block in distributed information, modeling, decision support and public access systems

•have a low cost of participation to encourage broad community involvement

16

Technologies and Standards used by ECHO

Technologies• J2EE- Java 2 Enterprise Edition

• Provides a scalable (in terms of simultaneous accesses) application server which hosts our business logic

• Oracle 9i• Provides a highly tunable relational database engine with spatial search capabilities

• XML• Provides a cross-platform, cross-language basis for interacting with ECHO

• A layered, compartmentalized architecture is used to allow for updates with minimal impact to the other components of the system, including replacing the data model

Standards• Basic Profile Compliant Web Services

• Provides a cross-platform, cross-language basis for requesting ECHO to perform certain functions on the behalf of a client user, or for ECHO to request functions of a provider

• OGC Catalog Service Specification • ECHO’s current API is based on this spec, and an adapter has been built to offer true

standards compliance• The layered architecture includes a place for protocol adapters, order

adapters and ingest adapters that accommodate the differences among participating systems, minimizing the impact on those existing systems

17

ECHO Background – How did we get here?

ECHO initiated as an enhancement to EOSDIS in response to:• User feedback on complexity and limitations of the “system-wide”

view of EOSDIS data provided by EOS Data Gateway (EDG).•Belief that the community could and would develop better client

capabilities tailored to their needs.• Evolving NASA Earth science vision of multiple, distributed,

heterogeneous data and service providers.• Availability of emerging technologies (e.g. web services).

Response was development of ECHO as enabling infrastructure.• “Externalized” metadata and made it accessible via APIs that

supported development of custom clients.• Extensible architecture that allows standard client and provider

interfaces to be added.• Support for data services.

Centralized “clearinghouse” model based on industry feedback.• Driven by performance and availability requirements.

18

SUPPORTING A COLLABORATIVE EARTH SCIENCE COMMUNITY WITH INFRASTRUCTURE SERVICES

Publishing Resources: Making them available for the Earth Science Community

Discovery of Resources: Finding resources that meet science needs

Consuming Resources: Accessing and using valuable resources, individually or in combination, to meet science needs

19

Flight Operations,Data Capture,Initial Processing,Backup Archive

DataTransportto DAACS/SIPS

Science DataProcessing,Info Mgmt, DataArchive, & Distribution

Distribution,Access,Interoperability,Reuse

EOSSpacecraft

Internet

Value-AddedProviders

InteragencyData

Centers

Int’l Partners& Data Centers

Data Acquisition

White SandsComplex(WSC)

Tracking& Data

Relay Satellite(TDRS)

ResearchUsers

EducationUsers

DistributedActive

ArchiveCenters

InstrumentTeams and

SIPSs

Data Processing

&MissionControl

EOS Polar Ground Stations

MediaPublic

ESIP2/3’s

RESACs

RACs

(Search,order,

distribution)

(Distribution)

EOSDIS Context Diagram

20

EOS Missions/Instruments

21

EOSDIS Today

EOSDIS provides• A production capability for standard science data products from EOS instruments• An “active archive” of Earth science data from EOS and other past and present

missions • A distributed information framework (data centers, SIPS, networks,

interoperability, other system elements) with partners supporting EOS investigators and other users in science, government, industry, education, and policy

Partnerships No.U.S. 8International 13

Missions No. Science Data Processing 11 Archiving and Distribution 28 Instruments supported 65

ESDIS Funded Entities No.DAACs 8SIPSs 9 of 13

EOSDIS Overall Metrics (FY2004)EOSDIS Systems

System Interface Control Documents (ICDs) 58Unique Data Products 3,911Distinct users accessing DAACs 2,085,597Distinct users obtaining data from DAACs 202,815

EOSDIS Systems Metrics (FY2004)ECS

SystemV0, TSS, LaTIS

Systems

Daily Ingest Volume (Level 0) 341 GB 83 GB (est.)Daily Archive Growth 4.02 TB 0.17 TBEnd User Daily Distribution Volume 1.56 TB 0.30 TBEnd User Distribution Products 11.4 M 22.7 MTotal Archive Volume at end FY04 (L0-L4) 3.25 PB 702 TBTotal Archive Products at end FY04 (L0-L4) 53 M Not availableFilename: EOSDIS_Today_12282004.xls

22

As middleware for a service-oriented enterprise, ECHO offers entrée to its capabilities through a set of publicly available Application Program Interfaces (APIs) (see Figure 1).These ECHO APIs are based on industry standards for performing web-based computing, specifically web services profile. These service interfaces are defined in the Web Services Definition Language (WSDL) and are accessible through Simple Object Access Protocol (SOAP).Using these standards, clients written in most contemporary programming languages are isolated from the underlying technologies that support the distributed communication and functionality. These clients may call the ECHO web services much like a local function call. Most current developer tools support these standard technologies (e.g. WSDL, SOAP) natively. More information about ECHO, including a user’s guide and the API specification is available at http://eos.nasa.gov/echo.