Earth Science Data and Information System(ESDIS) Project Update

October 15-16, 2008

National Snow and Ice Data Center & Physical Oceanography DAACUser Working Group MeetingPasadena, CA

Jeanne.Behnke@nasa.govJeanne.Behnke@nasa.govESDIS Project Code 423ESDIS Project Code 423NASA GSFCNASA GSFC

2

Topics

•• ESDIS OrganizationESDIS Organization•• System ContextSystem Context•• Key MetricsKey Metrics•• Data Data ArchitectureArchitecture•• Key ActivitiesKey Activities

–– Data DiscoveryData Discovery–– Customer Satisfaction & MetricsCustomer Satisfaction & Metrics–– Operations ManagementOperations Management

3

Earth Science Data andInformation System (ESDIS) Project

This Jason sea-surface height image showssea surface height anomalies with theseasonal cycle (the effects of summer, fall,winter, and spring) removed. Each image is a10-day average of data, centered on the dateindicated.Courtesy: NASA EOSDIS PhysicalOceanography DAAC

•• The EOSDIS:The EOSDIS:–– Ingests, archives, processes, andIngests, archives, processes, and

distributes an unprecedenteddistributes an unprecedentedvolume of science data for NASAvolume of science data for NASA’’ssflagship Earth science missionsflagship Earth science missions(e.g., Terra, Aqua, Aura, (e.g., Terra, Aqua, Aura, ICESatICESat))

–– Supports unique requirements of aSupports unique requirements of avariety of Earth science disciplinesvariety of Earth science disciplines(e.g., land, atmosphere, snow/ice,(e.g., land, atmosphere, snow/ice,and ocean) as well as inter-and ocean) as well as inter-disciplinary researchers, climatedisciplinary researchers, climatemodelers, and application usersmodelers, and application users(e.g., U.S. Forest Service)(e.g., U.S. Forest Service)

–– Employs state-of-the-art hardwareEmploys state-of-the-art hardwareand software technology to achieveand software technology to achieverequired data throughputrequired data throughput

•• The ESDIS Project is responsible for the Earth ObservingThe ESDIS Project is responsible for the Earth ObservingSystem Data and Information System (EOSDIS), one of theSystem Data and Information System (EOSDIS), one of thelargest civilian Science Information System in the worldlargest civilian Science Information System in the world

4

EOSDIS Manages DataFor All 24 EOS Measurements

ACRIMSAT(12/99)

Solar Output

ACRIM

SORCE (1/03)Solar Irradiance

TIM, SIM, XPSSOLSTICE

Terra (12/99)EnergyBudget

CERES

Albedo,Aerosols,Vegetation

MISR

SurfaceImaging

MODISASTER

LowerAtmospheric

Chemistry

MOPITT

Aqua (5/02)EnergyBudget

CERES

SurfaceImaging

MODIS

AtmosphericSounders

AMSR-EAIRS/AMSU/HSB

Aura (7/04)TraceGases

TES

AtmosphericDynamics and

Chemistry

HIRDLS,MLS, OMI

Mesosphere

Stratosphere

Troposphere

TRMM(11/97)

Energy Budget

CERES, LIS

EvaporationH2O

SeaSurfaceWinds

OceanBiology

LandBiology

Volcanology

EcosystemDynamics

O3

precursorgases,

aerosols

Mission & Science 04102007.ppt

O3ClOBrOOH

trace gasesaerosols

Land Ice andSnow Cover

Rain

QuikScat(6/99)

Sea Surface Winds

SeaWinds

TemperatureMoisture

Sea Ice

Fire Occurrence

METEOR 3M(12/01)

Aerosol Trace Gases

SAGE III

CloudSat(4/06)

Cloud Properties

CPR CALIPSO(4/06)

Cloud, AerosolProperties

CALIOP

Jason (12/01)Ocean Altimetry

Poseidon/JMR/DORIS

Rain

ICESat(1/03)

Ice Topographyand Altimetry

GLAS

5

Earth Science Data Availability by Mission

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

10/1984 - 2 years planned mission life

10/1978 - 10 yrs m life

ERBS (SAGE II)

ERS-1

NIMBUS-7 (TOMS I)

Meteor 3 TOMS (TOMS II)

UARS

TOPEX/Poseidon

JERS-1

OrbView-2 (SeaStar)

ERS-2

Earth Probe TOMS (TOMS III)

RadarSat 1

ADEOS I early failure

Jason-1

ACRIMSAT

Aqua

ICESat

Aura

SORCE

Landsat 7

QuikSCAT

Terra

TRMM

Heritage Missions

EOS Era Missions

No Planned EOL

KEY

4 year Data Access Period

requirement

GRACE

Transition to USGS

Extended Mission Life

Planned Mission Life

ADEOS II early failurePost Mission Estimate

April 2007 NASA Earth

Science Senior Review

Science Data Available

CERES data only

Meteor 3M (SAGE III)

EOSDIS,operationalsince August1994,preservesearth sciencedata andproductsfrom heritagemissions aswell as EOSmissions,ensuring longterm datarecords

6

Earth Science Context Diagram

Spacecraft

Data Acquisition

GroundStations

ScienceTeams(SIPS)Polar Ground Stations

Flight Operations,Data Capture, Initial

Processing & BackupArchive

DataTransportto DAACs

Science Data Processing,Data Mgmt., Data Archive

& Distribution

Distribution, Access,Interoperability &

Reuse

NASAIntegratedServicesNetwork(NISN)

MissionServices

DataProcessing& Mission

Control

Technology Infusion

Research

Education

Value-AddedProviders

InteragencyData Centers

InternationalPartners

EarthSystem Models

BenchmarkingDSSMeasurement

Teams

Tracking &Data Relay

Satellite (TDRS)

WWW

EarthSystem Models

ACCESS

ACCESS

EOSDIS ScienceData Systems

(DAACs)

DataPools

REASoNs

ECHO

7

EOSDIS Facilities

Data centers, collocated with centers of science disciplineexpertise, archive and distribute standard data products

produced by Science Investigator-led Processing Systems(SIPSs)

ASF DAACSAR ProductsSea Ice, Polar

Processes

SEDACHuman Interactions

in Global Change

LP DAACLand Processes

& Features

PO.DAACOcean Circulation

Air-Sea Interactions

ASDCRadiation Budget,Clouds, Aerosols,Tropo Chemistry

ORNL DAACBiogeochemical

Dynamics, EOS Land Validation

GES DISCAtmos Composition &

Dynamics, GlobalModeling, Hydrology,

Radiance

LAADS/MODAPS

Atmosphere

OBPGOcean Biology &Biogeochemistry

GHRCHydrological Cycle &

Severe Weather

CDDISCrustal Dynamics

Solid EarthNCAR, U of Col.HIRDLS, MOPITT,

SORCEGSFC

GLAS, MODIS,OMI, OBPG

LaRCCERES,SAGE III

GHRCAMSR-E, LIS

JPLMLS, TES

San DiegoACRIM

NSIDC DAACCryosphere, Polar

Processes

SIPSs

KeyData

Center

DD

Data CenterOperations

SMDE. Weiler

Astro-physics

EarthScienceM. Frielich

Helio-physics

Planetary

ResearchJ. Kaye

AppliedScience

T. Fryberger

FlightPrograms

S. VolzProgram Executive for

Earth Science Data SystemsM. Maiden

Program Executive forMission Operations

– C. Yuhas

Earth ScienceData and Information

System (ESDIS) ProjectD. Lowe

Science Systems Development

OfficeD. Marinelli

Science Operations

OfficeJ. Behnke

FlightProjectsG. Morrow

Appl. Eng.&

Technology

Sciences&

Exploration

PlanetaryScience

Astro-physics

GSFCR. Strain

EarthScienceG. Colon

400

500

600

423

...

...

Terrestrial Ecology• D. Wickland• B. Emmanual• W. Turner

ORNLDAAC

Program Scientists

PhysicalOceanography• E. Lindstrom

PhysicalOceanography

DAAC

AtmosphericDynamics• R. Kakar

GSFC EarthSciences DISC

Ocean Biology and Biogeochemistry• P. Bontempi

Earth Surfaceand Interior• J. LaBrecque Crustal

DynamicsDIS

AtmosphericRadiation• H. Maring

LandProcesses

DAAC

CryoshpereScience• S. Martin National Snow

and Ice DataCenter

420

430

440

...

...

Earth Science & Data Systems

Upper AtmosphereResearch• K. Jucks

GlobalHydrology

Resource Center

Ocean BiologyProcessing

Group

SAR Systems• C. Dobson

AtmosphericSciences

Data Center

SARData Center

Applications• S. Ambrose

Application Scientists Socioeconomic Data & Applic-ations Center

DRAFT

9

0

20

40

60

80

100

120

140

160

180

FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 to

date Sep

(in Millions)

EOSDIS Products Delivered: 1996 thru Sept 2008

17,263 K

10

11

12

13

14FY08 Repeat Web Visitors by end of September

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

2 3 4 5-6 7-9 10 - 14 15 - 24 25 - 49 50 - 99 100+

Number of Visits

Nu

mb

er

of

Vis

ito

rs

ASF GESDISC LARC LPDAAC MODAPS NSIDC ORNL PODAAC

FY2008 Data Center Web Activity Sep 2008

PreviousPreviousYearYear

CurrentCurrentMonthMonth

PreviousPreviousMonthMonthUnique VisitorsUnique Visitors

FY to dateFY to date

58,33858,33897,02697,02666,43966,439Total VisitorsTotal Visitors

108,372108,372132,315132,315117,519117,519Repeat VisitorsRepeat Visitors

647,010647,010774,619774,619690,359690,359Total VisitorsTotal Visitors

Sep 07Sep 07Sep 08Sep 08Aug 08Aug 08by Monthby Month

15

NASA Earth Science Interoperable Data Architecture

16

Earth Science Data Discovery

•• Global Change Master Directory (GCMD)Global Change Master Directory (GCMD)(http:(http://gcmd//gcmd..nasanasa..govgov))–– Provides a catalog ad portal for data discovery of NASA andProvides a catalog ad portal for data discovery of NASA and

other publicly available environmental data and services.other publicly available environmental data and services.•• ECHO (EOS Clearing HoECHO (EOS Clearing House)use)

–– Provides a data granule-level metadata for all NASA EOSProvides a data granule-level metadata for all NASA EOSdata.data.

–– ECHO enables common searches ofECHO enables common searches of earth science relevantearth science relevantdata across all NASA on-line data access systems.data across all NASA on-line data access systems.

–– XML-based structure allows more easy mapping betweenXML-based structure allows more easy mapping betweendiffering metadata formats.differing metadata formats.

–– Transition to the new Warehouse Inventory Search ToolTransition to the new Warehouse Inventory Search Tool(WIST)(WIST)

•• Individual DAAC ToolsIndividual DAAC Tools•• OGC ToolsOGC Tools•• Collaboration Tools and PortalsCollaboration Tools and Portals

17

- Over 17,000metadata records- Over 1,000worldwide dataproviders- Average 5 millionvisits per month

http://gcmd.nasa.gov/

18

ECHO Status – Holdings and Partners

•• Current HoldingsCurrent Holdings–– CollectionsCollections 2,6742,674–– GranulesGranules 85 million85 million–– BrowseBrowse 28 million28 million

•• Data PartnersData Partners–– Operational Public: Operational Public: 1010–– Test: Test: 1 1 (GHRC)(GHRC)

•• Client PartnersClient Partners–– OperationalOperational 33 (WIST, SNOW-I, ESA)(WIST, SNOW-I, ESA)–– In evaluation or testIn evaluation or test22 (WECHO, AQUA)(WECHO, AQUA)–– Active developmentActive development11 (WISRD)(WISRD)

19

2008 ECHO Improvements

•• Synchronize TT/NCR prioritization scheme with ECSSynchronize TT/NCR prioritization scheme with ECS–– Defect tracking & prioritizationDefect tracking & prioritization–– Severity 1 escalation:Severity 1 escalation: are worked 24x7 until fixed for ECHOare worked 24x7 until fixed for ECHO

•• Upgraded SupportUpgraded Support–– 8:00am 8:00am –– 7:00pm EST Operations support 7:00pm EST Operations support–– 24 Hour response time to 24 Hour response time to echo@echo echo@echo messagesmessages–– 48 Hour notification prior to Operational planned downtime48 Hour notification prior to Operational planned downtime

• Data Integrity fixes include better ingest validation; spatialrepresentations fixed and enforced data rules

• Performance & scalability enhancement– Ingest performance improved from 9K events/hr to 40K events/hr

• Recovery & failover– Deployed a new redundant WIST system (recovery in under 15

mins)– Data corruption or system corruption recovery in under 6 hrs

20

EOSDIS ACSI Customer Satisfaction Survey

•• EOSDIS sponsors an annual independent customer survey inEOSDIS sponsors an annual independent customer survey inconjunction with the American Customer Satisfaction Indexconjunction with the American Customer Satisfaction Index(ACSI).(ACSI).

•• EOSDIS consistently exceeds the Federal Government averageEOSDIS consistently exceeds the Federal Government average•• Ratings in the mid to upper 70s are considered Ratings in the mid to upper 70s are considered ““very goodvery good”” by the by the

rating organization, the CFI Grouprating organization, the CFI Group

72

75

67

30 40 50 60 70 80

E-Business News &Information

(Public/Private Sectors)

NASA EOSDIS

Federal Government(Overall)

NASA EOSDIS 772008

2007

2008

2008

21

ACSI and EOSDIS Metrics System

•• ESDIS will coordinate the actions to address the 2008ESDIS will coordinate the actions to address the 2008ACSI survey results and determine focus areas for theACSI survey results and determine focus areas for thenext yearnext year–– Work with DAACs to examine specific results and actionsWork with DAACs to examine specific results and actions–– Get the message out to users that we are looking at the ACSIGet the message out to users that we are looking at the ACSI

results and have a corresponding action plan to address the keyresults and have a corresponding action plan to address the keyareas identified for improvementareas identified for improvement

–– ESDIS provides CFI GroupESDIS provides CFI Group a list of email addresses collected ina list of email addresses collected inthe previous 12 months (37,229 in 2008; 23,827 in 2007)the previous 12 months (37,229 in 2008; 23,827 in 2007)

•• CFICFI received 2,763 surveys in 2008 (2291 in 2007)received 2,763 surveys in 2008 (2291 in 2007)

•• ESDIS will continue to develop the metrics systemESDIS will continue to develop the metrics system(EMS)(EMS)–– Provide opportunity to define new metrics on utilization of dataProvide opportunity to define new metrics on utilization of data

documentation, data search, and data formatting servicesdocumentation, data search, and data formatting services–– Sponsor use of Sponsor use of NetInsight NetInsight Web tracking for finding commonWeb tracking for finding common

access patterns, guiding web page re-organization, and enablingaccess patterns, guiding web page re-organization, and enablingmore efficient access to data and documentationmore efficient access to data and documentation

22

Data Management Coordination

•• Work to facilitate core (DAAC-SIPS) andWork to facilitate core (DAAC-SIPS) andcommunity (community (MEaSURESMEaSURES) involvement in NASA) involvement in NASAES dataES data

•• Two meetings:Two meetings: system engineering &system engineering &managementmanagement–– Upcoming focus is Data Center-SIPS interactionUpcoming focus is Data Center-SIPS interaction

scheduled for November 2008scheduled for November 2008–– Regular Regular teleconstelecons–– WIKI forumsWIKI forums

•• Support the Earth Science Data Systems WorkingSupport the Earth Science Data Systems WorkingGroup (ESDSWG) and ESIP FederationGroup (ESDSWG) and ESIP Federation

•• Standards involvement - ex: HDF, Metadata,Standards involvement - ex: HDF, Metadata,FGDC, FGDC, Process definitionProcess definition

23

24

Operating Budgets and Plans

•• Every year Every year DAACs DAACs and ESDIS prepare a workand ESDIS prepare a workplan and operations budgetplan and operations budget–– The work plan addresses contract tasksThe work plan addresses contract tasks

•• ESDIS looks for activities that will improve data centerESDIS looks for activities that will improve data centeroperations while maintaining a high level of customeroperations while maintaining a high level of customersatisfactionsatisfaction

–– Budgets 5 years for input to NASABudgets 5 years for input to NASA’’s Programs ProgramOperations PlansOperations Plans

•• FY010-16 submission prepared for NASA Science MissionFY010-16 submission prepared for NASA Science MissionDirectorate Program review Apr 2009Directorate Program review Apr 2009

•• Annual excel spreadsheet entitled Annual excel spreadsheet entitled ““the snowballthe snowball””–– Due January Due January –– February time frame February time frame–– Details mission/cost-work function per fiscal yearDetails mission/cost-work function per fiscal year

»» (e.g., ops, engineering, equipment traceable to contract(e.g., ops, engineering, equipment traceable to contractWBS)WBS)

–– Heritage from 1990Heritage from 1990’’ss•• Current budget projection is stable (i.e., flat)Current budget projection is stable (i.e., flat)

–– New mission data costs added as approved by ESDISNew mission data costs added as approved by ESDIS

25

ESDIS Project Contacts

•• Dawn Lowe, Project ManagerDawn Lowe, Project Manager–– dawn.dawn.lowe@nasalowe@nasa..govgov

•• Jeanne Behnke, Science Operations OfficeJeanne Behnke, Science Operations OfficeManagerManager–– jeannejeanne..behnke@nasabehnke@nasa..govgov

•• John Moses, Science Operations OfficeJohn Moses, Science Operations Office–– john.f.john.f.moses@nasamoses@nasa..govgov

•• Please feel free to contact any of us withPlease feel free to contact any of us withsuggestions or comments!suggestions or comments!