John Kusterer Head, ASDC
ASDC UPDATE CERES SCIENCE TEAM MEETING
MAY 5, 2015
May 5, 2015 ASDC Update for CERES STM Page: 2
Agenda
• CERES User Metrics • ASDC Private Cloud Pilot • Big Earth Data Initiative (BEDI) and Data Services
ASDC CERES USER METRICS
May 5, 2015 ASDC Update for CERES STM Page: 4
0
10
20
30
40
50
60
70
80
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Volu
me
(TB
)
TERRA AQUA NPP FLASH
CERES and FLASHFlux Archive Volume By Data Date Through March 2015
Total Volume Archived ~ 1,104 TB NPP Volume Archived ~ 14.1 TB
May 5, 2015 ASDC Update for CERES STM Page: 5
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
2010 2011 2012 2013 2014 2015
Num
ber o
f Ord
ers
CERES Data Orders (June 2010 – March 2015)
Total Orders: 27,242
May 5, 2015 ASDC Update for CERES STM Page: 6
CERES Data Distribution (June 2010 – March 2015)
0
10
20
30
40
50
60
70
80
90
100
2010 2011 2012 2013 2014 2015
Volu
me
(TB
)
May 5, 2015 ASDC Update for CERES STM Page: 7
Number of Users by Product (June 2010 – March 2015)
39 19
87 52 58
438 38 35
90 99
50 9
32 49
31 49
66 256
334 423
83 377
831
0 100 200 300 400 500 600 700 800 900
BDS CERES-MISR-MODIS
CER-NEWS ES8
FLASH-SSF SSF CRS ES9 ES4
ISCCP-D2like SFC
ZAVG FSW
FLASH-TISA SYN
SRBAVG AVG
SYN1deg-lite SSF1deg SYN1deg
ESG EBAF-Surface
EBAF-TOA
May 5, 2015 ASDC Update for CERES STM Page: 8
User Affiliations
30%
29%
22%
19%
Commercial Educational Government Other
Users by Country (June 2010 – March 2015)
ASDC PRIVATE CLOUD PILOT
May 5, 2015 ASDC Update for CERES STM Page: 11
ASDC Private Cloud Pilot
• ASDC has participated in numerous cloud experiments and research efforts • Cloudbursting, experiment with LaRC OCIO, others
• Learning has been exceptional, progress not so much • IT Security has been a challenge • Cost models evolving • Different groups leading different efforts
• Often looks like “solution looking for a problem” • Application hosting • Data storage • Collaborative processing
May 5, 2015 ASDC Update for CERES STM Page: 12
ASDC Private Cloud Pilot
• ASDC setting up a private cloud in ASDC facility initially only available to internal (LaRC) users • Engineers out IT Security issues • Allows us to collectively learn about cloud realities • Addresses real-world challenges (i.e., episodic/surge
processing requirements)
• Private cloud system being setup initially • Received from NCCS • 480 node 3 year old Dell system • All x86 based • Pilot targeted to be operational in September 2015 • Eucalyptus and Openstack cloud technologies being utilized
May 5, 2015 ASDC Update for CERES STM Page: 13
ASDC Private Cloud Pilot
296 206 288 768
5,760
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
AMI AMI-P ASDC Cloud Pilot:
Eucalyptus
ASDC Cloud Pilot:
Openstack
Total Capacity (Potential)
X86
Cor
es
x86 Cores
May 5, 2015 ASDC Update for CERES STM Page: 14
ASDC Private Cloud Pilot
• Will identify potential processing candidates to run on pilot cloud in coming months
• Will evaluate as we exercise cloud and adjust plans accordingly
BIG EARTH DATA INITIATIVE (BEDI) AND DATA SERVICES
May 5, 2015 ASDC Update for CERES STM Page: 16 ASDC Update for CERES STM
ASDC has four BEDI groups based on BEDI assigned tasks working on most current versions of CERES, CALIPSO, MISR, MOPITT, TES 1. OPeNDAP -‐ con&igure hardware and software to implement data
for availability through the OPeNDAP protocol and other Open Geospatial Consortium (OGC) services as appropriate.
2. GCMD and ECHO Reconciliation -‐ provide a thorough review of current metadata and ensure all required metadata for identi&ied ASDC data products is reconciled.
GIBS
3. GIBS -‐ coordinate with science teams to create and offer subsets identi&ied images via GIBS. Coordination includes discussions with science teams to de&ine and validate color scales and ingest testing with GIBS.
4. DOI -‐ assess the current DOI registration status for the most current versions of identi&ied products and provide the required metadata for each product to be registered (each DOI is updated based on Earth Science Data Types (ESDTs), EOSWEB Landing pages, Information Management System (IMS), GCMD, and ECHO as appropriate.
Big Earth Data Initiative (BEDI)
May 5, 2015 ASDC Update for CERES STM Page: 17
ASDC Data Services • Standardized services that can be accessed in many
different applications
• ArcGIS Image Service • Imagery and data service • Processing templates and functions • NASA Earthdata Webinar: https://www.youtube.com/watch?
v=N_BC7ZrWUwY
• OPeNDAP • provides subsetting, dataset aggregations, and conversion
to/from various data formats
• Web Mapping Service (WMS) • Imagery service with basic data access • Will serve imagery to GIBS
May 5, 2015 ASDC Update for CERES STM Page: 18
Questions?
Top Related