NSIDC!DAAC!User!Working!Group!! Technical!Interchange ...

NSIDC DAAC User Working Group

Technical Interchange Meeting 31 http://nsidc.org/nasa-daac-uwg/category/upcoming-meeting/

October 9-‐10, 2013

National Snow and Ice Data Center Boulder, CO

NSIDC DAAC UWG Technical Interchange Meeting October 9-‐10, 2013 2

Contents Meeting Goals ............................................................................................................................. 3

Agenda ......................................................................................................................................... 4

Travel logistics ............................................................................................................................ 6 Hotel .......................................................................................................................................... 6 Transportation from Denver International Airport (DIA) ............................................................ 7 Parking at NSIDC ...................................................................................................................... 8 Travel reimbursement information ............................................................................................ 8

Pre-meeting materials .............................................................................................................. 10

Contract Extraction .................................................................................................................. 49 Executive Summary ................................................................................................................ 49 2 Facing the Future ................................................................................................................. 49

2.1 Vision ............................................................................................................................ 49 2.2 Trends ........................................................................................................................... 50 2.3 Approach ....................................................................................................................... 53 2.4 Summary ....................................................................................................................... 53


Meeting Goals

• Complete the DAAC Data Product Review to ensure that the resources are concentrated on supporting priority Cryosphere data sets.

• Test usability of the first-phase redesigned NSIDC search and identify further improvements.

o Later phases of development will add advanced tools for refining, exploring, and comparing search results.

• Update the UWG and NASA on NSIDC DAAC data management activities and past action item progress. Collect feedback and guidance on proposed data management activities for prioritization in FY2014.

• Update the UWG on ESDIS and NASA vision for DAAC data, management and science.

• Review the development of DAAC/NSIDC applications that increase data usability and discuss potential use cases and opportunities for future development.


Agenda Note: This is a working Agenda – This agenda is NOT final. It will continue to evolve leading up to the meeting. The latest version can be found here: http://nsidc.org/nasa-daac-uwg/2013/04/agenda-uwg-meeting-31/

Wednesday, October 9, 2013

8:30-9:00

Welcome

▪ Coffee ▪ Logistics ▪ Introductions ▪ Agenda & Goals

Donna Scott

9:00-9:05

NSIDC Director Introduction Mark Serreze

9:05-9:15

CIRES Director Introduction Waleed Abdalati

9:15-10:00

DAAC Highlights

▪ Contract update ▪ Major Project Accomplishments

▪ Enterprise Data Base ▪ Search

▪ Data Teams ▪ Major Accomplishments in data

management

Ron Weaver/Amanda Leon

10:00-10:30

ESDIS Update Jeanne Behnke/Steve Bailey

10:30-10:45

Break

10:45-11:15

NASA cryospheric science update Tom Wagner

11:15-12:15

Bringing Value to NSIDC DAAC Data

▪ What is NSIDC doing with data? ▪ NASA GIBS ▪ General discussion on what UWG would find

valuable

TBD Developer/Amanda Leon

12:15-1:00

Catered Lunch

1:00-3:15

Search

▪ Introduction to project and search team ▪ One-on-one usability testing ▪ UWG debrief, general discussion

Amanda Leon/Sarah Tressel

3:15-3:30

Break


3:30-5:00

Product Review

▪ Introduction to team/goals of this session ▪ Breakout group review(s)

Lisa Booker

5:00-6:00

Chair/Coordinator Daily Wrap Up

▪ UWG free time

Donna Scott/Gina Henderson

6:00- Group Dinner at Fate

Thursday, October 10, 2013

8:30-9:00

Coffee

9:00-10:00

Product Review

▪ Complete breakout group review(s) ▪ Final comments ▪ Data sets for data accession (time permitting)

Lisa Booker/Donna Scott

10:00-11:45

Executive Session Gina Henderson

11:45-12:15

Start catered lunch

12:15-1:30

Working catered lunch

▪ Review past action items ▪ Summarize executive session actions

Donna Scott/Gina Henderson

1:30-2:00

Wrap Up/End


Travel logistics This information can also be found here: http://nsidc.org/nasa-daac-uwg/category/upcoming-meeting/

Hotel

Best Western Plus Boulder Inn 770 28th Street, Boulder, CO 80303 303-449-3800 www.BoulderInn.com Rooms not reserved by September 20 have been released from the discount block.

• Group code: call 800-233-8469 and reference code ” UWG @ NSIDC “ Arrival date: Tuesday, October 8, 2013

• Departure date: Thursday, October 10, 2013 • Group (government) rates: $109 for a room with one King bed • Guests will make own reservations and guarantee with individual credit cards. • Check in: 3:00 PM; Check out: 11:00 AM • Complimentary hot breakfast: 6:30 AM to 10:00 AM

NSIDC

FATE Brewing Company

Boulder Inn

Walking route


Transportation from Denver International Airport (DIA)

Rental Car Agencies at DIA Advantage Budget Hertz Alamo Dollar National Avis Enterprise Payless Airport shuttles Green Ride Boulder (Approx. 45 mins, advance reservation required) (303) 997-0238 http://greenrideboulder.com The Super Shuttle Boulder (Approx. 45 mins, Advance reservation required) (303) 444-0808 http://www.supershuttle.com Airport bus service RTD Bus service (1 hour 45 mins) $13 each way. Includes 15-minute walk www.rtd-denver.com Driving directions to Best Western Plus Boulder Inn (40.5 mi) Note: E470 is a toll road. Your rental car company will add the charges to your bill.

• Follow airport signs to Peña Blvd. • Take the ramp onto Peña Blvd. • Take exit 6B toward Boulder/Ft.Collins • Merge onto E-470 N • Continue onto Northwest Pkwy • Take the ramp onto US-36 W • Take the Baseline Rd exit toward CO-93. Stay in middle lane after exit. • Slight left onto the U.S. 36 W ramp to Estes Park • Keep right at the fork, follow signs for Frontage Rd • Slight right toward 28th Street Frontage Rd • Turn left onto 28th Street Frontage Rd • Hotel is one block on the right.


Parking at NSIDC

NSIDC will provide complimentary parking. Please leave yourself a few extra minutes to get your parking permit. When you arrive, park in the blue shaded areas on the map to the right. Permits can be obtained in room 201, the CIRES East Campus Communication Center (a.k.a the Message Center). The permit will then hang on your review mirror.

Travel reimbursement information

US Federal Government Employees Federal employees pay for their own travel expenses. To accommodate this requirement, NSIDC will provide a total cost for these employee’s meals during the meeting (breakfast and lunch), which can be reimbursed to NSIDC. This will be provided to you at the meeting and will not exceed standard per diem rates. University and Canadian UWG members NSIDC will reimburse travel expenses for US University and Canadian UWG members, by way of the Non-employee Reimbursement (NR) form (instructions below). Book flights and a rental car using your home institution’s process. After you complete your trip, follow the instructions below. Please complete this process within 10 days after returning home. Checklist for travel reimbursement process

• Gather all your receipts. • Double check that proof of payment is indicated on hotel and registration invoices. The

State of Colorado requires a zero balance hotel invoice indicating the method of payment. The hotel can fax you an invoice, should you need one.

• Download the NR form available here: http://nsidc.org/nasa-daac-uwg/files/2012/10/NR.xls.

• Fill out the “Payee Information” section, except for the “vendor ID” field (unless you know your CU vendor ID).

• Complete the “Expenses to be Reimbursed” section. • In the “Comments” field, explain any special circumstances, such as sharing a room or

car, not paying for lodging, travel delays etc. If there isn’t enough room on the form, please include a separate page with your receipts.

• Sign and date the form. • Submit the signed form and receipts.

Via email: Scan all documentation, with signatures, into pdfs and send to [email protected]; OR Via standard mail: Send all documentation (tape small receipts to standard-size paper, staple packet once) to:

CIRES Travel 216 UCB Boulder, CO 80309-0216


Per diem details CU-Boulder/CIRES uses federal rates. Incidental expenses are included in the per diem. They may be used for personal telephone calls, miscellaneous tips (bellhops, porters and maids), etc. Receipts are no longer required to reimburse actual costs of eligible meals. When meals are provided at no cost to an employee or included in a registration fee, the maximum allowable rate for the day will be prorated. Meals on travel days are eligible for reimbursement at 75% of the daily meal and incidental rate, regardless of flight time. The daily rate for Boulder, CO is $61.00. See the GSA’s Meals and Incidental Expenses Breakdown for more information.

NSIDC NASA DAAC UWG TIM Meeting 31

Boulder, CO October 9-10, 2013

Pre-Meeting Materials

NSIDC’s Vision and Objectives

Outline

•  NSIDC: §  Vision and Organization

•  DAAC: § New Contract § Metrics § DAAC Work Management Materials

NSIDC’s Vision To be indispensable to both users and science agencies in conveying cryospheric and related data and information to research communities and the broader public.

Our day-to-day mission supports this vision by: §  Knowing and responding to the wants and needs of our

worldwide users and agency sponsors §  Providing the tools and technology to make data access and

usage productive, efficient, and seamless §  Exploiting the synergies of our diverse portfolio to deliver

more value for users and agencies §  Working on the leading edge of data management and

technology

NSIDC: An overview 5

NSIDC’s Strengths In meeting its Vision, NSIDC draws its strength from: •  Collaboration between its data management and research activities that

helps us to better understand data, its applications and shortcomings, and the needs of the community.

•  Its growing portfolio in informatics research aimed at providing more data and information to more people and in better ways.

•  Its innovative contributions to education and outreach (e.g., ASINA). •  A work environment that places high value on responsibility, teamwork,

and ethics within a collaborative group.

The Major Arms of NSIDC

Creating tools for data access and discovery

Managing and distributing

scientific data

Engaging in scientific research

Educating the public about the cryosphere

Supporting data users

CIRES Organizational Chart Vice Chancellor for Research

Dean of the Graduate School

DIRECTOR

Associate Directors:Atmospheric and Climate

DynamicsEnviornmental Chemistry and

BiologySolid Earth Sciences

Cryospheric and Polar Sciences

NOAA Lab Directors (5)

Associate Director for

Administration

Management Support to Director

Council of

Fellows

Member's Council

Executive CommitteeCIRES Rostered

Tenure-Track and Research Faculty

Center Directors:-Center for Limnology-Earth Science & Observation Center-Center for Science & Technology Policy Research-National Snow and Ice Data Center

Associate Director

for Science

NSIDC Organization Chart

Funding Agencies Objectives

DATA MGMT 78%

SCIENCE 22%

NSIDC FY Ending June 30, 2012 Actual Expenses

NASA-DAAC 55%

NASA-OTHER

12%

NOAA 4%

NSF 23%

UNIV/OTHER

6%

NSIDC FY Ending June 30, 2012

NSIDC NASA DAAC Contract

Highlights of what we proposed

New DAAC Contract (June 1, 2013 – May 31, 2018)

•  Our submitted proposal outlines trends and themes that we believe will direct our activities.

•  Abbreviated text of our submission is distributed with this slide deck. What follows is a condensed version of the main points

New Contract – Observed Trends Trend 1: Research and Data The cryosphere is rapidly changing, and scientists’ data needs are also evolving rapidly. While it is impossible to provide exact statements as to the science support that NSIDC will lead, we identify a number of key areas: •  Arctic sea ice trends and environmental impacts. Arctic sea ice extent

and thickness will continue to decline. •  The changing mass balance of ice sheets and glaciers. We expect

continued high interest in understanding dynamic ice sheet dynamics and acceleration of outlet glacier flows. In general, glaciers are losing mass worldwide; some areas, notably the Himalayas, show very complex behavior change.

•  Permafrost and frozen ground. There is growing concern that as the planet continues to warm, thaw of permafrost will lead to release of stored carbon to the atmosphere, leading to a further global temperature.

•  Snow cover. There will be continued activity in the area of snow cover monitoring via passive microwave and optical sensors.

New Contract – Observed Trends Trend 2: Users and Data User trends can be categorized in two ways: user types and workflow patterns. •  User Types. We see a growing trend in the expansion of non-expert

users. •  User Workflow. The increasing amount of data available to users, the

broader base of people interested in cryospheric data, and technological advances have altered the needs and expectations of users in finding and obtaining data.. The process flow of data selection, data processing, and data consumption will be more tightly integrated. Mobile Device Interfaces: . User expectations for content available on mobile phones, tablets, and full-featured browsers differ; these differences point to opportunities to increase our user base as well as offer content targeted to those audiences. Finding and selecting data: Users will need increasingly sophisticated tools that help them target the most appropriate data for their research. Search needs will likely move beyond traditional temporal, spatial, and parameter searches into discovery based on data value thresholds, anomalies, and quality flags. Processing and using data: Users are increasingly looking for data to be delivered in their preferred format, for their preferred region. When doing scientific trend analysis, users commonly need parameter data captured over time for very specific locations. DAAC we can minimize the burden placed on the user; and increase their available time for conducting scientific research.

New Contract – Observed Trends Trend 3: Information Technology and Data In order to keep pace with requirements driven by research and user expectations, it is imperative that we continually assess our IT trajectory. These trends are categorized into the areas of software, hardware, networking and services.

Software. Software development continues to increase in complexity and users continue to request more advanced features, more complex software and better quality. Hardware. Demand-driven computing will be possible through advances in computer virtualization, networking, and data storage that allow us to adapt and deploy resources more effectively. Use of cloud computing and external processing centers such as university-provided supercomputing systems will grow. Networking. Network capacity and speed will continue to grow, although they will continue to be a bottleneck in delivering data to users. Systems and Services. There is increasing focus on commodity services within the realm of information technology.

By gaining efficiencies in infrastructure services, time and resources can be focused on user-specific functionality. By standardizing data formats and geographic grids, and incorporating new solid state storage technologies with minimal latency for data access, it becomes possible to create Analysis as a Service (AaaS) which will enable a suite of analysis tools for on-the-fly reprojection, reformatting, time series analysis, inter-sensor comparison, and other user-requested functions. Using the community of data centers, data providers, data users, and open source providers, new functionality can be collaboratively developed or acquired in much shorter time frames.

New Contract - Approach

The overall message of the trends indicates growing complexity and a need for agility. Engage the Community. To ensure that we remain focused on priorities, it is critical that we continuously reach out to our user community – whether cryospheric researchers or the broader base of non-expert users. Simplify Workflow. We can support users by offering targeted data and tools; by minimizing the effort users must exert to get appropriate data they are freed to focus on their key research activities. Similarly, within our organization we continually strive to improve ongoing operational and developmental processes to include as much automation and consistency as possible. This frees staff to work on new tasks that bring more value to our user community. Use Existing Capabilities. Within NASA’s Earth Science community, there now exists a wealth of available functionality that can be leveraged. Leverage Our University Ties. The University of Colorado encompasses world-class research and supporting infrastructure at all levels. The science resources of CIRES and other university organizations augment our science support through formal and informal collaborations.

New Contract – Financial

•  Period of Performance: June 1, 2013 – May 31, 2018

•  Awarded July 31, 2013, 1 year with 4 one year options – different than past contracts

•  Budget targets set for each yearly option •  Total contract value: $ 41.6 MY

New Contract – What is Different •  Incremental (yearly) funding with more NASA

oversight •  New assigned responsibilities (SMAP, IceSAT-2,

Operation IceBridge •  Greater utilization of ESDIS technologies and

services (e.g. ECS, GIBS, ECHO, URS …) •  Key needs:

§  Expanding our on-site scientist support §  Maintain relevance of data portfolio in a dynamic

cryosphere research world §  IT systems that match user needs/demands

NSIDC NASA DAAC Metrics

Top 10 Data Sets by Unique Hosts based on 2012 FTP Metrics

0

500

1000

1500

2000

2500

2010 2011 2012 2013*

Uniqu

e Ho

sts D

ownloa

ding Data

Near-‐Real-‐Time Daily Polar Gridded Sea Ice Conc. (NSIDC-‐0081)

Sea Ice Conc. from Nimbus-‐7 SMMR and DMSP SSM/I-‐SSMIS (NSIDC-‐0051)

Sea Ice Trends and Climatologies (NSIDC-‐0192)

MODIS/Terra Snow Cover Daily L3 Global 500m Grid (MOD10A1)

Images of AntarcPc Ice Shelves (NSIDC-‐0102)

AMSR-‐E/Aqua Daily L3 Soil Moisture (AE_Land3)

MODIS/Terra Snow Cover 8-‐Day L3 Global 500m Grid (MOD10A2)

Daily EASE-‐Grid Brightness Temperatures (NSIDC-‐0032)

AMSR-‐E/Aqua Daily L3 12.5 km Sea Ice Conc. (AE_SI12)

AMSR-‐E/Aqua L2A Brightness Temperatures (AE_L2A)

*2013 data through 25 September 2013.

Data Set Users (2012)

Near-‐Real-‐Time DMSP SSM/I-‐SSMIS Daily Polar Gridded Sea Ice ConcentraEons hXp://nsidc.org/data/nsidc-‐0081 1396

Sea Ice ConcentraEons from Nimbus-‐7 SMMR and DMSP SSM/I-‐SSMIS Passive Microwave Data hXp://nsidc.org/data/nsidc-‐0051

1181

Sea Ice Trends and Climatologies from SMMR and SSM/I-‐SSMIS hXp://nsidc.org/data/nsidc-‐0192 1166

MODIS/Terra Snow Cover Daily L3 Global 500m Grid hXp://nsidc.org/data/mod10a1 861

Images of AntarcEc Ice Shelves hXp://nsidc.org/data/nsidc-‐0102 853

AMSR-‐E/Aqua Daily L3 Surface Soil Moisture, InterpreEve Parameters, & QC EASE-‐Grids hXp://nsidc.org/data/ae_land3 781

MODIS/Terra Snow Cover 8-‐Day L3 Global 500m Grid hXp://nsidc.org/data/mod10a2 692

DMSP SSM/I-‐SSMIS Pathfinder Daily EASE-‐Grid Brightness Temperatures hXp://nsidc.org/data/nsidc-‐0032 566

AMSR-‐E/Aqua Daily L3 12.5 km Brightness Temperature, Sea Ice ConcentraEon, & Snow Depth Polar Grids hXp://nsidc.org/data/ae_si12

561

AMSR-‐E/Aqua L2A Global Swath SpaEally-‐Resampled Brightness Temperatures hXp://nsidc.org/data/ae_l2a 549

Top 10 Data Sets by Unique Hosts based on 2012 FTP Metrics

BoZom 10 Data Sets by Unique Hosts based on 2012 FTP Metrics

0

5

10

15

20

25

30

35

40

45

50

2010 2011 2012 2013*

DMSP-‐F8 SSM/I Pathfinder Global Level 2 Sea Ice ConcentraPons

Nimbus-‐7 SMMR Derived Monthly Global Snow Cover and Snow Depth

CLPX NCAR Data Analysis and Numerical Modeling

Airborne Cloud Radar (ACR) ReflecPvity, Wakasa Bay, Japan (AMSR-‐E ValidaPon)

GLAS/ICESat L2 AntarcPc and Greenland Ice Sheet AlPmetry Data (HDF5)**

IceBridge GPS/IMU L1B Primary PosiPon and Aetude SoluPon**

BalPc Sea Experiment (BALTEX) Ground-‐Based Radar Polar Volume Data (AMSR-‐E ValidaPon)

CLPX NOAA FSL Rapid Update Cycle 20 km (RUC-‐20) Dataset

GLAS/ICESat L2 Ocean AlPmetry Data (HDF5)**

GLAS/ICESat L2 Sea Ice AlPmetry Data (HDF5)**

*2013 data through 25 September 2013. **These data sets were new in 2012.

Data Set Users (2012)

DMSP-‐F8 SSM/I Pathfinder Global Level 2 Sea Ice ConcentraEons hXp://nsidc.org/data/nsidc-‐0028 0

Nimbus-‐7 SMMR Derived Monthly Global Snow Cover and Snow Depth hXp://nsidc.org/data/nsidc-‐0024 1

CLPX NCAR Data Analysis and Numerical Modeling hXp://nsidc.org/data/nsidc-‐0291 2

Airborne Cloud Radar (ACR) ReflecEvity, Wakasa Bay, Japan (AMSR-‐E ValidaEon) hXp://nsidc.org/data/nsidc-‐0212 2

GLAS/ICESat L2 AntarcEc and Greenland Ice Sheet AlEmetry Data (HDF5)** hXp://nsidc.org/data/glah12 3

IceBridge GPS/IMU L1B Primary PosiEon and Actude SoluEon** hXp://nsidc.org/data/iputg1b 4

BalEc Sea Experiment (BALTEX) Ground-‐Based Radar Polar Volume Data (AMSR-‐E ValidaEon) hXp://nsidc.org/data/nsidc-‐0209

5

CLPX NOAA FSL Rapid Update Cycle 20 km (RUC-‐20) Dataset hXp://nsidc.org/data/nsidc-‐0180 6

GLAS/ICESat L2 Ocean AlEmetry Data (HDF5)** hXp://nsidc.org/data/glah15 6

GLAS/ICESat L2 Sea Ice AlEmetry Data (HDF5)** hXp://nsidc.org/data/glah13 6

BoZom 10 Data Sets by Unique Hosts based on 2012 FTP Metrics

NSIDC NASA DAAC Work Management

DAAC Work Priorities

1.  Data management and user support for Mission data. §  OIB, Passive Microwave, EOS are priority

2.  Development efforts which promote value for cryosphere data users and efficiency of our workflow.

3.  Outreach which educates users and the general public about our data and the cryosphere and providers about the value of sound data management.

4.  Sustain and evolve IT infrastructure in support of our data and services.

Assigned Work Comes From …

Annual Workplan negotiation •  Drives specific work priorities in current and

next year •  Introduces strategic 3-5 year plan Situational modifications driven by •  ESDIS and HQ directives •  Perceptions of user needs from user feedback •  Data set status (data/sensor changes) •  UWG inputs

Managing the Work… •  Using a formalized approach to project planning

and resource management to meet DAAC commitments §  Gantt chart and resource plan §  Current documents available for FY 2013-14

•  NSIDC Center-wide development planning coordination § Monthly meetings to address NSIDC-wide

development priorities, staff skill alignments and resource needs

DAAC Work Categories

•  Six main categories plus Administration

•  Derived from ESDIS DAAC work plan instructions

Mission Support Engineering & Development

Operations User Support

External Coordination

Infrastructure


•  All remotely-sensed and in-situ data

•  Liaison with science and user communities

•  Science Support for data users

•  Data management coordination




Infrastructure


On-Going Missions/Projects •  Operation Icebridge (OIB)

§  http://nsidc.org/data/icebridge/ •  Passive Microwave (SMMR, SSM/I, SSMIS)

§  http://nsidc.org/data/polar_stereo §  http://nsidc.org/data/ease

•  Aqua/AMSR-E §  http://nsidc.org/data/amsre

•  ICESat/GLAS §  http://nsidc.org/data/icesat

•  Aqua & Terra/MODIS §  http://nsidc.org/data/modis

•  MEaSUREs §  http://nsidc.org/data/measures




Infrastructure

New Missions •  SMAP

§  http://smap.jpl.nasa.gov/mission/ •  Aquarius (Soil Moisture)

§  http://aquarius.nasa.gov/ •  ICESat-2

§  http://icesat.gsfc.nasa.gov/icesat2/

A high-level status for these missions/projects will be presented during the meeting.


•  Active in multiple ESDSWG subgroups •  ESIP Federation - leading Data Management

Training; past Executive Committee member; active in multiple groups

•  Participation in ISO 19115 IEEE Standards Committees

•  Coordinate with NSF polar data management •  Members in WDS and CODATA




Infrastructure


•  Guide overall system architecture •  Develop and maintain production code,

applications, and interfaces for science and data management needs

•  Adhere to core NSIDC development values: accountability, flexibility, predictability, and quality




Infrastructure

DAAC Work Categories Mission Support Engineering &

Development



Infrastructure

Completed Project •  Enterprise Database (EDB): Replace

underlying metadata infrastructure which powers: §  Data Search

http://nsidc.org/data/search/data-search.html

§  Catalog pages http://nsidc.org/data/mod10a1.html

§  Metadata http://nsidc.org/api/metadata?id=mod10a1

On-going Projects •  Data Search: Re-design and re-implement

NSIDC Data Search §  Current Search

http://nsidc.org/data/search/data-search.html •  Operation IceBridge Data Portal

§  http://nsidc.org/icebridge/portal/

A high-level status for these projects will be presented during the meeting.


•  First point of user contact •  Educate users on data holdings,

updates and utilization •  Provide feedback loop on user

needs •  Conduct public outreach via

Earth Observatory, Sensing our Planet, EarthData

3 Months of User Contacts




Infrastructure


•  Provide overall systems administration •  Support database administration •  Maintain and upgrade hardware •  Maintain facilities including green data center •  Protect the infrastructure through IT security




Infrastructure


•  Operate all DAAC data systems •  Produce, ingest, archive, and distribute data •  Perform metadata and data management •  Provide production code support




Infrastructure

How We Manage DAAC Work

Mission Support

Director

Deputy DAAC

Manager

DAAC Manager

Engineering & Development



Infrastructure


Mission Support

Director

Deputy DAAC

Manager

DAAC Manager




Infrastructure Managed by Service Groups

IT Manager

Operations

IT Security

System Admin

Science Comm. Manager

User Support

Web

Outreach Metadata

Coord.

Sustaining Eng.

Systems Arch.

Development Manager

User Support Infrastructure Operations


• Operations • Developers

• User Services

• Scientists

• Technical Writers

• Scientists

• Operations • Developers • Scientists

Acquire or

Produce Describe

Distribute Support

Product Team Lead

Managed by Product Teams

Mission Support

Mission Support

Director

Deputy DAAC

Manager

DAAC Manager




Infrastructure


Managed by Agile Teams

Product Owner

Define • Product Owner • Stakeholders

Build • Developers

Release

Evaluate • Users • Stakeholders


Mission Support

Director

Deputy DAAC

Manager

DAAC Manager




Infrastructure

NSIDC DAAC UWG Technical Interchange Meeting October 9-‐10, 2013

49

Contract Extraction

Executive Summary

NSIDC and the SI DAAC have a history of success based on the strong ties between our scientific and data management areas of expertise. These ties have led to our reputation as a trusted data source with high credibility, a position that we carefully foster in order to maintain our role in furthering global understanding of the cryospheric contribution to climate change. Our proposal demonstrates compliance with the requirements of the Statement of Work while showing sensitivity to the evolving needs of the research community that we serve. The rapidly-changing physical environment that is our scientific focus necessitates an approach that can support changes in requirements and priorities. The NSIDC DAAC has positioned itself to be able to respond to these changing goals by establishing an organization with streamlined processes that allow us to re-focus our efforts as indicated. By continuously engaging the broad research community and understanding their evolving needs for data and services, we remain attuned to future directions that may better serve them. We propose to strengthen our focus on the future through addition of postdoctoral research staff in both science and informatics. To successfully fill our role as a NASA data center, we strive to understand the changes that are occurring in the world and how we can take advantage of them to continuously improve our ongoing operations and evolutionary plans. Clearly the nature of data is changing – becoming more diverse, complex, and voluminous – and those changes parallel trends in research, users, and information technology. These changes drive users’ expectations for products, tools, and services. We must thoughtfully analyze the trends and expectations, and balance our resources to address the appropriate priorities and provide the high-quality results that our users expect. Our unique combinations of skills, which balance scientific and theoretical knowledge with practical experience, place the NSIDC DAAC in excellent position to meet the challenges of the next five years.

2 Facing the Future

2.1 Vision NSIDC’s vision is summarized simply: “To be indispensable to both users and science agencies in conveying cryospheric and related data and information to research communities and the broader public.” To meet this vision, NSIDC draws on its strengths in four areas:

• Collaboration between its data management and research activities to better understand data, their applications and shortcomings, and the needs of the community

• Its growing portfolio in informatics research aimed at providing more data and information to more people and in better ways

• Its innovative contributions to education and outreach • A work environment that places high value on responsibility, teamwork, and ethics within

a collaborative group. NSIDC has clarity and focus on this vision, and has established organizational components that support its realization. These create an environment in which the DAAC can thrive and succeed in meeting NASA’s goals for a Snow and Ice Data Center. Partnerships between the DAAC and NSIDC’s other programs leverage our center-wide strengths and provide benefits to all our programs.


50

2.2 Trends To realize our vision, we project needs for the next five years and beyond. We see trends in key areas: research, users, and information technology. These trends are closely intertwined with data on all levels; we focus on integrated approaches to satisfy the growing needs in each area. The acceleration of real-world change drives a corresponding need for agility in all aspects of our work. Trend 1: Research and Data The cryosphere is rapidly changing, and scientists’ data needs are also evolving rapidly. While it is impossible to provide exact statements as to the science support that NSIDC will lead, we identify a number of key areas: Arctic sea ice trends and environmental impacts. Arctic sea ice extent and thickness will continue to decline. It is now widely accepted that the Arctic will lose most or all of its perennial sea ice cover within this century, perhaps as early as 30 years from now. There will be a continued need to monitor and understand the evolution of ice extent and thickness using data from SSMIS, AMSR-E/AMSR-2, MODIS, ICESat-2, IceBridge, and other sources. NASA assets will be increasingly brought to bear to address environmental consequences of sea ice loss, including impacts on Arctic cloud cover, coastal erosion, weather extremes, marine and terrestrial ecosystems and Arctic amplification (the outsized rise in Arctic air temperatures at relatively lower latitudes). Addressing such complex issues will require improvements in data search, discovery and integration across disciplines and across agencies. The changing mass balance of ice sheets and glaciers. It is firmly established that the mass balance of both the Greenland and Antarctic ice sheets has turned negative, with a growing contribution to sea level rise; Greenland shows the largest change. We expect continued high interest in understanding dynamic ice sheet dynamics and acceleration of outlet glacier flows that will require the use of moderate and high-resolution satellite data such as from MODIS and ASTER. Routine observational needs will include monitoring of elevation changes by Operation IceBridge and ICESat-2, and surface melt over the Greenland ice sheet from SSMIS and scatterometry. In general, glaciers are losing mass worldwide; some areas, notably the Himalayas, show very complex behavior change. Data from ASTER, MODIS and other sources will be increasingly utilized to monitor and understand these ongoing changes. Permafrost and frozen ground. Much of the Arctic and subarctic is underlain by permafrost. Permafrost thaw has impacts on infrastructure and terrestrial ecosystems. There is growing concern that as the planet continues to warm, thaw of permafrost will lead to release of stored carbon to the atmosphere, leading to a further global temperature. We anticipate growing interest in monitoring and understanding the freeze/thaw states of soil and of high latitude soil moisture based on SMAP. Snow cover. Snow cover acts as both an indicator and a driver of climate and climate change. In many parts of the world, such as the US West, it is a key resource for agriculture, domestic consumption and recreation. Extreme snowfall events can disrupt transportation and commerce. While analysis of satellite data documents a significant decline in spring snow cover for the Northern Hemisphere, trends in other seasons are small. Some areas are seeing increased snowfall and there is growing evidence that, with a warmer atmosphere holding more water, there may be a shift towards fewer but more extreme snowfall events. There will be continued activity in the area of snow cover monitoring via passive microwave and optical sensors from both satellite and airborne platforms. Accurate satellite retrieval of snow water equivalent has been an elusive goal, and we see increased activity in this area.


51

Trend 2: Users and Data User trends can be categorized in two ways: user types and workflow patterns. User Types. We see a growing trend in the expansion of non-expert users. Climate watchers, policy makers, scientists outside the Earth sciences, and educators want to tap climate change information at the source without the complexity of understanding algorithms, data formats, and traditional GIS software. Modelers are increasingly looking for data that can readily be incorporated into complex climate models. Scientists new to the field, such as graduate students, and interdisciplinary scientists, whose primary field of study does not frequently require the use of remotely sensed data, are also in this category of user. NSIDC has seen a large increase in general-public users wanting to gain access to the data behind some of the headlines they see, particularly sea ice extent record lows. The increased attention also leads educators to design classroom activities around these data. These data users are not as skilled at working with the common formats in which NSIDC data are provided. User Workflow. The increasing amount of data available to users, the broader base of people interested in cryospheric data, and technological advances have altered the needs and expectations of users in finding and obtaining data. What users want and how they anticipate getting it are dramatically shifting -- largely as a result of technology advancements both in the field of research and in the tools that permeate everyday life. The process flow of data selection, data processing, and data consumption will be more tightly integrated to enable researchers to focus more on their scientific work. Mobile Device Interfaces: Use of tablets and mobile phones is growing rapidly, and is likely to surpass use of laptops. User expectations for content available on mobile phones, tablets, and full-featured browsers differ; these differences point to opportunities to increase our user base as well as offer content targeted to those audiences. Mobile devices will play a prominent role in support of field work, enabling researchers to capture metadata and documentation contemporaneously with their data. Finding and selecting data: There are numerous, and still expanding, choices of data sets for some parameters, such as sea ice concentration. Users will need increasingly sophisticated tools that help them target the most appropriate data for their research. Search needs will likely move beyond traditional temporal, spatial, and parameter searches into discovery based on data value thresholds, anomalies, and quality flags. Processing and using data: Users are increasingly looking for data to be delivered in their preferred format, for their preferred region. When doing scientific trend analysis, users commonly need parameter data captured over time for very specific locations. Currently, the user must download the data, extract the parameter and location information, and reformat; by moving processing into the DAAC we can minimize the burden placed on the user; and increase their available time for conducting scientific research. Trend 3: Information Technology and Data As always, information technology is evolving quickly and trends that are apparent now may already be obsolete in five years, replaced with currently unforeseen developments. In order to keep pace with requirements driven by research and user expectations, it is imperative that we continually assess our IT trajectory and make necessary adjustments at key points. Typically,


52

these trends are categorized into the areas of software, hardware, and networking. Since the distinctions are becoming less precise, we include systems as a fourth area of interest. Software. Software development continues to increase in complexity and users continue to request more advanced features, more complex software and better quality. In the future we see a continuing refinement of widely-adopted agile practices with heavy influence from Lean processes and an expansion to the entire enterprise. Agile development practices are being scaled and adopted broadly at NSIDC, both upward into the management of projects with new practices in portfolio management and outward into all of IT and the rest of the organization. This broadening focus and customization of agile processes will improve the flexibility, quality, and productivity of the entire organization. Web service-oriented architecture, a variant of Service-Oriented Architectures (SOA), is a critical foundation for scaling and reusing software. It provides a solution for easy language-agnostic code reuse as well as providing advantages for managing a complex network of software. Web services provide for better and easier reuse, both internally and externally. The complexity of solutions that are in work and requested for the future requires this modular, reusable architecture. Hypermedia, the linking of web services for real time dynamic discovery of features is a developing trend in web service development. Hypermedia makes it possible to navigate a complex set of web services programmatically, discovering which services are available on specific data and enabling the client to dynamically present functionality when appropriate. Due to the growing complexity of requirements, there is increasing emphasis on development collaboration in addition to reuse. Web services provide the foundation for applications to be built upon standard services deployed at multiple organizations or by multiple projects under the NSIDC umbrella, and give advanced users the ability to build their own solutions to meet specific needs. Open source and shared source software development will continue to grow, diffusing the burden of code maintenance among a broader community. Hardware. Demand-driven computing will be possible through advances in computer virtualization, networking, and data storage that allow us to adapt and deploy resources more effectively. Use of cloud computing and external processing centers such as university-provided supercomputing systems will grow, enabling significant data processing (such as on-the-fly data analysis tools, subsetting, and mapping) at the source and reducing network demand. Scale-out network attached storage technologies will provide the capabilities of a distributed networked file system without the current architectural complexities, thereby reducing the burden of storage management while providing the flexibility of cost-effective tiered data storage. Networking. Network capacity and speed will continue to grow, although they will continue to be a bottleneck in delivering data to users. This will necessitate the continuing development of tools and processes which minimize the volume of delivered data to well-refined subsets matching the users’ specific needs. It also points to expansion of higher-level data products and less reliance on high-volume lower-level data. Internally, high-speed (10 Gb) connectivity will be available between applications and data without the overhead and expense of a fiber channel infrastructure. Systems and Services. There is increasing focus on commodity services within the realm of information technology. Hardware virtualization has grown to a commoditized resource that can be purchased by the hour. Services that a traditional IT infrastructure maintained in-house (such as storage, backups, email, and even operating systems) are increasingly being pushed


53

outward to companies that can best implement economy-of-scale solutions. These hardware and software infrastructures, referred to as Platform as a Service (PaaS), are growing as replicable, disposable commodities. By gaining efficiencies in infrastructure services, time and resources can be focused on user-specific functionality. By standardizing data formats and geographic grids, and incorporating new solid state storage technologies with minimal latency for data access, it becomes possible to create Analysis as a Service (AaaS) which will enable a suite of analysis tools for on-the-fly reprojection, reformatting, time series analysis, inter-sensor comparison, and other user-requested functions. Using the community of data centers, data providers, data users, and open source providers, new functionality can be collaboratively developed or acquired in much shorter time frames.

2.3 Approach The overall message of the trends indicates growing complexity and a need for agility. The NSIDC DAAC is now well-positioned to face these challenges, thrive in this environment, and meet the needs of our users. Broadly, we use several themes to guide our work toward successful results: Engage the Community. To ensure that we remain focused on priorities, it is critical that we continuously reach out to our user community – whether cryospheric researchers or the broader base of non-expert users. Through the User Working Group, the IceBridge Science Working Group, NSIDC’s researchers, and the myriad of professional contacts that we maintain, the DAAC keeps abreast of evolving user needs. Given our broad diversity of data, input from a wide range of sources gives us a view to the necessary breadth of services we must provide. Simplify Workflow. In a time of limited resources, it is incumbent on us all to seek efficiencies wherever possible. We can support users in reaching this goal by offering targeted data and tools; by minimizing the effort users must exert to get appropriate data they are freed to focus on their key research activities. Similarly, within our organization we continually strive to improve ongoing operational and developmental processes to include as much automation and consistency as possible. This frees staff to work on new tasks that bring more value to our user community. Use Existing Capabilities. Within NASA’s Earth Science community, there now exists a wealth of available functionality that can be leveraged. We will continue to use ECHO as much as possible; its comprehensive metadata store provides access to the breadth of data that the community will increasingly require. Through collaborative development and reuse of software services based on standard interfaces, we will be able to avoid duplicative development while providing added-value functionality to users. Leverage Our University Ties. The University of Colorado encompasses world-class research and supporting infrastructure at all levels. The science resources of CIRES and other university organizations augment our science support through formal and informal collaborations. The University’s Office of Information Technology provides resources that would not otherwise be available to us, such as access to supercomputing systems.

2.4 Summary Clearly, the DAAC in 2018 will look very different from the DAAC in 2013. The research communities’ drive to understand rapidly changing Earth systems and processes gives us the mandate to continue to develop tools and exploit technology as fully as possible to satisfy their


54

growing needs. New NASA missions strengthen that mandate due to the unprecedented volume and complexity of data that we must successfully manage.

With a clear vision toward the future and an organization that is already fully-functioning, the NSIDC DAAC is ready to meet the challenges of the next five years. The following sections describe the specific tasks that we plan to meet the requirements of the Statement of Work, and the approaches that we will use to manage our evolving portfolio.

NSIDC!DAAC!User!Working!Group!! Technical!Interchange ...

Documents

Transcript of NSIDC!DAAC!User!Working!Group!! Technical!Interchange ...