Robert Wolfe

NASA Goddard Space Flight CenterCode 614.5, Greenbelt, MDrobert.e.wolfe@nasa.gov

Robert Wolfe

NASA Goddard Space Flight CenterCode 614.5, Greenbelt, MDrobert.e.wolfe@nasa.gov

MODIS (and VIIRS Land) Current and Future

use of HDF

MODIS (and VIIRS Land) Current and Future

use of HDF

Page Page 22

• 3 Nadir Spatial Resolutions– 250m (1,2), 500m (3-7), and 1km (8-36)

• 4 Focal Plane Assemblies (FPAs)– VIS, NIR, SMIR, and LWIR

• 36 Spectral Bands– Reflective solar bands (1-19, and 26),

thermal emissive bands (20-25, 27-36)

• On-Board Calibrators (OBCs):– Solar diffuser (SD)– SD stability monitor (SDSM)– Blackbody (BB)– Spectro-radiometric calibration

assembly (SRCA)– Space view (SV)

• Science Applications– Land, oceans, and atmosphere– Nearly 40 science products generated

and distributed

Terra (EOS-AM)Terra (EOS-AM)Launched Dec.1999Launched Dec.1999First light Feb. 2000First light Feb. 2000

Aqua (EOS-PM)Aqua (EOS-PM)Launched May 2002 Launched May 2002 First light June 2002First light June 2002

Terra and Aqua MODISTerra and Aqua MODIS

Wolfe - Nov 09 HDF Workshop 3

Current MODIS HDF UsageCurrent MODIS HDF Usage

• MODIS Level 1 (L1), Atmosphere and Land product formats are– HDF4– HDF4-EOS structural metadata (swath and grid)– ECS core metadata (ODL string)– Mostly SDSs and Attributes (some VDATA)– Internal compression– Geolocation stored in swath products both internally (reduced resolution)

and externally (full resolution)

• MODIS Oceans (SeaWiFS heritage) product formats are– HDF4– “ECS core equivalent” metadata stored as attributes (matches SeaWifs

format)– Mostly SDSs and Attributes (some VDATA for L3 intermediate products)– External compression (i.e. gzip)

3

Wolfe - Nov 09 HDF Workshop 4

MODIS HDF4 CommentaryMODIS HDF4 Commentary

Commentary – my views, not official MODIS views

• “Simple” HDF Attributes should have been used for both HDF-EOS structural metadata and ECS core metadataIf attributes had been used then:– It would have been easier for users to read metadata fields using

the standard HDF library (without SDP library)– It would have made it easier for Science Team developers to

read/write metadata

• Some lessons learned for Terra (and Aqua) were applied to the Aura mission data formats

4

Wolfe - Nov 09 HDF Workshop 5

Planned MODIS HDF UsagePlanned MODIS HDF Usage

• MODIS will (likely) provide products in HDF5 in 2011 timeframeL1, Land, Atmosphere:– Initially, HDF5 will be offered as one format available as a Post-processing

option (i.e. like GeoTIFF) for MODIS Collection 6 products• MODIS Collection 6 reprocessing will start early 2010 and be completed by end of 2010

– HDF5-EOS structural metadata– ECS core metadata fields are likely to be stored as simple HDF5 attributes– Internal compression will be used– Later, PGEs (science software) I/O may be rewritten to read/write HDF5

• Oceans– Switch to HDF5 will be driven by user community

• HDF5 will be used for Aquarius (produced by SeaWiFS group)

– SeaDAS user tool will be modified to read/write HDF5 as well as HDF4

5

VIIRS Land BandsVIIRS Land Bands

VIS NIR SWIR MWIR LWIR

VIIRS Visible, NIR and SWIR BandsVIIRS Visible, NIR and SWIR Bands

• bbb– ccc

H2OH2O

H2OH2OH2OH2O

H2OH2O

O2O2

CO2CO2

CO2CO2

H2OH2O

O2O2

O3O3

H2OH2O

O2O2

Wolfe - Nov 09 HDF Workshop 8

Current VIIRS Land PEATE HDF UsageCurrent VIIRS Land PEATE HDF Usage

• VIIRS Land PEATE converts NOAA IDPS provided products from HDF5 to HDF4 to be compatible with MODIS– Aggregates data to L1/L2 data to ~5 min. granules– Simple HDF attributes used for metadata

• All VIIRS Science Data Record (SDR, L1) and EDR Environmental Data Record (EDR, L2/3) software have been converted to run with HDF4 data– I/O wrapper is used in algorithms– DDR (Diagnostic Data Record, L2/3) software reads/writes HDF4

• NASA Direct Broadcast (DB) Laboratory is modifying I/O wrapper to read/write standard IDPS HDF5 products to support DB community

8

Wolfe - Nov 09 HDF Workshop 9

VIIRS HDF5 CommentaryVIIRS HDF5 Commentary

Commentary – my views, not official Land PEATE views

• HDF5 implementation by IDPS has several major flaws– Complex data structures were developed to support aggregation by

time and by products– Simple HDF5 features (dimensions, units, etc.) are not being used;

instead these fields are stored as HDF attributes– Un-aggregated granules are much shorter than 5 minutes (~1

minute) so they are many (many) files to deal with– For SDRs, each band is stored separately, unless specifically

ordered as band aggregates by users

9

Wolfe - Nov 09 HDF Workshop 10

Planned VIIRS Land PEATE HDF UsagePlanned VIIRS Land PEATE HDF Usage

Note: The VIIRS Land PEATE is funded to help the NASA science team to evaluate the VIIRS SDR and Land EDR products, not to distribute them (or the DDRs) to the general science community

• There will likely be a “convergence” MODIS/VIIRS SDR and Land EDR HDF5 product formats– A “rational” HDF5-EOS format will be used (see earlier MODIS Land

HDF5 discussion)– MODIS HDF4/5 converter will be modified to also work with VIIRS– Likely to be available in the early VIIRS post-launch timeframe (mid-

2011)– This will only be available to the general science community if NASA

policy changes

10

Wolfe - Nov 09 HDF Workshop 11

Planned VIIRS Ocean PEATE HDF UsagePlanned VIIRS Ocean PEATE HDF Usage

Note: The VIIRS Ocean PEATE is funded to help the NASA science team to evaluate the VIIRS SDR and Land EDR products, not to distribute them (or the DDRs) to the general science community

• If the NASA policy changes, there will likely be a “convergence” MODIS/VIIRS Ocean EDR HDF5 product formats produced by the VIIRS Ocean PEATE (see earlier MODIS Ocean HDF5 discussion)

11

Wolfe - Nov 09 HDF Workshop 12

MODIS/AquaMODIS/AquaHaze over China and KoreaHaze over China and KoreaOct. 28, 2009Oct. 28, 2009

Thank you!Thank you!