HDF OPeNDAP update
description
Transcript of HDF OPeNDAP update
![Page 1: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/1.jpg)
www.hdfgroup.org
The HDF Group
1ESIP Summer Meeting
HDF OPeNDAP update
Kent YangThe HDF Group
July 8 – 11, 2014
![Page 2: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/2.jpg)
www.hdfgroup.orgESIP Summer Meeting 2July 8 – 11, 2014
Why OPeNDAP?
• Check metadata remotely (in various forms)• Obtain the subset of data easily and efficiently• Hide the original data sources
• netCDF, HDF4, HDF5 or GeoTiFF, GRIB • Many popular earth science tools can visualize
and analyze the data via OPeNDAP• OPeNDAP output(including subsets) can be
downloaded as other formats
![Page 3: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/3.jpg)
www.hdfgroup.orgESIP Summer Meeting 3July 8 – 11, 2014
• The visualization of HDF(5) data by OPeNDAP• What users request the most
What makes HDF OPeNDAP support special?
![Page 4: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/4.jpg)
www.hdfgroup.orgESIP Summer Meeting 4
HDF Handlers
July 8 – 11, 2014
Access HDF via OPeNDAP
Users
Clients Servers HDF Files
AIRS/OMI
DAP
HDF Handlers(CF)
Aqua/Aura
![Page 5: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/5.jpg)
www.hdfgroup.orgESIP Summer Meeting 5
HDF4 handler update
• Add TRMM 7 support• level 1 and level 2 swath • Level 3 grid
• Improve the performance to handle AIRS version 6 grid and MOD08_M3 products
• Improve the performance to handle MODIS products with no-CF scale and offset rules
• Reduce the number of file open/close calls when using file netCDF module to access HDF data
• Map HDF4 char array to DAP String
July 8 – 11, 2014
![Page 6: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/6.jpg)
www.hdfgroup.orgESIP Summer Meeting 6
MOD08_M3 and AIRS version 6 Performance
• Disable the generation of StructMetadata as a DAP attribute for the CF option
• Use the special features in these products to efficiently build DDS and DAS
July 8 – 11, 2014
![Page 7: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/7.jpg)
www.hdfgroup.orgESIP Summer Meeting 7
TRMM 7 Demo – swath(level 1B)
July 8 – 11, 2014
![Page 8: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/8.jpg)
www.hdfgroup.orgESIP Summer Meeting 8
TRMM 7 Demo – swath(level 2A25)
July 8 – 11, 2014
![Page 9: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/9.jpg)
www.hdfgroup.orgESIP Summer Meeting 9
TRMM 7 Demo multi-grid(level 3A25)
July 8 – 11, 2014
![Page 10: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/10.jpg)
www.hdfgroup.orgESIP Summer Meeting 10
TRMM 7 Demo grid(level 3B43)
July 8 – 11, 2014
![Page 11: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/11.jpg)
www.hdfgroup.orgESIP Summer Meeting 11
HDF5 handler update
• Support GPM level-1 products• Make the products follow CF
• Reduce the number of file open/close calls when using file netCDF module to access HDF data
July 8 – 11, 2014
![Page 12: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/12.jpg)
www.hdfgroup.orgESIP Summer Meeting 12
GPM HDF5 structure
July 8 – 11, 2014
![Page 13: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/13.jpg)
www.hdfgroup.orgESIP Summer Meeting 13
HDF5 handler demo: GPM level 1A
July 8 – 11, 2014
![Page 14: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/14.jpg)
www.hdfgroup.orgESIP Summer Meeting 14
HDF5 handler demo: GPM level 1B
July 8 – 11, 2014
![Page 15: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/15.jpg)
www.hdfgroup.orgESIP Summer Meeting 15
Hyrax file-netCDF module
• Can work with HDF handlers to convert HDF files to netCDF-3 or netCDF-4 classic files that follow the CF conventions
• We also help debug and provide fixes for this module to help NASA
July 8 – 11, 2014
![Page 16: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/16.jpg)
www.hdfgroup.orgESIP Summer Meeting 16
File netCDF module demo
• Can use besstandalone program• End users can install Hyrax and use this program to
convert HDF4 and HDF5 files to netCDF3 or netCDF4
![Page 17: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/17.jpg)
www.hdfgroup.orgESIP Summer Meeting 17July 8 – 11, 2014
More challenges
• CF conventions evolve and tools evolve• New versions of existing HDF products and new
HDF products may require significant updates• The data aggregation service requires decent
performance of the data service per file
![Page 18: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/18.jpg)
www.hdfgroup.orgESIP Summer Meeting 18
Future - Performance
• Reduce the DDS and DAS access time • Provide a caching mechanism in the handler level
July 8 – 11, 2014
![Page 19: HDF OPeNDAP update](https://reader035.fdocuments.net/reader035/viewer/2022062519/56814e46550346895dbbc612/html5/thumbnails/19.jpg)
www.hdfgroup.org
The HDF Group
19ESIP Summer Meeting
Earth Science GroupTed Habermann
Aleksandar JelenakH. Joe Lee
Joel PlutchakJohn Readey
Kent Yang
July 8 – 11, 2014