HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability...
-
Upload
anabel-thornton -
Category
Documents
-
view
214 -
download
0
Transcript of HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability...
![Page 1: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/1.jpg)
HDF-EOS Workshop VII 1
HDF-EOS5 Validator Tool
Bob BaneJingli YangRichard Ullman
Data Usability TaskNASA/GSFC
![Page 2: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/2.jpg)
HDF-EOS Workshop VII 2
HDF-EOS5 Validator
Validates HDF-EOS5 files against guidelinesGuidelines are encoded in XMLOutput is a report with specific non-conformance messages
![Page 3: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/3.jpg)
HDF-EOS Workshop VII 3
Why HDF-EOS5 Validator?
Insures that product files: Have all data and metadata items
they are supposed to contain Those items are of correct name,
type, and size
Validator is a tool mainly for data producers to check format of files before delivering them to users.
![Page 4: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/4.jpg)
HDF-EOS Workshop VII 4
HDF-EOS5 Validator
ValidatorHDF-EOS5 file
Guidelines(in XML)
Validatorreport
![Page 5: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/5.jpg)
HDF-EOS Workshop VII 5
XML Description of File Format Guidelines
Like a checklist List of features that must/should be in
file And their characteristics Not all-inclusive
Extra things in file will not be reported
![Page 6: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/6.jpg)
HDF-EOS Workshop VII 6
What can be validated?
Swaths Every swath Swaths with a specific name Fields
Data Every/specific name
Geolocation Every/specific name
Dimensions Name, size
![Page 7: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/7.jpg)
HDF-EOS Workshop VII 7
What can be validated (cont)?
Attributes Global/Swath/Field Type
String Value/Set/regular expression
Integer/Float Value/Range
![Page 8: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/8.jpg)
HDF-EOS Workshop VII 8
Examples: StringValue
<StringValue/> Any string
<StringValue matches=“seconds *”/> “seconds” followed by spaces
<StringValue><OneOf><s>foo</s><s>bar</s></OneOf></StringValue> “foo” or “bar”
![Page 9: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/9.jpg)
HDF-EOS Workshop VII 9
Examples: IntValue
<IntValue/> Any integer value
<IntValue min=“17” max=“42”/> Integer between 17 and 42 inclusive
<IntValue> min=“0”/> Non-negative integer
![Page 10: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/10.jpg)
HDF-EOS Workshop VII 10
Example: Dimension
<Dimension name=“Latitude”/> name is required
<Dimension name=“Height” size=“100”/> size is fixed
<Dimension name=“X” min=“5” max=“10”/> Size can be in range 5-10
![Page 11: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/11.jpg)
HDF-EOS Workshop VII 11
Examples: Attributes<Attributes> <Mandatory> <Attribute name=“InstrumentName”> <StringValue/> </Attribute> </Mandatory> <Optional> <Attribute name=“FieldCount”> <IntValue min=“0” max=“10”/> </Attribute> </Optional></Attributes>
![Page 12: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/12.jpg)
HDF-EOS Workshop VII 12
Overall File Structure
<HDF-EOS-Validation-Definition> <Attributes>… <Swaths> <Every-Swath> <Attributes>… <Swath-Dimensions> <Mandatory-Dimensions> <Dimension … /> <Optional-Dimensions> <Dimension … />
![Page 13: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/13.jpg)
HDF-EOS Workshop VII 13
Overall File Structure (cont.)
<GeolocationFields> <Every-GeolocationField> <DataType … /> <Dimensions> <d>Lat</d><d>Lon</d> <Attributes>… <GeolocationField-Named name=“LatLon”> <DataType … /> <Dimensions> … <Attributes> …
![Page 14: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/14.jpg)
HDF-EOS Workshop VII 14
Implementation
Command lineCHDF-EOS5 libraryOpen source XML parsing libraries expat SCEW
Portable to any HDF-EOS5 platform
![Page 15: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/15.jpg)
HDF-EOS Workshop VII 15
Sample Output Report$ validate_hdfeos5 -x HIRDLS-definition.xml -h test.h5
SWATH HIRDLS Missing mandatory dimension: nWavel
SWATH HIRDLS GEOLOCATION_FIELD Time ATTRIBUTE Units string "TAI " doesn't match <StringValue value="seconds"/>
SWATH HIRDLS Missing mandatory geolocation field: Altitude
SWATH HIRDLS Missing mandatory data field: TemperatureNormChiSq
Errors: 4
![Page 16: HDF-EOS Workshop VII1 HDF-EOS5 Validator Tool Bob Bane Jingli Yang Richard Ullman Data Usability Task NASA/GSFC.](https://reader036.fdocuments.net/reader036/viewer/2022082818/56649ef35503460f94c061de/html5/thumbnails/16.jpg)
HDF-EOS Workshop VII 16
StatusSupports needs of Aura (our test case) Swaths (no Grids/Points yet) Prototype descriptions
For HIRDLS, MLS, OMI, TES Based on Aura Data Format Guidelines
document Tested against Aura data samples, and
correctly reported variances from guidelines Goal is for product producers to
write/maintain their own description files