Bormin Huang1, Alok Ahuja1, Yagneswaran Sriraja1, Hung-Lung Huang1, Mitchell D. Goldberg2, Timothy J. Schmit2, and Roger W. Heymann2

1Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin-Madison, 2NOAA, National Environmental Satellite, Data, and Information Service

LOSSLESS COMPRESSION STUDIES OF GRATING SPECTROMETER DATA FOR NOAA GOES-R HYPERSPECTRAL ENVIRONMENTAL SUITE

To support GOES-R data compression studies for the possible grating-type HES sounder, we investigate various 2D and 3D lossless compression methods using the grating-type Atmospheric Infrared Sounder (AIRS) data.

These data compression techniques are applicable for GOES-R rebroadcast

We show that an average lossless compression ratio of 3.89average lossless compression ratio of 3.89 is achievable for NASA EOS AIRS ultraspectral sounder data

1. INTRODUCTION1. INTRODUCTION 6. TOWARDS HARDWARE IMPLEMENTATION FOR 6. TOWARDS HARDWARE IMPLEMENTATION FOR REAL-TIME SATELLITE REBROADCASTREAL-TIME SATELLITE REBROADCAST

3D Wavelet decomposition for ultraspectral sounder data

Granule 9 00:53:31 UTC -12 H (Pacific Ocean, Daytime)

Granule 16 01:35:31 UTC +2 H (Europe, Nighttime)

Granule 60 05:59:31 UTC +7 H (Asia, Daytime)

Granule 82 08:11:31 UTC -5 H (North America, Nighttime)

Granule 120 11:59:31 UTC -10 H (Antarctica, Nighttime)

Granule 126 12:35:31 UTC -0 H (Africa, Daytime)

Granule 129 12:53:31 UTC -2 H (Arctic, Daytime)

Granule 151 15:05:31 UTC +11 H (Australia, Nighttime)

Granule 182 18:11:31 UTC +8 H (Asia, Nighttime)

Granule 193 19:17:31 UTC -7 H (North America, Daytime)

10 selected AIRS digital counts granules on March 2, 2004

5. TOWARDS ERROR RESILIENCE IN SATELLITE NOISY TRANSMISSION5. TOWARDS ERROR RESILIENCE IN SATELLITE NOISY TRANSMISSION

7. SUMMARY7. SUMMARY

2. ULTRASPECTRAL DATA FOR LOSSLESS COMPRESSION STUDIES2. ULTRASPECTRAL DATA FOR LOSSLESS COMPRESSION STUDIES

Average number of erroneous pixels after 3D JPEG2000 (Part 2) and 3DWT-RVLC decoding due to 1 bit random error added in different wavelet resolutions for 4 granules

• Each granule consists of 135 scan lines containing 90 cross-track footprints per scan line.• Publicly available via anonymous ftp (ftp://ftp.ssec.wisc.edu/pub/bormin/Count)

AIRS digital counts at wavenumber 800.01cm-1 for the 10 selected granules (with the 1502-channel subset)

3. CIMSS-DEVELOPED DATA PREPROCESSING SCHEME3. CIMSS-DEVELOPED DATA PREPROCESSING SCHEMECIMSS’s Bias-Adjusted Reordering (BAR) data preprocessing scheme (Huang et al. 2004) improves the performance of state-of-the-art compression methods (2D CALIC, 2D JPEG-LS, 2D JPEG2000 (Part 1), 3D JPEG2000 (Part 2))

Texas Instruments’ TMS320C6416 DSP board

CIMSS’s BAR data preprocessing scheme significantly improves the compression ratios of such state-of-the-art compression methods as 2D JPEG2000 (Part 1), 3D JPEG2000 (Part 2), 2D CALIC, and 2D JPEG-LS

After applying the BAR preprocessing scheme, the standard state-of-the-art compression methods perform almost equally well !!

New compression methods (Lossless PCA, PPVQ, FPVQ) show better results on ultraspectral sounder data than standard compression methods, achieving an average lossless compression ratio of 3.89average lossless compression ratio of 3.89

Newly developed 3DWT-RVLC method provides significantly better error resilience than 3D JPEG2000 (Part 2)

Acknowledgement: This work is prepared in support of National Oceanic & Atmospheric Administration (NOAA) GOES-R data compression research under grant NA07EC0676.

Compression ratios of 3DWT-RVLC vs. the DSP version of 3DWT-RVLC

41R

64R

63R

62R

61R

11R

11R

14R

24R3

4R44R5

4R12R

13R

31R

21R

L2SRAM

EXTERNAL SDRAM

ULTRASPECTRALSOUNDER DATA

DSP MEMORY

1 2

3 4

5 6

7 8

2

Error contamination after JPEG2000 decodingError contamination after JPEG2000 decoding

Error-Correcting Channel Coding StudiesError-Correcting Channel Coding Studies

Error-Resilient Source Coding Study: Error-Resilient Source Coding Study: 3D Wavelet Reversible Variable-length Coding 3D Wavelet Reversible Variable-length Coding (3DWT-RVLC, (3DWT-RVLC, Huang et al. 2005Huang et al. 2005) has significantly better error resilience than 3D JPEG2000 (Part 2)) has significantly better error resilience than 3D JPEG2000 (Part 2)

Location of erroneous pixels after 2D JPEG2000 source decoding for 1-bit header error after channel decoding

Bit error rate vs. signal-to-noise ratio after LDPC channel decoding of a JPEG2000 compressed granule

A memory-limited DSP version of 3DWT-RVLC is implemented to explore A memory-limited DSP version of 3DWT-RVLC is implemented to explore feasibility of 3DWT-RVLC for feasibility of 3DWT-RVLC for real-time satellite rebroadcast processingreal-time satellite rebroadcast processing

TMS320C6416 two-level cache-based architecture

LM1000 PCI board carrier card Block data transfer to the DSP memory

Each granule is divided into 8 blocks to feed into the external memory on the DSP board

Granule 9 16 60 82 120 126 129 151 182 193 Average3DWT-RVLC 2.53 2.60 2.40 2.67 2.52 2.40 2.70 2.46 2.41 2.39 2.51

DSP 3DWT-RVLC 2.37 2.44 2.28 2.52 2.37 2.28 2.52 2.32 2.27 2.27 2.36

Spatial scenes for 1 bit-error corruption in lowest wavelet resolution for 3D JPEG2000 (Part 2) and 3DWT-RVLC compressed granule 182

Location of erroneous pixels after 2D JPEG2000 source decoding for 1-bit nonheader error after channel decoding

Digital Video Broadcasting Second Generation Low Density Parity Check (DVB-S2 LDPC) code from Comtech AHA

It consists of BCH outer code + LDPC inner code

We investigated code rates of 9/10 and 8/9 for JPEG2000 compressed ultraspectral sounder data

• Lossless PCA (Huang et al. 2004)

• Predictive Partitioned Vector Quantization (PPVQ) (Huang et al. 2004)

• Fast Precomputed Vector Quantization (FPVQ) with optimal bit allocation (Huang et al. 2005)

4. CIMSS-DEVELOPED NEW LOSSLESS COMPRESSION METHODS4. CIMSS-DEVELOPED NEW LOSSLESS COMPRESSION METHODS

2.05

2.25

2.45

2.65

2.85

3.05

3.25

3.45

9 16 60 82 120 126 129 151 182 193Granule

Com

press

ion

Rati

o .

2D JPEG2000

BAR + 2D JPEG2000

3D JPEG2000

BAR + 3D JPEG2000

2D JPEG-LS

BAR + 2D JPEG-LS

2D CALIC

BAR + 2D CALIC

3.7

3.8

3.9

4

9 16 60 82 120 126 129 151 182 193

Granule

Com

pre

ssio

n R

atio

.

PPVQ

FPVQ

Lossless PCA

Future work will include more DSP implementations of other compression methods