CS148: Introduction to Computer Graphics and Imaging...

CS148: Introduction to Computer Graphics and Imaging

Compression

CS148 Lecture 16 Pat Hanrahan, Winter 2009

Key Concepts

Lossless vs. lossy compressionKolmolgorev complexityPredictive coding / Huffman codingJPEG / Discrete cosine transform (DCT)JPEG2000 / Wavelets


Image and Video Data Rates

Image 640x480x24b = ~3/4 MB

Full screen Image 1024x768x24b = ~2.5MB

DVD 720x480x24bx30f/s = ~30 MB/s

High Definition DVD 1920x1080x24bx30f/s = ~178MB/s

Film 4000x3000x36bx30f/s = ~1.5GB/s 8 TB for one 90 minute movie!


Lossless vs. Lossy Compression

Lossless All information stored Exact original can be reconstructed

Lossy Some information discarded Goal: discard information humans won’t

notice Much higher compression ratios possible


Kolmogorov Complexity

What is the shortest program that can generate the data?

17 KB JPEG

Re(c), Im(c), xmin, xmax, ymin, ymax (24B)

z0 = x + iy

zn+1 = z2n + c


Run Length Encoding

BWBBBBBBBBBBBBWWWWWWBBBW

BW\{12}B\{6}W\{3}BW


Code Book

Alphabet: A, B, C, DFrequencies: ¼, ¼, ¼, ¼ Code: 00, 01, 10, 11

ABCDBDAC = 8 characters00 01 10 11 01 11 00 11 = 16 bits

¼ * 2 + ¼ * 2 + ¼ * 2 + ¼ * 2 = 2 bits/ch on average


Code Book

Alphabet: A, B, C, DFrequencies: ½, ¼, 1/8, 1/8Code: 0, 10, 110, 111

ABACADAB = 8 characters0 10 0 110 0 111 0 10 = 14 bits

½ * 1 + ¼ * 2 +1/8 * 3 +1/8 * 3 = 1 ¾ bits/ch on average


Entropy


Huffman CodingA

(.10)

B (.15)

C(.30)

D(.16)

E(.29)


Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

0 1


Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

0 1

ABD(.41)

0 1


Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

0 1

ABD(.41)

0 1 0 1

CE(.59)


Huffman Coding

A (.10)

B (.15)

C(.30)

D(.16)

E(.29)

AB (.25)

0 1

ABD(.41)

0 1 0 1

CE(.59)

0 1

ABCDE(1.0)


Lossy Compression

Chroma subsamplingTransform Coding

Fourier / DCT (JPEG) Wavelets (JPEG2000)


Chroma Subsampling

+


Transform Coding

X = 2 2 2 2 3 4 5 6 6 6 6 H(X) = 2.0049

D(X) = 2 0 0 0 1 1 1 1 0 0 0 H(D(X)) = 1.3222

Difference Operator


Bases

Basis vectors b0, b1, … , bn

Express any vector as

a0 * b0 + a1 * b1 + … + an * bn

where the coefficients ai are scalars.


Pixel Basis


Another Basis


Two Observations on Images

Characterize images by the frequencies that are present

The frequencies in natural images fall off as 1/f That is, high frequencies are less common

The human visual system is less sensitive to high frequencies

That is, it is more important to preserve low frequencies than high frequencies


Discrete Cosine Transform (DCT)


DCT


DCT

262,144 pixels 43384 largest terms, 16%(dropped 218,760 terms)


DCT

262,144 pixels 8353 largest terms, 3.2%(dropped 253,791 terms)


Quantization

/

DCT Image Quantization Matrix

Quantized DCT Image

=


Storage Order


Error

error = abs(original – compressed) * 8


Peak Signal-to-Noise Ratio

� Original Image (I) and new image (I’)

MSE =1N

N!

i=0

!I(i)" I !(i)!2

RMSE =#

MSE

PSNR = 10 log10

"I2max

MSE

#

= 20 log10

"Imax

RMSE

#

!p" q!2 =

!"""#

"""$

(avg(pi)" avg(qi))2

(max(pi)"max(qi))2

(lum(p)" lum(q))2

· · ·


Error Metrics


Wavelets


Haar Wavelets

Scaling function = average

Wavelet = difference

Transform:S = (A + B)/2D = (A – B)/2

Invert:A = S + DB = S – D


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

1 Transform Step

Averages• Smoothed version of signal• Lower resolution image

Differences• Local, high frequencies• Details missing from low resolution part


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

-1 -2 0 0


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 -1 -2 0 05.5


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -.5 -1 -2 0 05.5


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -1 -2 0 0

0 -1 -2 0 0

-.5

-.5

5.5


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 5.5 0 -1 -2 0 0

6.25 0 -1 -2 0 0

-.5

-.5


Haar Wavelets

6 8 5 9 5 5 6 6

7 7 5 6 -1 -2 0 0

7 0 -1 -2 0 0

6.25.75 0 -1 -2 0 0

-.5

-.5

5.5


Haar Wavelets

6 8 5 9 5 5 6 6

6.25 0 -1 -2 0 0

Full Transform

High Res DetailsMedium Res DetailsLow Res DetailAverage Value

-.5.75


2D Wavelet Transform

Standard– Apply full transform horizontally, then full

transform vertically– Creates long, thin basis functions – bad for

image compressionNon-standard

– Repeatedly apply 1 step horizontally, then 1 step vertically

– Creates square basis functions – good for image compression


What else needs compression?

VideoTextures (different requirements than plain

images)Geometry (including animations)Anything else?


Things to Remember

LosslessNo error, but not lots of compression

LossyError can be driven by human perceptionMore error, but lots of compressionDifferent error metrics (max, avg, perceptual)

Different Basis FunctionsJPEG / Discrete cosine transform (DCT)JPEG2000 / Wavelets

CS148: Introduction to Computer Graphics and Imaging...

Documents

Transcript of CS148: Introduction to Computer Graphics and Imaging...