MSI

Image and Video CompressionImage and Video Compression

Edward J. DelpEdward J. DelpVideo and Image Processing Laboratory (VIPER) Video and Image Processing Laboratory (VIPER)

School of Electrical and Computer EngineeringSchool of Electrical and Computer EngineeringPurdue UniversityPurdue University

22

MSI

OverviewOverview

ContributorsContributors• Wojciech SzpankowskiWojciech Szpankowski• Ananth GramaAnanth Grama• Edward DelpEdward Delp

What are the demands on compressionWhat are the demands on compression• New approaches: scalable techniques and New approaches: scalable techniques and

pattern matching approachespattern matching approaches• Error robustness: concealmentError robustness: concealment• SecuritySecurity

33

MSI

Purdue UniversityPurdue University

Purdue has a rich 65 year history in Purdue has a rich 65 year history in video and imagingvideo and imaging

Why do compression?Why do compression?

44

MSI

The “Digital Image” ProblemThe “Digital Image” Problem

A 1024x1024 image has 1,048,576 A 1024x1024 image has 1,048,576 pixels at pixels at • 24 bits/pixel = 25,165,824 bits24 bits/pixel = 25,165,824 bits

A video (NTSC/CCIR 601) A video (NTSC/CCIR 601) • 760x480 = 345,600 pixels 760x480 = 345,600 pixels

• 30 frames/sec = 10,368,000 pixels/sec30 frames/sec = 10,368,000 pixels/sec

• 16 bits/pixel(4:2:2) = 165,888,000 bits/sec16 bits/pixel(4:2:2) = 165,888,000 bits/sec

55

MSI

Digital Video RatesDigital Video Rates

CIF (4:1:1) with 12 bits/pixel CIF (4:1:1) with 12 bits/pixel

31,104,000 bits/sec31,104,000 bits/sec CCIR 601 (4:2:2) with 16 bits/pixel CCIR 601 (4:2:2) with 16 bits/pixel

165,888,000 bits/sec165,888,000 bits/sec HDTV (GA 1920x1080, 4:2:2, 60 HDTV (GA 1920x1080, 4:2:2, 60

frames/sec, Proscan) with 20 bits/pixel frames/sec, Proscan) with 20 bits/pixel

2,488,320,00 bits/sec2,488,320,00 bits/sec

66

MSI

ScalableScalable

Scalable - Scalable -

““Author and compress ONCE Author and compress ONCE decompress on ANY platform feed by decompress on ANY platform feed by ANY data pipe”ANY data pipe”

77

MSI

ScalabilityScalability

Date rate scalabilityDate rate scalability SNR or quality scalabilitySNR or quality scalability Spatial scalabilitySpatial scalability Temporal scalabilityTemporal scalability Computational scalabilityComputational scalability ““Content” scalabilityContent” scalability

88

MSI

Scalable CompressionScalable Compression

ApplicationsApplications• Internet delivery (aid in QoS)Internet delivery (aid in QoS)• Image and video database search - browsingImage and video database search - browsing• Video serversVideo servers• Teleconferencing and telemedicineTeleconferencing and telemedicine• Wireless networksWireless networks• Kodak’s Photo-CDKodak’s Photo-CD• Distributed multimedia documentsDistributed multimedia documents

99

MSI Scalability in JPEGScalability in JPEG• Progressive modeProgressive mode• JPEG 2000JPEG 2000

Scalability in MPEG-2Scalability in MPEG-2• Scalability is layeredScalability is layered

Scalability in MPEG-4Scalability in MPEG-4• LayeredLayered• ““Content”Content”

Scalability: StandardsScalability: Standards

1010

MSI

Embedded CodingEmbedded Coding

Continuously scalableContinuously scalable All compressed data embedded in a All compressed data embedded in a

single bit streamsingle bit stream Embed the important information at the Embed the important information at the

beginning of the bit streambeginning of the bit stream Can truncate at any data rate or Can truncate at any data rate or

decoded qualitydecoded quality

1111

MSI

Scalable CompressionScalable Compression

Two new approachesTwo new approaches• Color Embedded Zero Tree Compression Color Embedded Zero Tree Compression

(CEZW)(CEZW)• Scalable Adaptive Motion Compensation Scalable Adaptive Motion Compensation

Wavelet Compression (SAMCoW)Wavelet Compression (SAMCoW)

1212

MSI

Original

CEZW

JPEG

SPIHT

Scalable Color CompressionScalable Color Compression

1313

MSI

Coding ArtifactsCoding Artifacts

Original

CEZW

JPEGSPIHT

1414

MSI

ComparisonComparison

JPEG 0.25 bits/pixel CEZW 0.25 bits/pixel

1515

MSI

2D-Pattern Matching Compression2D-Pattern Matching Compression

Where does this pattern match in image Where does this pattern match in image or video frame?or video frame?• Central Theme is lossy extension to Central Theme is lossy extension to

Lempel-Ziv algorithmLempel-Ziv algorithm• Strong theoretical underpinningsStrong theoretical underpinnings• Use for both images and videoUse for both images and video• Use for synthetic images and text - fits into Use for synthetic images and text - fits into

MPEG-4MPEG-4

1616

MSI

Pattern Matching CompressionPattern Matching Compression

Pattern Matching

JPEG

1717

MSI

Error Concealment (1)Error Concealment (1)

1818

MSI

Error Concealment (2)Error Concealment (2)

1919

MSI

Security:WatermarkingSecurity:Watermarking

2020

MSI

ViBEViBE

ViBE has four componentsViBE has four components• Scene change detection and identificationScene change detection and identification• Hierarchical shot representationHierarchical shot representation• Pseudo-semantic shot labelingPseudo-semantic shot labeling• Active browsing based on relevance Active browsing based on relevance

feedbackfeedback ViBE provides an extensible framework ViBE provides an extensible framework

2121

MSIZoom in

Zoom out

Zoom in

Zoom out

2222

MSI

Browser InterfaceBrowser Interface

Relevance SetSimilarity Pyramid Control Panel

2323

MSI

Proposed EquipmentProposed Equipment

Encoders/Decoders Encoders/Decoders • Used for populating databases with video Used for populating databases with video

and images using current standardsand images using current standards Networking SystemsNetworking Systems• Used to test new ideas in scalable Used to test new ideas in scalable

compression and pattern matching compression and pattern matching techniquestechniques