In the name of Allah - ce.sharif.educe.sharif.edu/courses/85-86/1/ce823/resources/root/lecture...
Transcript of In the name of Allah - ce.sharif.educe.sharif.edu/courses/85-86/1/ce823/resources/root/lecture...
In the name of Allah
the compassionate, the merciful
Digital Image ProcessingDigital Image Processing
S. KasaeiS. KasaeiSharif University of Technology
Room: CE 307 E-Mail: [email protected]
Home Page: http://ce.sharif.eduhttp://ipl.ce.sharif.eduhttp://sharif.edu/~skasaei
Kasaei 4
Course SyllabusCourse SyllabusLecture: Sundays & Tuesdays, 13:30-15:00, Room Kh. 7.
Website:
http://ce.sharif.edu/courses/85-86/1/ce823/Check this site often for important announcements, files needed for computer exercises, and the PDF versions of handouts & homework.
Course Description: 40-823+ provides an introduction to image processing theory and techniques.
Kasaei 5
Course SyllabusCourse SyllabusTopics include: 2-D system theory, image perception, image sampling and quantization, image transforms, image representation, image enhancement, image filtering and restoration, image analysis, and image/video compression.
Prerequisites: 40-763 (Digital Signal Processing) or40-933 (Digital Image Processing)
Text Book:Fundamentals of Digital Image Processing, by Anil K. Jain, Prentice Hall, 1989.Chapters 1, 2, 3, 4, 5, 7, & 9 will be wholly covered, the rest will be partially covered.
Kasaei 6
Course SyllabusCourse SyllabusAdditional topics will be included (e.g., video compression, wavelet transform, texture analysis, watershed, snakes…).
Written Homework Problems:Written homework problems will be assigned by the end of each chapter.
Computer Exercises:Computer exercises will also be assigned over the course. They can be provided either in Matlab or C.
Term Project:There will be a term project, which can be proposed by the student. Students are supposed to present the final result, associated with related software & technical report.
Kasaei 7
Course SyllabusCourse SyllabusExams:There will be one midterm and one final exam.
Grading Policy:Written & computer assignment: 3 pts.Project: 3 pts.Project Report: 1 pts.Project Presentation: 1 pts.Midterm exam: 3 pts. (hold at: 1385.8.30)Final exam: 9 pts. (hold at: 1385.10.20, 14:30)Submitted paper: 2 extra pts.
Project Topic Confirmation Due:1385.10.20
Kasaei 8
Course SyllabusCourse Syllabus
Instructor Office Hour:Sundays, 17:00-18:00, Room CE 307.
Teaching Assistants:Ms. M. Hassanzadeh & Mr. A.A. Darabi
Kasaei 9
Chapter 1:Chapter 1:
Introduction to Image Processing
Kasaei 10
IntroductionIntroduction
Kasaei 11
IntroductionIntroduction
Kasaei 12
IntroductionIntroduction
Kasaei 13
IntroductionIntroduction
Kasaei 14
IntroductionIntroductionImage:any 2-D function that bears information.
Digital Image:an array of real or complex numbers represented by a finitenumber of bits.
Digital Image Processing:digital processing of any 2-D data.
Applications:remote sensing via satellites, image transmission & storage, medical processing, radar, sonar, robotics.
Kasaei 15
Basic SubjectsBasic SubjectsImage representation & modeling,
Image transform,
Image enhancement,
Image restoration,
Image analysis,
Image data compression.
Kasaei 16
Image Representation & ModelingImage Representation & ModelingAn image can present:
luminance of objects in a scene (picture),
absorption characteristics of body tissue (X-ray),
radar cross section of a target (radar imaging),
temperature profile of a region (infrared imaging),
gravitation field in an area (geophysical imaging).
Kasaei 17
Image Representation & ModelingImage Representation & Modeling
Infrared image. Angiography image.
Kasaei 18
Image Representation & ModelingImage Representation & Modeling
X-ray images.
Kasaei 19
Image Representation & ModelingImage Representation & Modeling
UV images.
Kasaei 20
Image Representation & ModelingImage Representation & Modeling
CT image.
Kasaei 21
Image Representation & ModelingImage Representation & Modeling
Multispectral Geostationary Operational Environment Satellite image.
Kasaei 22
Image Representation & ModelingImage Representation & Modeling
Scanning Electron Microscope images.
Kasaei 23
Image Representation & ModelingImage Representation & Modeling
Computer Generated Images.
Kasaei 24
Image Representation & ModelingImage Representation & ModelingFundamental requirement of digital processing: sampling & quantization.
The sampling rate (# pixels/unit area) has to be large enough to preserve useful information.
Quantization is an A/D conversion of a sampled image into a finite number of gray levels.
e.g., raster scanned common TV signal bandwidth: 4 MHzminimum sampling rate: 8 MHzframe pixels for 30 frames/sec: 8x10m/30=266,000image size for 512-line raster: 512x512.
Kasaei 25
Image Representation & ModelingImage Representation & ModelingA classical method of signal representation is by an orthogonal series expansion, such as Fourier series.
For images, analogous representation is possible via 2-D orthogonal function called basis images.
For sampled images, the basis images can be determined from unitary matrices called image transforms.
Any given image can be expressed as a weighted sum of basis images.
Statistical models describe an image as a member of an ensemble.
Kasaei 26
Image Representation & ModelingImage Representation & Modeling
Kasaei 27
Image FormatsImage Formats
Kasaei 28
Switching Between FormatsSwitching Between Formats
Kasaei 29
Choosing a ThresholdChoosing a Threshold
Kasaei 30
Using MasksUsing Masks
Kasaei 31
Image EnhancementImage EnhancementImage enhancement goal is to accentuate certain image features for subsequent analysis or display.
Enhancement process itself does not increase the inherent information content in the data.
It is usually the first main step of every image processing task whose performance efficiently affects the algorithm.
It is interactive, application-dependent, image-dependent, and its performance criteria is subjective (perceptual).
Examples include: contrast & edge enhancement, pseudocoloring, noise filtering, sharpening, & magnifying.
Kasaei 32
Image EnhancementImage Enhancement
Kasaei 33
Transformed HistogramTransformed Histogram
Kasaei 34
Histogram EqualizationHistogram Equalization
Kasaei 35
Geometric TransformsGeometric Transforms
Kasaei 36
NearestNearest--Neighbor vs. BilinearNeighbor vs. Bilinear
Kasaei 37
Bilinear InterpolationBilinear Interpolation
Kasaei 38
Resizing ImagesResizing Images
Kasaei 39
Rotating ImagesRotating Images
Kasaei 40
Median FilteringMedian Filtering
Kasaei 41
Image RestorationImage RestorationRefers to removal or minimization of known degradations in images.
A typical problem is to find an estimate of image, given the point spread function, the degraded image, and statistical properties of the noise.
Examples include deblurring due to sensor limitations or its environment, noise filtering, blotch & scratch removal, correction of geometric distortion.
Wiener filter gives the best linear MSE of the object from the observations.
Kasaei 42
Image RestorationImage Restoration
Kasaei 43(a) Original Image. (b) Blotch detection. (c) Motion vectors. (d) Blotch removal. (e) Scratch removal
Image RestorationImage Restoration
(a) (b) (c)
(d) (e)
Kasaei 44
(a) Original Image. (b) Image with 64x64 zeros in the SDFT domain. (c) Image with a lost block of size 64x64 pixels. (d) Restored image, PSNR = 21.26 dB.
Image RestorationImage Restoration
(a) (b) (c) (d)
Kasaei 45
Image AnalysisImage AnalysisIt is concerned with making quantitative measurementsfrom an image to produce a description of it.e.g., reading a label on a grocery item,
sorting different parts on an assembly line,measuring quantitative information to make a decision.
It requires extraction of certain features that aid in identification of the object.
Segmentation techniques are used to isolate the desired object from the scene so that measurements can be made on it subsequently.
Kasaei 46
Image WatermarkingImage Watermarking
Original Image
WatermarkSignal
WatermarkedImage
Kasaei 47
ContentContent--Based Image RetrievalBased Image Retrieval
Query Image
Kasaei 48
ContentContent--Based Image RetrievalBased Image Retrieval
Two images with similar color histogram.
Kasaei 49
Proposed MethodProposed Method
Our Method. Gabor Wavelet.
Kasaei 50
Proposed MethodProposed Method
Kasaei 51
Proposed MethodProposed Method
After imposing Zernike.
Kasaei 52
Camera parameter estimation using hardware or software solution
Video out Removing of unwanted background (using chroma keyer)
Camera parameters
Virtual scenes generator
MixerSensors
Image
Virtual StudiosVirtual Studios
Kasaei 53Marker-based systems.
Virtual StudiosVirtual Studios
Kasaei 54Orad blue screen pattern.
Virtual StudiosVirtual Studios
Kasaei 55
Proposed Virtual StudioProposed Virtual Studio
Kasaei 56
Proposed Virtual StudioProposed Virtual Studio
Kasaei 57
Fingerprint Face Hand Iris
Retina Signature Voice Facial Thermograms
Biometric TechniquesBiometric Techniques
Kasaei 58
Optical sensor. Different silicon sensors.
Thermal silicon sensor.
Biometric TechniquesBiometric Techniques
Kasaei 59Examples of poor FP images.
Biometric TechniquesBiometric Techniques
Kasaei 60
Two different FP images of the same individual.
Biometric TechniquesBiometric Techniques
Kasaei 61
Biometric TechniquesBiometric Techniques
Commercial FP authentication systems.
Kasaei 62
Proposed Authentication TechniqueProposed Authentication Technique
OriginalImage
Block Directions
SegmentedImage
ExtractedFeatures
Kasaei 63Two photos of the same individuals.
Biometric TechniquesBiometric Techniques
Kasaei 64Facial recognition device and method.
Biometric TechniquesBiometric Techniques
Kasaei 65
Proposed Face Detection TechniqueProposed Face Detection Technique
Face detection results.
Kasaei 66
Facial recognition systems.
Biometric TechniquesBiometric Techniques
Kasaei 67
Security monitoring.
Biometric TechniquesBiometric Techniques
Kasaei 68
Measures the unique pattern of the iris-colored portion of theeye, to identify individuals.
Iris sensors.
Biometric TechniquesBiometric Techniques
Kasaei 69
Retinal sensor.
Biometric TechniquesBiometric Techniques
Kasaei 70Comparison of biometric technologies.
Biometric TechniquesBiometric Techniques
Kasaei 71Comparison of biometric technologies.
Biometric TechniquesBiometric Techniques
Kasaei 72
Image Data CompressionImage Data CompressionAmount of data associated with visual information is very large.
Transmission & storage of raw data is a enormous task.
Typical TV images generate data rates exceeding 10 million bps.
Data compression techniques aim at reduction of interpixel, psychovisual, coding, temporal, & spectral redundancies.
They fall into two main categories: lossless & lossly.
Applications include: broadcast TV, remote sensing, computer communications, teleconferencing, facsimile transmission.
Image storage is required for education & business documents, medical images, and large databases.
Kasaei 73
Image Data CompressionImage Data Compression
Kasaei 74
Image Data CompressionImage Data Compression
Compression performance.
Kasaei 75
Image/Video Compression StandardsImage/Video Compression Standards
Kasaei 76
Image/Video Compression StandardsImage/Video Compression Standards
Kasaei 77
JPEG at 0.125 bpp(192:1)
JPEG2000 at 0.125 bpp(192:1)
JPEG JPEG vs. JPEG2000vs. JPEG2000
Kasaei 78
Proposed Image CompressionProposed Image Compression
OriginalImages
ReconstructedImages
CR: 66.5:1bpp: 0.120PSNR: 38.1
CR: 56:1bpp: 0.143PSNR: 23.98
Kasaei 79
ObjectObject--Based Video CompressionBased Video Compression
Decomposition of Scene:– Object segmentation process is the most difficult task!– This stage is not standardized yet.– Each object is specified by its shape, motion, & texture.– Both shape & texture change.
Examples of video object planes (VOP)s:
Kasaei 80
ObjectObject--Based Video CompressionBased Video Compression
Video Object Plane (VOP) Extraction:– A fully adaptive, noise robust, & fast method.
• Stationary Background Assumed.• Object Extraction Techniques:
– Global Object Extraction.– Moving Object Extraction.– Still Object Extraction.
• Noise Removal:– Discrete Wavelet Transform:
• Decreasing Computational Cost.• Decreasing Noise Effect.• Spatial Scalability.
– Adaptive Spatial & Temporal Noise Cancellation.
Kasaei 81
Proposed Coding AlgorithmProposed Coding AlgorithmContent-Based Coding:– Background Coding:
• EZW Coding Scheme.– Moving VOP Coding:
• Shape Coding.• Texture & Motion Coding.
– Still VOP Coding:• Smooth Changing Objects (scripts):
– DPCM with no motion compensation.• Burst Changing Objects (slides/presentations):
– Slides & Presentations:– EZW Coding.
– Multiplexer:• Based on MPLS protocol.• High priority objects use the maximum bandwidth.
Kasaei 82
Object SegmentationObject Segmentation
( ) ( ) ( )yxBgyxFyxObj ii ,,, −=
Kasaei 83
Object SegmentationObject Segmentation
Difference Noise:– Gaussian Noise:– Due to intensity changes.– Can be reduced by a
Hypothesis test.Hypothesis test: – Noise if H0 True.– Object if H1 True.
Post-Processing:– Morphological Opening.
( )
( )( )( )( )objx
objx
dif
i
dif
i
dif
cameradif
thyxDPHthyxDPH
yxObjD
dDif
>=
≤=
=
−⋅=
=
∑
,
,
,
2exp
2
1
2
1
0
2
2
2
2
2
σ
σπσ
σσ
χ
Kasaei 84
Change Detection Masks (Change Detection Masks (CDMsCDMs))
Kasaei 85
Noise ReductionNoise Reduction
Using spatial distance.
Using temporal information.
Kasaei 86
Object ExtractionObject Extraction
Kasaei 87
Multiplexer Script ModeMultiplexer Script Mode
Multiplexer:• Is applied to assign maximum bit-rate to
higher priority bit-streams.
Script Mode:• As scripts change rarely & are DPCM coded,
their size is far less than the moving VOPsScripts have higher Priority.
• Assign maximum bandwidth to still images when ready.
• Status Change Detection.
Kasaei 88
Multiplexer Slide ModeMultiplexer Slide Mode
Slide Mode:• Slides are a burst of information.• The entire still image changes in a very few frames.• The size of compressed still images is more than
Intra-VOP.• Moving VOPs have higher priority.• Assign Maximum bandwidth to moving VOPs when
ready.• Status Change Detection.
In both modes:• the higher priority bit-streams is transmitted between
the lower priority bit-streams.
Kasaei 89
Experimental ResultsExperimental ResultsB
ackg
roun
d
10 fps, 80dB
Sta
ndar
d JP
G
10 fps, 99dB
Pro
pose
d E
ZW
56.61.28
1280.56
2560.28
3840.18
5120.14
99 dB7 KB
BandwidthKbps
Transmission TimeSec.
PSNRSize
Kasaei 90
Experimental ResultsExperimental Results
10 fps, ~48 dB
10 fps, ~37 dB
Pro
pose
d In
tra
Cod
ing
Intra
-Fra
me
MP
EG
-456.60.11
1280.05
2560.025
3840.0167
5120.0125
49.04 dB<1KB
Bandwidth(Kbps)
Transmission Time(Sec.)
PSNRSize
The EndThe End