Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria,...
Transcript of Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria,...
![Page 1: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/1.jpg)
TOC 1/110
Digital image processing
Aline RoumyInria, Rennes, team Sirocco
![Page 2: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/2.jpg)
The course: you and me!
• Who am I? Researcher at Inria, Rennes.
Research topic:communication of visual data (image and video)
Research areas:image processing and signal processing,information theory.
• Who are you?
TOC 2/110
![Page 3: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/3.jpg)
Course organization
• introduction to image processing: 3 lectures of 1:10 each
• When Quiz in the slides, answer on Socrative
https://b.socrative.com/login/student/Room Name: ALINER
No need to give your true name (no marks for the answer)• 2 computer lab sessions: 3:30 each in matlab
I general exercises on image processing (seen in the course)I one project
• evaluation: during the computer lab session(s). You will getmarks for the two sessions + multiple choice questions test.
TOC 3/110
![Page 4: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/4.jpg)
Part 1 - General introduction
What is image processing?
TOC 4/110
![Page 5: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/5.jpg)
...a part of Computer Vision (vision par ordinateur)
• What it is?Technology to design machines that can see.
• First level: visionacquisition of an image thanks to a chain that containsoptical devices and sensors
• Second level: image processingmodify the image in order to highlight some elements ofinterest (objects, edges)
• Third level: interpretation, decision makinguse artificial intelligence (AI) to do
I pattern recognition (reconnaissance): identify some shapes inthe image (face, coin, ...)
I image restoration (noise removal, object removal...)I 3D scene reconstruction
TOC 5/110
![Page 6: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/6.jpg)
First level: acquisition
digitalize = sample (finite number of points)+ quantize (finite number of possible gray levels)
Source: T. Hassner. Computer VisionTOC 6/110
![Page 7: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/7.jpg)
Second level: image processingDetect “simple” elements of an image
digitalize = sample (finite number of points)TOC 7/110
![Page 8: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/8.jpg)
Computer vision: a way to extend human perception?
• Yes!I because the human visual system is limited
- in frequencies (wavelength),- in angle,- in acquisition frequencies (number of images per second):high speed camera (camera haute cadence) with thousandsimages per second...
• and No! Computer vision systems are still limited...- a machine is less intelligent than our brain... but the battlekeeps going: (Google’s AI Wins Against Go Genius, March2016)- limited computational capacity of a machine vs our brain- we use much more information (not only vision) to makedecision
TOC 8/110
![Page 9: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/9.jpg)
Application of Computer Vision (vision par ordinateur):Machine Vision (vision industrielle)
• Use:automatic inspection, (quality inspection)process control,and robot guidance
• Works well because...camera designed for a specific taskcontrolled environment
TOC 9/110
![Page 10: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/10.jpg)
Application of Computer Vision (vision par ordinateur):natural images
Very challenging (still under research) because Uncontrolledenvironment:
• Acquire an imageCorrect color balance,
Reconstruct image from projections
• Prepare for display or printingAdjust image size, Color mapping
• Facilitate picture storage and transmissionEfficiently store an image in a digital camera,
Send an image from mobile
• Enhance and restore imagesTouch up personal photos
• Extract information from imagesRead 2-d bar codes, Character recognition
• Many more ... image processing is ubiquitousTOC 10/110
![Page 11: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/11.jpg)
Application
Face detection
TOC 11/110
![Page 12: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/12.jpg)
Application
Face blurring for privacy protection
TOC 12/110
![Page 13: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/13.jpg)
Application
Object removal (Movie production)
TOC 13/110
![Page 14: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/14.jpg)
Application
Super Resolutiona.k.a. UpscalingSeiki’s U-Vision Upscaling HDMI Cable (CES 14)
input upsizing super-resolution
TOC 14/110
![Page 15: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/15.jpg)
What is an image?
• Digital image: 2D array I [x , y ]
• Each element is called a pixel or pel (from “picture element”)• Gray levels - 1 byte = 8 bits ∈ [0, 255],
I 0 - blackI 255 - white
TOC 15/110
![Page 16: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/16.jpg)
What is an image?a row of an image: is a curve
the whole image: a surface
TOC 16/110
![Page 17: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/17.jpg)
Images as matrices
• Digital image: is a 2D array (or matrix) with Nrows rowsand Ncolumns columns with elements I [i , j ] ∈ {0, . . . , 255}
TP In MATLAB, an image is a matrix “I” with elementsI (i , j) ∈ {0, . . . , 255}: () NOT [ ]for i = 1, . . . ,Nrows , j = 1, . . . ,Ncolumns.
TP In Matlab, I (:, i) denotes the i-th column of I.
TP Warning: type of the 2D array (uint8 or double)To display an image stored as uint8 imshow(I)To display an image stored as double imshow(I/255)Look at the type of your matrix in the Matlab’s workspacewindow → see Matlab
TOC 17/110
![Page 18: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/18.jpg)
What is an image?
Reading and displaying order of I [x , y ] is:as a matrix, not as the cartesian coordinate plane
x = row index / y = column indexBUT the definition is Nb columns × Nb rows (convention)
TOC 18/110
![Page 19: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/19.jpg)
Some types of images
Gray level imageI [x , y ] ∈ [0, 255]
Gray level imageI [x , y ] ∈ {0, 1}
Color imageIR [x , y ]IG [x , y ]IB [x , y ]
and many more types (3D, High Dynamic Range (HDR)...)
Quiz 1,2,3
TOC 19/110
![Page 20: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/20.jpg)
Color Components
but there exists other representations for color images (LuminanceChrominance)
TOC 20/110
![Page 21: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/21.jpg)
Outline of the coursePart 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
![Page 22: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/22.jpg)
Part 2 - Image transformation
How to formalize an image transformationsuch that it can be automatically processed by a computer?
TOC 22/110
![Page 23: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/23.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 23/110
![Page 24: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/24.jpg)
Image transformationWhat is an image?• from the mathematical perspective:
I an image is a matrixI many tools exist to manipulate matrices
• from the human perspective:I an image contains semantic informationI need to interpret the image beyond the luminance values
Transformation:T
T : [0, 255]Nrows×Ncolumns → [0, 255]Nrows′×Ncolumns′
im[x , y ] 7→ IM[u, v ]• im= input or original image• IM= transformed image• x , y and u, v are the spatial coordinates of a pixel
Goal of a transformation:get a a new representation of the input pictureto ease the extraction of particular properties of the picture.
TOC 24/110
![Page 25: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/25.jpg)
3 kinds of transformation
There exist 3 types of transformation:
• Point to point transformation: Theoutput value at a specific coordinatedepends only on one input value but notnecessarily at the same coordinate;
• Local to point transformation: Theoutput value at a specific coordinatedepends on the input values in theneighborhood of that same coordinate;
• Global to point transformation: Theoutput value at a specific coordinatedepends on all the values in the inputimage.
Note that the complexity increases with the size of the consideredneighborhood...
TOC 25/110
![Page 26: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/26.jpg)
Part 3 - Point-to-point transformation
When a pixel of the new (transformed) image only depends ona single pixel of the input image
TOC 26/110
![Page 27: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/27.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 27/110
![Page 28: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/28.jpg)
Geometric transformation
• Geometric transformation such as rotation, scaling...
• Examples:
original a. b.
Quiz 4,5,6
TOC 28/110
![Page 29: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/29.jpg)
Contrast enhancement/reduction
Images with different contrast
low contrast high contrast
TOC 29/110
![Page 30: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/30.jpg)
Contrast definition
Many definitions for the contrast for a NxM image f [x , y ]:
• standard deviation (average variation) of the Gray levels
C =
√√√√ 1
N M
N∑x=1
M∑y=1
(f [x , y ]−mean)2
• variation between the minimum and the maximum of the Graylevels
C =maxx ,y
(f [x , y ]
)−minx ,y
(f [x , y ]
)maxx ,y
(f [x , y ]
)+ minx ,y
(f [x , y ]
)
TOC 30/110
![Page 31: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/31.jpg)
Functions applied to the luminance
• I: input luminance value I’: output luminance value
Quiz 7
TOC 31/110
![Page 32: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/32.jpg)
Gray level histograms
zebra image
• the histogram represents the “distribution” of the gray level inan image
• H(k) = #pixels having gray level k .
TOC 32/110
![Page 33: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/33.jpg)
Gray level histograms
• To measure a histogram:I For B-bit image, initialize 2B counters with 0I Loop over all pixels x , yI When encountering gray level f [x , y ] = i , increment counter
#i
• Normalized histogram can be thought of as an estimate of theprobability distribution (pdf) of the continuous signalamplitude
• Use fewer, larger bins to trade off amplitude resolution againstsample size.
Quiz 8, 9, 10
TOC 33/110
![Page 34: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/34.jpg)
Histogram adjustment... with linear transform
Goal: Augment the contrast (i.e. augment the difference betweentwo displayed gray levels).
Idea: Find an affine transformation
g = F (f )
that is applied to each pixel of the input image f [x , y ],such that the distribution of gray levels for the output imageg [x , y ] spans the whole range [0, 255].
F : [min,max ] → [0, 255]
x 7→ F (x) = ax + b
where x stands for the gray level.i.e. find a and b.
Quiz 11
TOC 34/110
![Page 35: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/35.jpg)
Histogram adjustement... with linear transform andsaturation
TOC 35/110
![Page 36: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/36.jpg)
Histogram equalization
Idea: Find a non-linear transformation
g = T (f )
that is applied to each pixel of the input image f [x , y ], such that auniform distribution of gray levels results for the output imageg [x , y ].
Use the cumulative distribution function (cdf) (Fonction derepartition).
Quiz 12
TOC 36/110
![Page 37: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/37.jpg)
Histogram equalization example
#pixels vs gray level
zebra image
TOC 37/110
![Page 38: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/38.jpg)
Histogram equalization example
Original image zebra ... after histogram equalization
TOC 38/110
![Page 39: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/39.jpg)
Quantization... if time permits
The quantization is a process to represent a large set of valueswith a smaller set.Scalar quantization:
Q : X → C = {yi}i=1,2,...,N
x 7→ Q(x)
• N is the number of quantization level;
• X can be continuous (ex: R) or discrete;
• C is always discrete (codebook,dictionary);
• if X discrete, card(X) > card(C);
• Since card(X) > card(C), Q is non injective, we loose someinformation (lossy compression).
Quiz 13
TOC 39/110
![Page 40: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/40.jpg)
Uniform scalar Quantization (if time permits)
In uniform scalar quantization, the quantization step size is fixed,no matter the signal amplitude is.
Definition:
• The quantization thresholds areuniformly spread:∀i ∈ {1, ...N}, ti+1 − ti = ∆
• The output values are the centers ofthe quantization interval:∀i ∈ {1, ...N}, yi = ti+1+ti
2
Implementation:
Q(x) = ∆× b x
∆+ 0.5c
TOC 40/110
![Page 41: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/41.jpg)
Uniform scalar Quantization
Application to images:
quality resolution (resolution tonale)
Quiz 14 15
TOC 41/110
![Page 42: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/42.jpg)
Binarization: by Gray-level thresholding
Original imagePeter f [x , y ]
ThresholdedPeter t[x , y ]
Mask1− t[x , y ]
Thresholding: Loop over all pixels x , y
• if f [x , y ] > threshold, t[x , y ] = 1
• else t[x , y ] = 0.
Mask: logical imageregion of interest=1rest=0f.*mask=foreground
How can holes be filled?
TOC 42/110
![Page 43: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/43.jpg)
Binarization: to extract foreground
Original imagePeter f [x , y ]
Mask1− t[x , y ]
Peter foregroundf [x , y ].(1− t[x , y ])
Thresholding: Loop over all pixels x , y
• if f [x , y ] > threshold, t[x , y ] = 1
• else t[x , y ] = 0.
Mask: logical imageregion of interest=1rest=0f.*mask=foreground
Question: How can holes be filled?
TOC 43/110
![Page 44: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/44.jpg)
How can holes be filled?
1. By adjusting the threshold
There exists automatic threshold determination.In TP, we will do it manually.
2. Decision based on a single pixel is not sufficientneed decision based on neighborhood:local to point transformation
TOC 44/110
![Page 45: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/45.jpg)
Part 4 - Local to point transformation
When a pixel of the new (transformed) image depends on alimited set of pixels of the input image
TOC 45/110
![Page 46: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/46.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 46/110
![Page 47: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/47.jpg)
Morphological image processing for binary images
• Binary images are common
I Source image is binary:Text, line graphics
I Features extracted from a gray-scale/color imageEdges;Presence/absence of some object in the image
• Representation of individual pixels as 0 or 1, convention:object (foreground) = 1 (white)background = 0 (black)
• Morphological image processing has been generalized togray-level images via level sets
TOC 47/110
![Page 48: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/48.jpg)
Structuring element• Structuring element = a mask
3× 3 square cross
• For each pixel (i , j), apply the mask/structuring elementcentered in (i , j)
mask
image 8x8
For ease of visualization (unlike convention):background 0 black → white/ object 1 white → blue
TOC 48/110
![Page 49: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/49.jpg)
Erosion
Erosion if one pixel under the mask belongs to the background(white), then the centered pixel becomes background (white)
i.e. erosion of the object
• Erosion of the object!
TOC 49/110
![Page 50: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/50.jpg)
Erosion
Quiz 16 Which a or b is the eroded image? The mask is
image 8x8 a. eroded image? b. eroded image?
TOC 50/110
![Page 51: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/51.jpg)
Dilation
Dilation if one pixel under the mask belongs to the object (blue),then the centered pixel becomes object
• Dilation of the object!
• Duality: erosion is dilation of the background
TOC 51/110
![Page 52: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/52.jpg)
Dilation
Quiz 17 Which (a or b) is the dilated image? The mask is
image 8x8 a. dilated image? b. dilated image?
TOC 52/110
![Page 53: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/53.jpg)
Application: Morphological edge detectors
f[x,y]object
dilate(f)-fa.
f-erode(f)b.
dilate(f)-erode(f)c.
Quiz 18,19, 20
TOC 53/110
![Page 54: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/54.jpg)
Morphological operators
• Erosion if one pixel under the mask belongs to thebackground, then the centered pixel becomes background
i.e. erosion of the object
• Dilation if one pixel under the mask belongs to the object,then the centered pixel becomes object
i.e. dilation of the object
• Opening erosion and then dilationGOAL: remove noise (small elements)
• Closing dilation and then erosionGOAL: fill in holes in the object or make connex 2 objectsthat were separated due to noise
TOC 54/110
![Page 55: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/55.jpg)
Application: Denoising
f[x,y] binarized opening
Source: Gonzalez and Woods. Digital Image Processing.
TOC 55/110
![Page 56: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/56.jpg)
Application: Recognition with closing
Source: Gonzalez and Woods. Digital Image Processing.
TOC 56/110
![Page 57: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/57.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 57/110
![Page 58: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/58.jpg)
Filtering
• Filtering is a local to point transformation: the new pixelvalue depends on the current but also neighboring pixels
• For each pixel, we apply a mask.
• If the filter is linear, then, for each applied mask, the centeredpixel becomes a linear (more precisely convex) combination ofthe pixel values in the mask.
input image filter output image
TOC 58/110
![Page 59: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/59.jpg)
Image filtering: the steps
filter h[u,v]
Image I[i,j] Image I[i,j] .* filter h[u,v]
J is the output of filtering of I by h:
J[2, 2] = a× I [1, 1] + b × I [1, 2] + c × I [1, 3]
+ d × I [2, 1] + e × I [2, 2] + f × I [2, 3]
+ g × I [3, 1] + h × I [3, 2] + k × I [3, 3]
TOC 59/110
![Page 60: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/60.jpg)
Image filtering: the steps
filter h[u,v]
Image I[i,j] Image I[i,j] .* filter h[u,v]
J[i , j ] = a× I [i − 1, j − 1] +b × I [i − 1, j ] +c × I [i − 1, j + 1]+ d × I [i , j − 1] +e × I [i , j ] +f × I [i , j + 1]+ g × I [i + 1, j − 1] +h × I [i + 1, j ] +k × I [i + 1, j + 1]
TOC 60/110
![Page 61: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/61.jpg)
Filtering: the equations
• A compact way to write the equations of 2D linear filtering is:2D discrete convolution.
• Reminder: 1D discrete convolution
g [x ] = f [x ] ∗ h[x ] =∑k
h[x − k] f [k] =∑k
h[k] f [x − k]
I f: inputI g: outputI h: filter, convolution kernel
• a linear filter: is the convolution of f by the filter h i.e. it isthe linear combination of the neighboring pixel values
TOC 61/110
![Page 62: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/62.jpg)
Image filtering: the equations
J[x , y ] = I [x , y ] ∗ h[x , y ] =∑u,v
h[x − u, y − v ] I [u, v ]
=∑
u,v∈{−1,0,1}2h[u, v ] I [x − u, y − v ]
Example: Gaussian filter
I[x,y] J[x,y]
TOC 62/110
![Page 63: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/63.jpg)
Image filtering: border effect and normalization
Border effect: solutions:
• Set the borders of J to 0
• Convolution of part of themask, where I exists
• mirror of the image I [i , j ]
Normalization:to visualize the filtered image, oneneeds to normalize the output by di-viding by the sum of the filter coeffi-cients.
TOC 63/110
![Page 64: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/64.jpg)
Filtering what? some frequency components in the image
• What is a frequency in an image?
To answer the question:
• What is a frequency in a signal?
high frequencies
low frequency
sum of all 4 frequencies
TOC 64/110
![Page 65: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/65.jpg)
What is a frequency... in the image
• low frequency: smooth areas
• high frequency: abrupt changesin the luminance
TOC 65/110
![Page 66: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/66.jpg)
What is a frequency... in the image
low/ high frequency?
TOC 66/110
![Page 67: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/67.jpg)
What is a frequency... in the image
TOC 67/110
![Page 68: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/68.jpg)
LOW pass filtering
• Mean filter
• Gaussian
G (x , y) = 12πσ2 e−
x2+y2
2σ2discrete G (x , y) σ = 1
Discretized approximation to G (x , y):- by averaging over a pixel area (0.001 precision)- fspecial in Matlab uses the value of the Gaussian at
the centre of a pixel in the mask.
TOC 68/110
![Page 69: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/69.jpg)
LOW pass filtering
• Median
Quiz 21 to 24
TOC 69/110
![Page 70: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/70.jpg)
LOW pass filtering: results
original image
5 x 5 Gaussian filter
5 x 5 mean filter
5 x 5 median filter
TOC 70/110
![Page 71: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/71.jpg)
LOW pass filtering: results
Quiz 25, 26
25 Is it true?LP filtering keeps low frequencies of an image.
26 Is it true?image a (below) contains higher frequencies than image b.
a b
TOC 71/110
![Page 72: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/72.jpg)
Image filtering: type of filters
• LOW pass filter:eliminate high frequencies.Application: smoothing,blurring, erasing noise...
• HIGH pass filter:eliminate low frequencies.Application:edge detection, enhancedetails...
TOC 72/110
![Page 73: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/73.jpg)
HIGH pass filtering
Properties:
• Edge (contour): corresponds to abrupt changes in thelumimance i.e. high frequencies
• i.e. HP filters exhibit regions with high gradient of the graylevel values
Quiz 27
TOC 73/110
![Page 74: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/74.jpg)
Edge detection in 1D
• Edges corresponds to the extrema of the norm of the 1stderivative or equivalently to the zeros of the 2nd derivative.
TOC 74/110
![Page 75: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/75.jpg)
HIGH pass filtering
2D derivation operators: gradient, laplacian...
• 1st derivative: Gradient
∇f |x0,y0 =∂f
∂x(x0, y0)ex +
∂f
∂y(x0, y0)ey ,
norm |∇f |2 =
(∂f
∂x
)2
+
(∂f
∂y
)2
• 2nd derivative: Laplacian
∇2f =∂f
∂x2+
∂f
∂y2
• edges corresponds to the extrema of the norm of the gradientor equivalently to the zeros of the Laplacian.
TOC 75/110
![Page 76: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/76.jpg)
Gradient computation
Discretization of the Gradient Quiz 28,29
∆I
∆x=
I (x + ∆x)− I (x)
∆x
Let us assume that ∆x = 1
• vertical axis direction,(detection of horizontal edges) , then∆I = I [1 + i , j ]− I [i , j ]
filter:
−101
or
(−11
)
• horizontal axis direction,(detection of vertical edges) , then∆I = I [i , j + 1]− I [i , j ]
filter: (−1, 0, 1) or (−1, 1)
TOC 76/110
![Page 77: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/77.jpg)
HIGH pass filtering (gradient): results
original image
Gradient horizontal
Gradient vertical
Gradient norm Quiz 30
TOC 77/110
![Page 78: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/78.jpg)
Other HP filters
Quiz 31, 32
original Prewitt Sobel
TOC 78/110
![Page 79: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/79.jpg)
HIGH pass filtering: results after thresholding
original image
Prewitt
Gradient norm
Sobel
TOC 79/110
![Page 80: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/80.jpg)
Edge detection based on Laplacian
• Second derivative of a contniuous 2D function f = Laplacian
∇2f =∂2f
∂x2+∂2f
∂y2
• for a discrete image I , discretized by
∇2I = I [i + 1, j ] + I [i − 1, j ] + I [i , j + 1] + I [i , j − 1]− 4 I [i , j ] 0 1 01 −4 10 1 0
• the edges correspond to zero crossing of the Laplacian
TOC 80/110
![Page 81: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/81.jpg)
Result of edge detection
original Sobel Laplacian
TOC 81/110
![Page 82: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/82.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 82/110
![Page 83: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/83.jpg)
Downsampling (if time permits)Aliasing: occurs when downsampling without low pass filtering
Properly sampled image of brick wall. Spatial aliasing
Downsampling with proper low pass filtering:
TOC 83/110
![Page 84: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/84.jpg)
Interpolation/Upsampling
TOC 84/110
![Page 85: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/85.jpg)
Interpolation/Upsampling
• Nearest Neighbor (NN) method
1D 2D
Drawback: blocking effect
TOC 85/110
![Page 86: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/86.jpg)
Bilinear Interpolation
1D 2D
Avoid blocking effect BUT non smooth i.e. non natural
TOC 86/110
![Page 87: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/87.jpg)
Bicubic Interpolation
• linear combination of the 16 neighboring pixels. The weightsdepend on the distance between the neighboring pixel and thepixel to be interpolated
• bicubic interpolation achieves a good complexity/renderingtradeoff
1D 2D
TOC 87/110
![Page 88: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/88.jpg)
Interpolation: the results
TOC 88/110
![Page 89: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/89.jpg)
Interpolation: the results
TOC 89/110
![Page 90: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/90.jpg)
Interpolation: the results
TOC 90/110
![Page 91: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/91.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 91/110
![Page 92: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/92.jpg)
Image segmentation is...
• the process of partitioning a digital image into multiplesegments,
• the process of assigning a label to every pixel in an image suchthat pixels with the same label share certain characteristics.
Example: counting holes.
original detect holes label
TOC 92/110
![Page 93: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/93.jpg)
Label connected components in 2-D binary image
Source: A. Boucher IFI
TOC 93/110
![Page 94: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/94.jpg)
first iteration
TOC 94/110
![Page 95: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/95.jpg)
first iteration
TOC 95/110
![Page 96: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/96.jpg)
first iteration
TOC 96/110
![Page 97: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/97.jpg)
SECOND iteration
TOC 97/110
![Page 98: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/98.jpg)
SECOND iteration
TOC 98/110
![Page 99: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/99.jpg)
Label connected components in 2-D binary image
• in this example, 2 iterations were sufficient to performsegmentation,
• in general, more than 2 iterations are needed,
• the iterations are performed until there is no more change inthe labels: so indeed, we should have perform a last iterationto be sure that the labels would no change.
Quiz 33 to 35
TOC 99/110
![Page 100: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/100.jpg)
Part 5 - Global to point transformation
When a pixel of the new (transformed) image depends on a thewhole input image
TOC 100/110
![Page 101: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/101.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 101/110
![Page 102: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/102.jpg)
DCT:Discrete cosine transform
Change of basis:Decomposition of the input patch xn1,n2into the coefficients Xk1,k2 in the new basis
Xk1,k2 =
N1−1∑n1=0
(N2−1∑n2=0
xn1,n2 cos
[π
N2
(n2 +
1
2
)k2
])
cos
[π
N1
(n1 +
1
2
)k1
]basis functions of the
2D-DCT for 8× 8patches
Used in jpeg image compression, and mpeg-2, mpeg-4, hevc video
compression Quiz 36 to 39
TOC 102/110
![Page 103: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/103.jpg)
Part 6 - Quality measure
How to assess that an image transformation works correctly?
TOC 103/110
![Page 104: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/104.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 104/110
![Page 105: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/105.jpg)
An objective quality measure
Peak signal-to-noise ratio (PSNR):
• measures the distortion between two images I and K
• defined via the mean squared error (MSE):
MSE =1
m n
m∑i=1
n∑j=1
(I [i , j ]− K [i , j ])2
and
PSNR = 10 · log10
(MAX 2
I
MSE
)For instance, MAX = 255.
• if I = true image and K the processed image, PSNR is aquantitative measure of the quality of the image
• The higher the better...
TOC 105/110
![Page 106: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/106.jpg)
Part 7 - Machine learning
How to learn automatically how humans understand/do things?
TOC 106/110
![Page 107: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/107.jpg)
Part 1 - General introduction
Part 2 - Image transformation3 kinds of transformation
Part 3 - Point-to-point transformationGeometric transformationContrast enhancement/reductionGray level histogramsQuantization, binarization
Part 4 - Local to point transformationMorphological image processingImage filtering, convolution
LOW pass filteringHIGH pass filteringDownsampling and Interpolation/Upsampling
Segmentation
Part 5 - Global to point transformation
Part 6 - Quality measure
Part 7 - Machine learning
TOC 107/110
![Page 108: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/108.jpg)
Artificial intelligence (AI)
AI applied to images: computer vision
• System of experts: emulates the decision-making ability of ahuman expert. (Wikipedia)
Expert: specifies all steps he took to make the decisionMachine: is programmed such as to reproduce the steps.
• Machine learning gives ”computers the ability to learnwithout being explicitly programmed.” [A. Samuel 59] (Wikipedia)
Expert: gives the decision for a set of training examplesMachine: is programmed such as to mimic the decision.
Big interest into machine learning due to deep learning!!
TOC 108/110
![Page 109: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/109.jpg)
An example: digit recognition solved with machine learning
Algorithm: nearest neighbor search.
Source: V. Athitsos. Nearest Neighbor Retrieval and Classification.
TOC 109/110
![Page 110: Aline Roumy Inria, Rennes, team Siroccopeople.irisa.fr/Aline.Roumy/teaching/digital_image...Inria, Rennes, team Sirocco The course: you and me! Who am I?Researcher at Inria, Rennes.](https://reader035.fdocuments.net/reader035/viewer/2022081619/60f7bf1fbea3555a342b5bde/html5/thumbnails/110.jpg)
Pro’s and con’s of machine learning
+ efficient... – if many many (training) samples
– expert needs to give decisions for each training data(supervised)
– there exists non supervised (k-means clustering) but lessefficient
Today’s most efficient machine learning algorithms: deep learning.Classification with Deep Convolutional Neural Networks by AKrizhevsky won ImageNet 2012 challenge with major decrease inclassification error.
Since then, tremendous work in the community on deep learning.
First layers in deep learning... look like classical image processing!!!
TOC 110/110