Lecture 22.5: additional topics in image...
Transcript of Lecture 22.5: additional topics in image...
![Page 1: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/1.jpg)
Extended Introduction to Computer Science CS1001.py
Lecture 22.5: additional topics in image processing
Instructors: Benny Chor, Amir Rubinstein Teaching Assistants: Michal Kleinbort, Yael Baran
School of Computer Science Tel-Aviv University
Spring Semester, 2013-14 http://tau-cs1001-py.wikidot.com
Based on material prepared by Asaf Zaritsky
![Page 2: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/2.jpg)
Lecture 22: Plan • Resolution and quantization (reminder)
• Noise reduction (using lecture 21 slides)
• Edge Detection
• Segmentation
2
![Page 3: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/3.jpg)
3
Pixel Resolution
Image from wikipedia
Demo for pixel resolution effects: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/
• Pixel resolution == pixel count
![Page 4: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/4.jpg)
Image Quantization
24 bit RGB 16 colors
• Number of bits per pixel
• Note that both images have the same pixel & spatial resolution
![Page 5: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/5.jpg)
Image from: http://micro.magnet.fsu.edu/primer/digitalimaging/digitalimagebasics.html
Demo for quantization effects: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/bitdepth/index.html
Image Quantization
![Page 6: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/6.jpg)
• We will use the slides and code from the previous lecture
• Warm up exercise: • Recall the function join_h from last time, which joins 2
images horizontally (used for easy display and comparisons).
• Write a function join(), which appends a variable number of images (vertically or horizontally)
Noise reduction
![Page 7: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/7.jpg)
Edge Detection
7
• Edge - sharp change in intensity between close pixels • Usually captures much of the meaningful information in the image
images extracted using Sobel filter from: http://micro.magnet.fsu.edu/primer/java/digitalimaging/russ/sobelfilter/index.html
![Page 8: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/8.jpg)
Erosion and Dilation
8
• Erosion - the removal of pixels from the periphery of a (white) feature. • Dilation - the adding of pixels to that periphery.
• If we assume bright foreground and dark background: • Erosion shrinks foreground areas, and holes grow. • Dilation enlarges foreground areas, and holes shrink.
Erosion Dilation
Images from: http://micro.magnet.fsu.edu/primer/java/digitalimaging/russ/erosiondilation/index.html
![Page 9: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/9.jpg)
Erosion and Dilation - examples
Erosion
Dilation
![Page 10: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/10.jpg)
Dilation for Edge Detection
Original Dilation Edges
Diff
![Page 11: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/11.jpg)
Erosion and Dilation - Code
def erosion(A, k=1): return local_operator(A, ??, k) def dilation(A, k=1): return local_operator(A, ??, k)
def local_operator(A, op, k=1): n,m = A.dim() res = copy(A) # brand new copy of A for i in range(k,n-k): for j in range(k,m-k): res[i,j] = op(items(A[i-k:i+k+1,j-k:j+k+1])) return res
![Page 12: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/12.jpg)
diff - Code def diff(im1, im2): ''' creates the diff im1-im2 ''' assert im1.dim() == im2.dim() n,m = im1.dim() out = Matrix(n,m) for i in range(n): for j in range(m): out[i,j] = (im1[i,j] - im2[i,j]) return out
![Page 13: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/13.jpg)
PIL's edge detection vs. our own
13
Original Diff PIL's filter
from PIL import ImageFilter im = Image.open("./coins.jpg") new = im.filter(ImageFilter.FIND_EDGES) new.show()
![Page 14: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/14.jpg)
Segmentation
14
From Wikipedia: • In computer vision, image segmentation is the process of partitioning a digital
image into multiple segments (sets of pixels, also known as superpixels). The goal of segmentation is to simplify and/or change the representation of an
image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines,
curves, etc.) in images.
• Applications in medical imaging: - Locate tumors and other pathologies - Measure tissue volumes - Diagnosis, study of anatomical structure
Image from: http://www.sonycsl.co.jp/person/nielsen/applets.html
![Page 15: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/15.jpg)
Segmentation by Thresholding
15
• Simplest segmentation method
• Apply a threshold to turn a gray-scale image into a binary image (BW) –
this is called Binary Segmentation
• Can apply more than one threshold, creating >2 segments
• The key is to select the appropriate threshold values
Human HT29 colon-cancer cells http://www.broadinstitute.org/bbbc/image_sets.html Binary segmentation,
threshold = 20
![Page 16: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/16.jpg)
Binary Segmentation
Threshold = 20 Threshold = 40 Threshold = 60
• Which threshold is the best?
Original
![Page 17: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/17.jpg)
Thresholding Demo
17
Demo for threshold effects on segmentation: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/automaticthresholding/index.html
![Page 18: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/18.jpg)
Variance
• Variance (σ2) of a set is a measure of how much the set is spread out. Defined as the mean distance2 from the set’s center.
A good threshold for segmentation: • Minimizes intra-segment variance • Maximizes inter-segment variance
![Page 19: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/19.jpg)
Otsu method for threshold calculation • Assumes that the image contains two classes of pixels - foreground and
background.
• Finds an optimal threshold for segmentation: – A threshold that maximizes the variance between these classes. – Uses image histogram: grey level distribution in an image.
• x-axis – grey hues • y-axis – number of pixels with a particular hue
Gray-scale histogram
![Page 20: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/20.jpg)
Otsu's formula For every threshold t denote:
Background: <= t (low gray levels) Foreground :> t (high gray levels) w_back – number of Background pixels w_fore – number of Foreground pixels mean_back – mean value of the Background pixels mean_fore – mean value of the Foreground pixels
The Otsu threshold is the threshold that maximizes the var_between over all possible thresholds t.
var_between(t) = w_back * w_fore * (mean_back - mean_fore)2
![Page 21: Lecture 22.5: additional topics in image processingtau-cs1001-py.wdfiles.com/local--files/lecture... · Segmentation . 14 . From Wikipedia: • In computer vision, image segmentation](https://reader033.fdocuments.net/reader033/viewer/2022041800/5e511a861e9dbe547e6d5635/html5/thumbnails/21.jpg)
Otsu's Threshold = 38
Original
Otsu threshold - Run