PowerPoint Presentation

Course Syllabus

Color Camera models, camera calibrationAdvanced image pre-processing Line detectionCorner detectionMaximally stable extremal regionsMathematical Morphology binarygray-scaleskeletonizationgranulometrymorphological segmentationScale in image processingWavelet theory in image processingImage CompressionTextureImage Registration rigidnon-rigidRANSACReferencesBook: Chapter 5, Image Processing, Analysis, and Machine Vision, Sonka et al, latest edition (you may collect a copy of the relevant chapters from my office)Papers:Harris and Stephens, 4th Alvey Vision Conference, 147-151, 1988.Matas et al, Image Vision Geometry, 22:761-767, 20042TopicsLine detectionInterest points Corner DetectionMoravec detectorFacet modelHarris corner detectionMaximally stable extremal regions

3Line detectionUseful in remote sensing, document processing etc.Edges: boundaries between regions with relatively distinct gray-levelsthe most common type of discontinuity in an image

4Line detectionUseful in remote sensing, document processing etc.Edges: boundaries between regions with relatively distinct gray-levelsthe most common type of discontinuity in an imageLines:instances of thin lines in an image occur frequently enough it is useful to have a separate mechanism for detecting them.

5Line detection: How?Possible approaches: Hough transform

6Line detection: How?Possible approaches: Hough transform (more global analysis and may not be considered as a local pre-processing technique)

7Line detection: How?

8Line detection: How?

9Lines and corner for correspondenceInterest points for solving correspondence problems in time series data.Corners are better than lines in solving the above 10Lines and corner for correspondenceInterest points for solving correspondence problems in time series data.Corners are better than lines in solving the above Consider that we want to solve point matching in two images

?11Lines and corner for correspondenceInterest points for solving correspondence problems in time series data.Corners are better than lines in solving the above Consider that we want to solve point matching in two images

A vertex or corner provides better correspondence?12CornersChallengesGradient computation is less reliable near a corner due to ambiguity of edge orientation

?13CornersChallengesGradient computation is less reliable near a corner due to ambiguity of edge orientation

Corner detector are usually not very robust. ?14CornersChallengesGradient computation is less reliable near a corner due to ambiguity of edge orientation

Corner detector are usually not very robust. This deficiency is overcome either by manual intervention or large redundancies. ?15CornersChallengesGradient computation is less reliable near a corner due to ambiguity of edge orientation

Corner detector are usually not very robust. This deficiency is overcome either by manual intervention or large redundancies. The later approach leads to many more corners than needed to estimate transforms between two images.?16Corner detection

17Corner detection

18Continued 19Harris corner detectorKey idea: Measure changes over a neighborhood due to a shift and then analyze its dependency on shift orientation20Harris corner detectorKey idea: Measure changes over a neighborhood due to a shift and then analyze its dependency on shift orientationOrientation dependency of the response for lines21Harris corner detectorKey idea: Measure changes over a neighborhood due to a shift and then analyze its dependency on shift orientationOrientation dependency of the response for lines22Harris corner detectorKey idea: Measure changes over a neighborhood due to a shift and then analyze its dependency on shift orientationOrientation dependency of the response for linesHigh response for shifts along the edge direction; low responses for shifts toward orthogonal direction directionAnisotropic response23Key idea: continued Orientation dependence of the shift response for cornersHigh response for shifts along all directionsIsotropic response24Harris corner: mathematical formulation An image patch or neighborhood W is shifted by a shift vector = [x, y]T

25Harris corner: mathematical formulation An image patch or neighborhood W is shifted by a shift vector = [x, y]TA corner does not have the aperture problem and therefore should show high shift response for all orientation of . 26Harris corner: mathematical formulation An image patch or neighborhood W is shifted by a shift vector = [x, y]TA corner does not have the aperture problem and therefore should show high shift response for all orientation of . The square intensity difference between the original and the shifted image over the neighborhood W is

27Harris corner: mathematical formulation 28Harris corner: mathematical formulation 29Harris corner: mathematical formulation 30Continued 31Continued 32Continued 33Continued 34Continued 35Continued 36Continued 37Continued 38Continued 39Harris matrix The matrix AW is called the Harris matrix and its symmetric and positive semi-definite.

40Harris matrix The matrix AW is called the Harris matrix and its symmetric and positive semi-definite. Eigen-value decomposition of of AW gives eigenvectors and eigenvalues (1, 2) of the response matrix..41Harris matrix The matrix AW is called the Harris matrix and its symmetric and positive semi-definite. Eigen-value decomposition of of AW gives eigenvectors and eigenvalues (1, 2) of the response matrix. Three distinct situations:Both 1 and 2 are small no edge or corner; a flat region

42Harris matrix The matrix AW is called the Harris matrix and its symmetric and positive semi-definite. Eigen-value decomposition of of AW gives eigenvectors and eigenvalues (1, 2) of the response matrix. Three distinct situations:Both 1 and 2 are small no edge or corner; a flat regioni is large but ji is small existence of an edge; no corner

43Harris matrix The matrix AW is called the Harris matrix and its symmetric and positive semi-definite. Eigen-value decomposition of of AW gives eigenvectors and eigenvalues (1, 2) of the response matrix. Three distinct situations:Both 1 and 2 are small no edge or corner; a flat regioni is large but ji is small existence of an edge; no cornerBoth 1 and 2 are large existence of a corner

44Harris matrix

45Algorithm: Harris corner detectionFilter the image with a Gaussian

46Algorithm: Harris corner detectionFilter the image with a GaussianEstimate intensity gradient in two coordinate directions

47Algorithm: Harris corner detectionFilter the image with a GaussianEstimate intensity gradient in two coordinate directionsFor each pixel c and a neighborhood window WCalculate the local Harris matrix A48Algorithm: Harris corner detectionFilter the image with a GaussianEstimate intensity gradient in two coordinate directionsFor each pixel c and a neighborhood window WCalculate the local Harris matrix ACompute the response function R(A)

49Algorithm: Harris corner detectionFilter the image with a GaussianEstimate intensity gradient in two coordinate directionsFor each pixel c and a neighborhood window WCalculate the local Harris matrix ACompute the response function R(A)Choose the best candidates for corners by selecting thresholds on the response function R(A)

50Algorithm: Harris corner detectionFilter the image with a GaussianEstimate intensity gradient in two coordinate directionsFor each pixel c and a neighborhood window WCalculate the local Harris matrix ACompute the response function R(A)Choose the best candidates for corners by selecting thresholds on the response function R(A) Apply non-maximal suppression

51Examples

52Examples

53Examples

54Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255. building

55Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255.Initially its an empty image and then some dots (local minima) appear and starts growing56Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255.Initially its an empty image and then some dots (local minima) appear and starts growingNew dots appear and starts growing and so on57Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255.Initially its an empty image and then some dots (local minima) appear and starts growingNew dots appear and starts growing and so onFrom time to time two disconnected regions get merged 58Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255.Initially its an empty image and then some dots (local minima) appear and starts growingNew dots appear and starts growing and so onFrom time to time two disconnected regions get merged Finally, all regions get merged into a single component.59Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by thresholding a gray level image (intensity values [0,1,,255]) at all possible thresholds starting from 0 and ending at 255.BUT, the important observation here is that 60Maximally Stable Extremal Regions (MSER)Key idea: Consider a movie generated by th