Motion illusion, rotating snakes. Slide credit Fei Fei Li

Click here to load reader

download Motion illusion, rotating snakes. Slide credit Fei Fei Li

of 49

  • date post

    14-Dec-2015
  • Category

    Documents

  • view

    229
  • download

    2

Embed Size (px)

Transcript of Motion illusion, rotating snakes. Slide credit Fei Fei Li

  • Slide 1

Motion illusion, rotating snakes Slide 2 Slide credit Fei Fei Li Slide 3 Slide 4 Slide 5 Image Filtering Computer Vision James Hays, Brown 09/11/2013 Many slides by Derek Hoiem Slide 6 Next three classes: three views of filtering Image filters in spatial domain Filter is a mathematical operation of a grid of numbers Smoothing, sharpening, measuring texture Image filters in the frequency domain Filtering is a way to modify the frequencies of images Denoising, sampling, image compression Templates and Image Pyramids Filtering is a way to match a template to the image Detection, coarse-to-fine registration Slide 7 Image filtering Image filtering: compute function of local neighborhood at each position Really important! Enhance images Denoise, resize, increase contrast, etc. Extract information from images Texture, edges, distinctive points, etc. Detect patterns Template matching Slide 8 111 111 111 Slide credit: David Lowe (UBC) Example: box filter Slide 9 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0 0000000000 0000000000 000 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Credit: S. Seitz Image filtering 111 111 111 Slide 10 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 010 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz Slide 11 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 01020 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz Slide 12 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz Slide 13 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz Slide 14 0102030 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz ? Slide 15 0102030 50 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 Image filtering 111 111 111 Credit: S. Seitz ? Slide 16 0000000000 0000000000 00090 00 000 00 000 00 000 0 00 000 00 0000000000 00 0000000 0000000000 0102030 2010 0204060 4020 0306090 6030 0 5080 906030 0 5080 906030 0203050 604020 102030 2010 00000 Image filtering 111 111 111 Credit: S. Seitz Slide 17 What does it do? Replaces each pixel with an average of its neighborhood Achieve smoothing effect (remove sharp features) 111 111 111 Slide credit: David Lowe (UBC) Box Filter Slide 18 Smoothing with box filter Slide 19 One more on board Slide 20 Practice with linear filters 000 010 000 Original ? Source: D. Lowe Slide 21 Practice with linear filters 000 010 000 Original Filtered (no change) Source: D. Lowe Slide 22 Practice with linear filters 000 100 000 Original ? Source: D. Lowe Slide 23 Practice with linear filters 000 100 000 Original Shifted left By 1 pixel Source: D. Lowe Slide 24 Practice with linear filters Original 111 111 111 000 020 000 - ? (Note that filter sums to 1) Source: D. Lowe Slide 25 Practice with linear filters Original 111 111 111 000 020 000 - Sharpening filter - Accentuates differences with local average Source: D. Lowe Slide 26 Sharpening Source: D. Lowe Slide 27 Other filters 01 -202 01 Vertical Edge (absolute value) Sobel Slide 28 Other filters -2 000 121 Horizontal Edge (absolute value) Sobel Slide 29 How could we synthesize motion blur? theta = 30; len = 20; fil = imrotate(ones(1, len), theta, 'bilinear'); fil = fil / sum(fil(:)); figure(2), imshow(imfilter(im, fil)); Slide 30 Filtering vs. Convolution 2d filtering h=filter2(g,f); or h=imfilter(f,g); 2d convolution h=conv2(g,f); f=imageg=filter Slide 31 Key properties of linear filters Linearity: filter(f 1 + f 2 ) = filter(f 1 ) + filter(f 2 ) Shift invariance: same behavior regardless of pixel location filter(shift(f)) = shift(filter(f)) Any linear, shift-invariant operator can be represented as a convolution Source: S. Lazebnik Slide 32 More properties Commutative: a * b = b * a Conceptually no difference between filter and signal But particular filtering implementations might break this equality Associative: a * (b * c) = (a * b) * c Often apply several filters one after another: (((a * b 1 ) * b 2 ) * b 3 ) This is equivalent to applying one filter: a * (b 1 * b 2 * b 3 ) Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [0, 0, 1, 0, 0], a * e = a Source: S. Lazebnik Slide 33 Weight contributions of neighboring pixels by nearness 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5, = 1 Slide credit: Christopher Rasmussen Important filter: Gaussian Slide 34 Smoothing with Gaussian filter Slide 35 Smoothing with box filter Slide 36 Gaussian filters Remove high-frequency components from the image (low-pass filter) Images become more smooth Convolution with self is another Gaussian So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have Convolving two times with Gaussian kernel of width is same as convolving once with kernel of width 2 Separable kernel Factors into product of two 1D Gaussians Source: K. Grauman Slide 37 Separability of the Gaussian filter Source: D. Lowe Slide 38 Separability example * * = = 2D convolution (center location only) Source: K. Grauman The filter factors into a product of 1D filters: Perform convolution along rows: Followed by convolution along the remaining column: Slide 39 Separability Why is separability useful in practice? Slide 40 Some practical matters Slide 41 How big should the filter be? Values at edges should be near zero Rule of thumb for Gaussian: set filter half-width to about 3 Practical matters Slide 42 What about near the edge? the filter window falls off the edge of the image need to extrapolate methods: clip filter (black) wrap around copy edge reflect across edge Source: S. Marschner Slide 43 Practical matters methods (MATLAB): clip filter (black): imfilter(f, g, 0) wrap around:imfilter(f, g, circular) copy edge: imfilter(f, g, replicate) reflect across edge: imfilter(f, g, symmetric) Source: S. Marschner Q? Slide 44 Practical matters What is the size of the output? MATLAB: filter2(g, f, shape) shape = full: output size is sum of sizes of f and g shape = same: output size is same as f shape = valid: output size is difference of sizes of f and g f gg gg f gg g g f gg gg full samevalid Source: S. Lazebnik Slide 45 2006 Steve Marschner 45 Median filters A Median Filter operates over a window by selecting the median intensity in the window. What advantage does a median filter have over a mean filter? Is a median filter a kind of convolution? Slide by Steve Seitz Slide 46 2006 Steve Marschner 46 Comparison: salt and pepper noise Slide by Steve Seitz Slide 47 Project 1: Hybrid Images Gaussian Filter! Laplacian Filter! A. Oliva, A. Torralba, P.G. Schyns, Hybrid Images, SIGGRAPH 2006 Hybrid Images, Gaussian unit impulse Laplacian of Gaussian Slide 48 Take-home messages Linear filtering is sum of dot product at each position Can smooth, sharpen, translate (among many other uses) Be aware of details for filter size, extrapolation, cropping 111 111 111 Slide 49 Practice questions 1.Write down a 3x3 filter that returns a positive value if the average value of the 4-adjacent neighbors is less than the center and a negative value otherwise 2.Write down a filter that will compute the gradient in the x-direction: gradx(y,x) = im(y,x+1)-im(y,x) for each x, y Slide 50 Practice questions 3. Fill in the blanks: a) _ = D * B b) A = _ * _ c) F = D * _ d) _ = D * D A B C D E F G HI Filtering Operator Slide 51 Next class: Thinking in Frequency