Summary : Depth discontinuities by pixel-to-pixel stereo

24
Jeong Kanghun CRV (Computer & Robot Vision) Lab.

description

Summary : Depth discontinuities by pixel-to-pixel stereo. Jeong Kanghun CRV (Computer & Robot Vision) Lab. Index of paper. Introduction Stereo formulation Cost function Occlusion penalty and match reward Pixel dissimilarity Hard constraints - PowerPoint PPT Presentation

Transcript of Summary : Depth discontinuities by pixel-to-pixel stereo

Page 1: Summary :  Depth discontinuities by pixel-to-pixel stereo

Jeong KanghunCRV (Computer & Robot Vision) Lab.

Page 2: Summary :  Depth discontinuities by pixel-to-pixel stereo

Index of paper

1. Introduction2. Stereo formulation

1. Cost function1. Occlusion penalty and match reward2. Pixel dissimilarity

2. Hard constraints1. Intensity variation accompanies depth

discontinuities2. Constraints related to search

3. Searching along epipolar scanlines1. Two dual optimal algorithms2. A faster algorithm

4. Propagating information between scanlines

5. Experimental results6. Comparison with previous work7. Conclusion

Page 3: Summary :  Depth discontinuities by pixel-to-pixel stereo

Abstract

The algorithm matches individual pixels in corresponding scanline pairs

Allowing occluded pixel to remain unmatched, then propagates the information between scanlines by means of a fast postprocessor.

1. 5 microseconds per pixel per disparity on a workstation.

Page 4: Summary :  Depth discontinuities by pixel-to-pixel stereo

1. Introduction

A method for detecting depth discontinuities from a stereo pair of images.

Disparity map Postprocessing step propagates information

between the scanlines to refine the disparity map and the depth discontinuities.1. The image sampling problem is overcome by using a

measure of pixel dissimilarity that is insensitive to sampling

2. The algorithm handles large untextured regions which present a challenge to many existing stereo algorithms

3. Unlikely search nodes are pruned to reduce dramatically the running time of dynamic programming

Page 5: Summary :  Depth discontinuities by pixel-to-pixel stereo

2. Stereo formulation

IL (x), IR (y) are images of the same scene point. (Throughout this paper, x denotes a pixel in the left scanline, while y denotes a pixel in the right scanline.)

Unmatched pixels are occluded. Disparity : = The depth-discontinuity pixels are labelled

as those pixels that border a change of at least two levels of disparity and that lie on the far object.

( )x x y

Page 6: Summary :  Depth discontinuities by pixel-to-pixel stereo

2. Stereo formulation(cont.)

Page 7: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.1 Cost function

N is number of occlusion (not the number of occluded pixels).

M is matched pixel. Kocc : 25 Kr : 5

Constant occlusion penalty

Constant match reward

Dissimilarity between pixels

Page 8: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.2 Pixel dissimilarity

We propose instead to use the linearly interpolated intensity functions surrounding two pixels to measure their dissimilarity, in a method that is provably insensitive to sampling

IR and IL incident upon two corresponding scanlines of the left and right cameras, respectively.

the linearly interpolated function between the sample points of the right scanline.

chosen as the pixels whosedissimilarity is to be measured

Page 9: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.2 Pixel dissimilarity(cont.)

The dissimilarity between the pixels is computed as the minimum of this quantity and its symmetric counterpart

definition of d is symmetrical.

Page 10: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.2 Pixel dissimilarity(cont.)

Imin and Imax

Page 11: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.2 Pixel dissimilarity(cont.)

The quantity d is insensitive to sampling in the sense that, without noise or other distortions.

d(xi , yi) = 0 whenever yi is the closest sampling point to the y value corresponding to xi.

Page 12: Summary :  Depth discontinuities by pixel-to-pixel stereo
Page 13: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2 Hard constraints

All match sequences to satisfy certain constraints.

The second set facilitates a systematic, efficient search.

Page 14: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.1 Intensity variation accompanies depth discontinuities

For a scene to be artificially altered by placing a textured background.

Namely that intensity variation accompanies depth discontinuities (Think of it as a weak intensity edge)

Intensity variation occurring at a depth discontinuity (as the result of an intensity difference between the near object and the far object)

The same disparity as the near object (on next slide)

Page 15: Summary :  Depth discontinuities by pixel-to-pixel stereo
Page 16: Summary :  Depth discontinuities by pixel-to-pixel stereo

2.2.2 Constraints related to search

If (x, y) is a match, either x + 1 or y + 1 must be matched.

Page 17: Summary :  Depth discontinuities by pixel-to-pixel stereo

3. Searching along epipolar scanlines

To find the optimal match sequence by conducting an exhaustive search.

X corresponds to an occluded pixel

Page 18: Summary :  Depth discontinuities by pixel-to-pixel stereo

3.1 Two dual optimal algorithms

Fig. 5b and Fig. 5cMatched pixels

Page 19: Summary :  Depth discontinuities by pixel-to-pixel stereo

3.2 A faster algorithm

The running time would not be reduced because of the difficulty in determining whether there is a lower-cost match above or to the left of another match.

Page 20: Summary :  Depth discontinuities by pixel-to-pixel stereo

4. Propagating information between scanlines

We have devised a method for postprocessing the disparity map by propagating reliable disparity values into regions of unreliable disparity values.

Each pixel is assigned a level of reliability

Page 21: Summary :  Depth discontinuities by pixel-to-pixel stereo

5. Experimental Result

Slightly defocused of 10 mm It is important to note that even in principle

this problem can never be eliminated completely.

The algorithm is heavily dependent upon local information.

Using size of 630 X 480 images. The maximum disparity is set 14.

Page 22: Summary :  Depth discontinuities by pixel-to-pixel stereo

Conclusion

Detecting depth discontinuities is an important problem that is rarely emphasized in stereo matching.

The algorithm is fast and able to compute disparities and depth discontinuities in some situations where previous algorithms would fail.

Page 23: Summary :  Depth discontinuities by pixel-to-pixel stereo

Figure 8 : Result

Page 24: Summary :  Depth discontinuities by pixel-to-pixel stereo