Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel...
-
Upload
tristian-trenton -
Category
Documents
-
view
217 -
download
1
Transcript of Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel...
![Page 1: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/1.jpg)
Efficient High-Resolution Stereo Matching using Local Plane Sweeps
Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014
Yongho Shin
![Page 2: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/2.jpg)
2
High-resolution images require long time for computing a disparity map– Complexity for general local methods for 2x size images
Problems
𝑂 (𝑊𝐻𝑁𝐷 )
x4
𝑂 (210𝑊𝐻𝑁𝐷)
![Page 3: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/3.jpg)
3
Semi-global matching– Optimize following energy function
– NP-hard problem!!• Approximate methods operate in adequate computing time, but still
slow• Dynamic programming gives faster way, but erroneous result
– Instead do dynamic programming along many directions
– It cannot model second-order smoothness
Related works
𝑂 (𝑊𝐻𝐷 )
![Page 4: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/4.jpg)
4
Efficient large-scale stereo matching– Stereo matching based on search space reduction
• Computation GCPs• Delaunay triangulation on GCPs• Matching on triangles with restricted range
Related works
![Page 5: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/5.jpg)
5
VERY RELATED WORKSegment-Based Stereo Matching Using Belief Propagation
![Page 6: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/6.jpg)
Initial matching– Any matching method can be used
Matching with a segmenta-tion
Initial matching
Extraction of reliable pixels
Extraction of model parameter
from each segment
Assignment of optimal parameter
for each segment by BP
Noisy result
![Page 7: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/7.jpg)
Extraction of reliable pixels– Simple cross checking method is used– Occlusion region can be detected
Matching with a segmenta-tion
Initial matching
Extraction of reliable pixels
Extraction of model parameter
from each segment
Assignment of optimal parameter
for each segment by BP
Left image Right image
Left result Right result
![Page 8: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/8.jpg)
Extraction of model param-eter from each segment– At each segment, a model param-
eter is extracted using reliable pixels and robust statistical tech-nique
– Add the parameter to a parameter set
Matching with a segmenta-tion
Initial matching
Extraction of reliable pixels
Extraction of model parameter
from each segment
Assignment of optimal parameter
for each segment by BP
Reliable pixels Segments
![Page 9: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/9.jpg)
Extraction of model parame-ter from each segment– At each segment, a model parame-
ter is extracted using reliable pixels and robust statistical technique
– Add the parameter to a parameter set
Matching with a segmenta-tion
Initial matching
Extraction of reliable pixels
Extraction of model parameter
from each segment
Assignment of optimal parameter
for each segment by BP
Parameter Set
Parameter
![Page 10: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/10.jpg)
Assignment of optimal pa-rameter for each segment by BP– Assign an optimal parameter for
each segment as total energy can be minimized
Matching with a segmenta-tion
Initial matching
Extraction of reliable pixels
Extraction of model parameter
from each segment
Assignment of optimal parameter
for each segment by BP
Parameter #29Parameter #29
Parameter Set
![Page 11: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/11.jpg)
Matching with a segmentation
a : Initial disparity mapb : Interpolated resultc : Reliable pixel mapd : Result from a segmentation
a bc d
![Page 12: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/12.jpg)
12
What they did– Make plane parameter by segment and initial disparity
map
– Find optimal plane parameters for each segment of the image
– Select optimal parameters by BP
Matching with a segmenta-tion
![Page 13: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/13.jpg)
13
PROPOSED METHOD
![Page 14: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/14.jpg)
14
What they do– Make plane parameter by feature points
– Find optimal plane parameters for each tiles of the im-age• Allowing objects having curved surface
– Select optimal parameters by SGM
Information for understand-ing
![Page 15: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/15.jpg)
15
PROPOSED METHODHypothesis generation
![Page 16: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/16.jpg)
16
Feature matching– By Harris corner keypoints and upright DAISY descriptors
– Matching only points along near epipolar line• Due to stereo matching• But, they allow small vertical misalignments
– First round• Initial set of matches are selected using the ratio test heuristic
– Second round• For obtaining more matched features• Horizontal search range is reduced using local estimates
Hypothesis generation
![Page 17: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/17.jpg)
17
Vertical alignment– Correct for small vertical misalignments from errors in
rectification
– By fitting a global linear model using RANSAC with matched features
Hypothesis generation
𝑑𝑦=𝑎𝑦+𝑏
![Page 18: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/18.jpg)
18
Disparity plane estimation– Cluster matched points and find plane parameters
• Find k number of planes
– Using variational approach used for mesh simplification• Graph based approach with priority queue
Hypothesis generation
![Page 19: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/19.jpg)
19
PROPOSED METHODLocal plane sweeps
![Page 20: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/20.jpg)
20
Plane for tiles having parallax– Because there are curved objects in the world– Hence, gives range of ±T pixels of parallax from plane– For each plane, investigate similarity among range 2T
• Optimize by SGM
Local plane sweep
![Page 21: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/21.jpg)
21
Identifying in-range disparities– By disparity map, they give cost U
Local plane sweep
AD NCC JUMP
![Page 22: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/22.jpg)
22
PROPOSED METHODProposal generation
![Page 23: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/23.jpg)
23
Initial proposals– Find the planes with associated points within each tile
Online proposals– Find frequent plane parameter for each tile– Propagate the parameter to neighbors
Proposal generation
![Page 24: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/24.jpg)
24
PROPOSED METHODGlobal optimization
![Page 25: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/25.jpg)
We have– Plane parameters for each tile– Cost U– Energy function
–Power SGM!!
Global optimization
![Page 26: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/26.jpg)
26
EXPERIMENTS
![Page 27: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/27.jpg)
27
Quantitative results
![Page 28: Efficient High-Resolution Stereo Matching using Local Plane Sweeps Sudipta N. Sinha, Daniel Scharstein, Richard Szeliski @ CVPR 2014 Yongho Shin.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c775503460f9492be59/html5/thumbnails/28.jpg)
28
Qualitative results