Fast LineSegment Extraction for SemiDense Stereo Matching...Haar Feature Extraction Integral images...
Transcript of Fast LineSegment Extraction for SemiDense Stereo Matching...Haar Feature Extraction Integral images...
Fast LineSegment Extraction for SemiDense Stereo Matching
By Brian McKinnon and Jacky Baltes
University of Manitoba
Overview
Motivation Related Work Implementation
Line segment extraction SemiDense stereo matching
Results Conclusion
Motivation
Initial work was done with regionbased stereo matching Region extraction was expensive Matching regions with a change in disparity was difficult Most of the information used was on the edges of the
region boundary
Related Work
Most line segment extraction algorithms are an extension of the generalized Hough Transform Global methods can handle noise images Computation is expensive False lines can be produced on texture surfaces
Local methods, like chain coding, provide a faster alternative More sensitive to noise
Related Work
Due to instability end point matching of line segments is difficult
Line segment matching is generally achieved using Color information around the segment Topological information
Colinear, parallel, intersection, length ration
LineSegment Extraction
Intergral Image Generation Haar Feature Extraction Gradient Thinning Binary Segmentation Overlap Cleanup
Samples are grayscale 320x240 images
Integral Image Generation
In an Integral image each element contains the sum of all pixels above and to the left of the current position
The sum of any sized rectangular region can be extracted in constant time with four table lookups and three arithmatic operations
Haar Feature Extraction
Integral images are best used for the extraction of variable sized Haar features
Horizontal and vertical Haarfeatures are used to replaceSobel for edge detection
In our experiments, 4x4, 8x8, and 12x12 features are extracted
Gradient Thinning
This method is based on Canny edge detection with Haar features and an efficientgradient orientation structure
If the gradient magnitude exceeds a user defined threshold the gradient orientation is used activate an Edge Cell
Edge Cell (EC)
The EC is an efficient binary data structure for representing gradient orientation
The EC is two layers with eight/4 cells in each layer
The inner layer is offset by /8 from the outer layer
The entire EC has a /8 resolution
Edge Cell (EC)
The EC can be tested quickly for horizontal, vertical, and diagonal orientations using a binary comparisons
Two EC's can be compared using a binary test for: Equality if all cells in the EC's are overlapping Similarity if at least one cell from each EC is overlapping
Binary Segmentation
Line segments are extracted used a simple binary segmentation algorithm
Split Edge Cells (SEC) are created by making two edge cells with the inner and outer layer Each layer receives a seperate segmentation
Using an 8 connected binary segmentation neighbouring pixels are added to the line segment if they are ”similar” to the SEC
Overlap Cleanup
After the segmentation each EC is a member of two line segments
Segments are cleaned in two ways If a segment is contained entirely within one or two
longer line segments then it is discarded Two partially overlapping segments are cut at a point that
maximizes the area of a triangle formed from the start, cut, and end point of the line segments
Resulting Line Segments
SemiDense Stereo Matching
Line segment matching between stereo views is achieved using a dynamic programming approach
The matching function uses the Haar features that were computed during the edge dection phase
Using the two line segment, ls1 is aligned on the row header, and ls2 is aligned on the column header
Matching Function
The DP table is easiest to work with when match values are being maximized In Summed Square Error the best value is 0, which
means an arbitrary value must be assigned to missing elements
The matching function uses feature vectors constructed using the six Haar features extracted during the edge detection stage ( 3 different sized horizontal and vertical features )
Matching Function The match function has been modified from the one in the paper for efficiency
Dynamic Programming Table
The DP table exists as a combination of two tables Match sum table (ST) that stores the current feature
match score Count table (CT) that store the number of matches
contributing to the match score
DP = ST/CT DP(x,y) = DP(xi,yi) represents the operation
ST(x,y) = ST(xi,yi) + R CT(x,y) = CT(xi,yi) + 1
Insertion Rules
Entries are inserted to the table using the following rules The first row and column are forced to match with the
first element If a line segment diverges from the centre line, it cannot
converge back No point from one line can match more than two points
in the second line
Matching Example Direction
The red line is the diagonal, blue and green move away from the diagonal or parallel to it
Matching Example – Sample Matches
A sample set of match values
Matching Example – Ct and St
The resulting count and match table generated from the previous values
Matching Example – Dp Result
The matching set is back traced from best value along the far edge of the table
Results
The algorithm was evaluated using 27 image pairs from the 2005 and 2006 Middlebury stereo sets
Compared against the ground truth and ignoring areas of occlusion the algorithm was correct in78.8 +/ 10% of matched pixels with a coverage of 7.0 +/ 2.4% of the image
Running time is 0.5 to 1 sec / 320x240 frame @ 1.8Ghz
Results Middlebury
Results Unrectified
Results Video
Video sequence provided by Daimler Ag 2007 Processing 5 sec / 640x480 frame @ 1.8Ghz
Conclusion
Line segments can be extracted quickly and accurately using fast binary segmentation methods with Edge Cells
The dynamic programming based matching criteria can quickly produce accurate semidense stereo matches
The main benefit of this method is that it operates without assumptions of rectification or color correction
Future Work
Complete the depth map by filling in the dense information
Improve the performance of the algorithm to achieve realtime processing
Use the depth information for SLAM in USAR environments
Thank you
Try out the software by downloading from aalab.cs.umanitoba.ca/~ummcki01/academicsoftware Available as a Linux binary
Insertion Algorithm
compute_best