Epipolar Plane Image Rectification and Flat Surface Detection in...

Research ArticleEpipolar Plane Image Rectification and Flat SurfaceDetection in Light Field

Lipeng Si Hao Zhu and QingWang

School of Computer Science Northwestern Polytechnical University Xirsquoan 710072 China

Correspondence should be addressed to Qing Wang qwangnwpueducn

Received 13 April 2017 Revised 9 July 2017 Accepted 7 August 2017 Published 19 September 2017

Academic Editor Panajotis Agathoklis

Copyright copy 2017 Lipeng Si et alThis is an open access article distributed under the Creative CommonsAttribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Flat surface detection is one of the most common geometry inferences in computer vision In this paper we propose detectingprinted photos from original scenes which fully exploit angular information of light field and characteristics of the flat surfaceUnlike previous methods our method does not need a prior depth estimationThe algorithm rectifies the mess epipolar lines in theepipolar plane image (EPI)Then feature points are extracted from light field data and used to compute an energy ratio in the depthdistribution of the scene Based on the energy ratio a feature vector is constructed and we obtain robust detection of flat surfaceApart from flat surface detection the kernel rectification algorithm in our method can be expanded to inclined plane refocusingand continuous depth estimation for flat surface Experiments on the public datasets and our collections have demonstrated theeffectiveness of the proposed method

1 Introduction

With the rapid development of light field theory [1 2] lightfield cameras such as Lytro [3] and Raytrix [4] are nowavailable for consumer and industrial use Different from2D image captured by traditional camera light field camerarecords extra angular information of the real world and itprovides more possibilities for many traditional computervision tasks [5ndash8]

Flat surface detection is such a prominent task to makeplanar structure inference from natural scenes One potentialapplication of this issue is detection of printed photo in face-based verification [9] Face identification has been widelyapplied in industrial world However a common problemof such system suffers whether the face is a real one or justa printed photo of the authorized face The main reasonis the loss of depth information when the camera recordsthe real world Traditional methods always assume that theprinted faces contain detectable texture patterns or requirea user interaction to solve this problem [10] However thesemethods are unreliable or inefficient Depth estimation [11]can be another option before the authentication but it maybring other problems in depth estimation such as occlusion[12] and shading [13]

In this paper we analyze the variant and invariant featuresof flat surface in EPI representation and propose an algorithmto detect the flat surface without depth estimation whichfully exploits angular information of the light field and thecharacteristics of flat surface Our main contributions are asfollows

(i) An algorithm to rectify EPI for a flat surface whichcan also be expanded to other tasks such as refocusingin an inclined plane

(ii) A framework to detect the flat surface in light field bya two-stage algorithm without depth estimation

The rest of this paper is organized as follows In Section 2we review the background and the previous works aboutflat surface detection In Section 3 the detailed algorithm isdescribed And we give the experimental results in Section 4Our conclusion and future work are arranged in Section 5Some proofs of related properties are provided in Appendix

2 Background and Related Work

A light field is a function defined in 4D space named119891(119909 119910 119906 V) [1] to describe light rays in physical world where

HindawiJournal of Electrical and Computer EngineeringVolume 2017 Article ID 6142795 9 pageshttpsdoiorg10115520176142795

2 Journal of Electrical and Computer Engineering

(119909 119910) and (119906 V) describe the distribution of the light inspatial and angular space respectively Under the two-planeparametrization when we fix one spatial dimension and oneangular dimension (119909lowast 119906lowast) (or (119910lowast Vlowast)) the EPI appearsFor each point in the real world there is a correspondingepipolar line in the EPI and the slope of this line has a linearrelationship with the depth of this point

As a basic problem in computer vision flat surfacedetection has been researched for decades but it is not welldeveloped Most of techniques detect flat surface using aprior depth estimation Zhao et al [11] proposed to detectflat surface using the disparity map of the scene however thismethod depends on the accuracy of disparity estimation andis sensitive to the alignment errors of disparity estimationRaghavendra et al [15] proposed a similar strategy Insteadof the disparity map used in [11] they proposed to obtain arough depth map by using the focus measure Undoubtedlythe method also suffers the same problem that is inaccuracyof disparity estimationGhasemi andVetterli [14] proposed toextract energy feature vector based on the change of gradientof EPI and to distinguish the flat surface from nonflat one byusing a Bayes classifier The method computes the slope of allepipolar lines and then takes the variance it is still a depth-based method

Different from the traditional ldquodepthmap to plane fittingrdquostrategy [11 15 16] we detect the Lisad-1 feature point [17]in 4D light field and then fit the function of the flat surfaceby using several feature points in light field directly finallythe robustness of the function which we build is tested byanother several feature points If the scene is a flat surfacethe function which is built from several feature points shouldalso be suitable for other points and vice versa

3 The Proposed Approach forDetecting Flat Surface

It is well observed that for a flat surface which is parallelto the camera plane all epipolar lines in the EPI have thesame slope since they are in the same depth However thisinvariant property went when the flat surface is tilted todifferent angles By analyzing the properties of flat surfacefunction it is noticed that no matter what angle the flatsurface is tilted to the difference of slope stays the same(this property is discussed in Section 33 and its proof can befound in Appendix) Based on this invariant property we firstpropose to rectify the slope of epipolar lines in EPI into a samevalue 1198780 and then to detect the Lisad-1 feature points [17] ofthe rectified EPI (the most important advantage of the Lisad-1 feature points is that it provides the slope of each featurepoint and the extraction of their depth does not suffer anyocclusion or shading problems as they are salient points)Theslope of each feature point ought to be equal to 1198780 if the planefunction is true Finally we combine the energy ratio fromdifferent EPIs as an energy vector and employ a classifier todistinguish flat surface from natural scenes

31 Epipolar Plane Image Rectification On a flat surface thedepth value of a point has a linear relationship with its 2D

Depth

＞ＪＮＢ = ax + b

X axis

Figure 1 The relationship between the horizontal axis 119909 in imageand its depth is linear if the scene is a flat surface

image coordinates When the point is in an EPI with a fixed119910lowast its depth can be expressed by a linear function only on 119909(Figure 1)

119889 (119909) = 119886119909 + 119887 (1)

where 119889(119909) is the depth of the point (119909 119910lowast) And we canderive the slope of each point by

119878 (119909) = 1dis (119909) =

1119891119904119889 (119909) =

119886119909 + 119887119891119904 (2)

where dis(119909) is the disparity of the point (119909 119910lowast) 119878(119909) is theslope of the point 119909 in EPI with the fixed 119910lowast 119891 is the focallength of the lens and 119904 is the baseline between two viewsAs the camera parameters 119891 and 119904 are constants they areignored Equation (2) can be rewritten as

119878 (119909) = 119886119909 + 119887 (3)

If the linear function is determined the slope of eachpoint in EPI can be known and the slope of epipolar lines inEPI can be normalize into a same value Suppose one point(119909 119906) in the original EPI its slope is 119878(119909lowast) where 119909lowast satisfiesthe following function

119909lowast + 119906 1119878 (119909lowast) = 119909 (4)

If the slope we hope to normalize to is 1198780 (it is called targetslope later) the target point of shearing (119909 119906) is (1199091015840 119906) where1199091015840 meets the following function

1199091015840 = 119909lowast + 119906 11198780 (5)

In a brief summary for a point (1199091015840 119906) in the rectified EPIthe corresponding point in the original EPI is

(1199091015840 minus 119906 11198780 + 1199061

119878 (1199091015840 minus 119906 (11198780)) 119906) (6)

We can refer to Figure 2 to understand the procedure ofrectification And Figure 3 gives an example of the originalEPI and the rectified EPI We can see that only one slope isremained in EPI after the rectification

Journal of Electrical and Computer Engineering 3

(x u)

xlowast

((x u)

Figure 2 The relationship between two points in the original EPI and the rectified EPI The black line refers to the location of the epipolarline in the original EPI and the red dashed line refers to the location of the epipolar line in the rectified EPI After the rectification the slopeof the red dashed line is the same as the slope of the right black line

(a) (b)

Figure 3 (a) is the original EPI and the slopes of different coordinates are different (b) is the rectified EPI and there is only one sloperemained in this EPI

32 Line Function Determination By substituting the depthrange [119889min 119889max] and the image size [1119873] into (1) 4 con-straints are determined to search the value of the parameters(119886 119887)

119886 + 119887 ge 119889min119886 + 119887 le 119889max

119886119873 + 119887 ge 119889min119886119873 + 119887 le 119889max

(7)

By solving these constraints the searching space of (119886 119887) isobtained (it is labeled with red in Figure 4) Then (119886 119887) isgenerated by dividing the searching space into discrete grids

We rectify the EPI by using each possible combination of(119886 119887) A Lisad-1 feature [17] is the local extrema in scale-depthspaces by convolving the EPIs with scale variation kernelsAs the Lisad-1 feature point provides the slope informationthe feature points are extracted and the ratio 119903 of the featurepoints which have the same slope with the target slope 1198780 inall feature points is computed

119903 = sum|119865|119894=1 119892 (119878 (119865119894) = 1198780)|119865|

119892 (119909) = 1 119909 = 10 119909 = 1

(8)

where 119865 is the set of the Lisad-1 feature points extracted fromthe rectified EPI |119865| is the size of the set 119865 119878(119865119894) is the slopeof the 119894th feature point

For a flat surface if (119886 119887) is known all epipolar linesshould have the same slope value with the target slope 1198780 (seeFigure 3) after rectification So we select the optimal (119886 119887)which results in the largest ratio 119903

33 The Proposed Strategy Practically we can not use theratio of only one EPI to detect the flat surface since it is tooregional to represent the whole scene We take the followingtwo useful properties into consideration to solve the problem

(i) The value of 119886 should be a constantwith different fixed119910lowast if the scene is a flat surface(ii) The value of 119887 should have a linear relationship with

the variable 119910 if the scene is a flat surface

These two properties are obvious and can be proved easily(see Appendix) With these two properties we formulate ourstrategy as a plane fitting stage and a feature extracting stage

In the plane fitting stage the plane function (the common119886 and the function of 119887 with 119910) is fitted by using a series ofparameters (119886 119887) calculated from several EPIs And in thefeature extracting stage the parameters (119886 119887) of each EPI arecomputed by using the function of the plane If the scene isa flat surface the plane function that we build from previousEPIs should also be suitable for other EPIs that is to say theslope value in all rectified EPIs should be equal to the value1198780 and vice versa This is the core idea of our strategy Thedetailed description can be seen in Algorithm 1

34 Expand to Inclined Plane Refocus The traditionalmethod shears the EPI [3 18] to achieve refocusing thedisplacement of each point is the same value as the plane wehope to refocus on is a plane which is parallel to the cameraplane in which all points in the plane are in a same depth

119871120572 (119909 119906) = 1198710 (119909 + 119906(1 minus 1120572) 119906) (9)

where 1198710 denotes the input EPI and 119871120572 denotes the shearedEPI by a value of 120572

Under the framework of our algorithm we can obtainthe line function of each EPI after the plane fitting stage and


P0 (NdＧＣＨ minus dＧＲ

N minus 1dＧＲ minus dＧＣＨ

N minus 1)

a

a + b = dＧＣＨa + b = dＧＲ

P2 (dＧＲ 0)

P1 (dＧＣＨ 0)

P3 (NdＧＲ minus dＧＣＨ

N minus 1dＧＣＨ minus dＧＲ

N minus 1)

Na + b = dＧＲ

Na + b = dＧＣＨ

b

Figure 4 With the 4 constraints mentioned in (7) the solution space of the parameters (119886 119887) is obtained and it is labeled with red

(1) Input Light field LF slope range [119889min 119889max] thenumber of training samples 1198701 the number of testingsamples 1198702

(2)OutputThe energy ratio 119877 of the feature points whichhave the same slope value with 1198780 in all feature pointswhich come from all rectified EPIs

(3) lowastPlane fitting stagelowast(4) Choose1198701 EPIs from LF randomly(5) Calculate the best parameters (119886119894 119887119894) of each EPI by

using the algorithm mentioned in Section 32(6) Select the 1198860 which has the most frequency in all 119886119894 as

the common 119886 And fit the linear function of 119887 with 119910by using the 119887119894 whose 119886119894 equals to 1198860

(7) lowastFeature extracting stagelowast(8) Choose1198702 EPIs from LF randomly(9) Calculate the parameters (119886119894 119887119894) of each EPI by using

the common 119886 and the linear function of 119887 with 119910(10) Rectify each EPI with its (119886119894 119887119894) by using the algorithm

mentioned in Section 31(11) Extract the Lisad-1 feature points of all rectified EPIs(12) Count the ratio 119877 of the feature points which have

same slope value with 1198780 for each EPI and contruct athe feature vector by combining these ratios in adescending order

(13) Employ a classifier to distinguish flat surface fromnatural scenes

Algorithm 1 The proposed strategy for flat surface detection

then we shear each point in EPI with a different displacementaccording to the line function

119871119886119887 (119909 119906) = 1198710 (119909 + Δ119909 119906) Δ119909 = 119906 1119886119909 + 119887 (10)

where 1198710 denotes the input EPI and 119871119886119887 denotes the shearedEPI by two parameters (119886 119887) of the line function

In other words we just need to set the target slope 1198780as +infin in which the epipolar line is perpendicular to thehorizontal axis Two refocus results of ourmethod can be seenin Figure 5 as the data captured by us is a 3D light field (1angular dimension) there is only defocus blur in horizontaldirection and no defocus blur in vertical blur

35 Depth Estimation for Inclined Plane Similarly if thescene is a flat surface we can estimate its depth with thecommon 119886 and the function of 119887 obtained from the planefitting stage The detailed description is as follows

(i) Calculate the parameters (119886 119887) of the line function foreach EPI

(ii) Calculate the disparity of each point in EPI accordingto

dis (119909) = 1119886119909 + 119887 (11)

We detect the flat surface by using a small set of EPIs andwe fit the function of this plane at the same time With thefunction of the flat surface the depth map can be obtained bysubstituting the coordinate into the function of the plane

Different from the traditionalmultilabelingmethods [19]our depth estimation results are continuous since we knowthe function of the flat surface Two of our results are shownin Figure 6

4 Experimental Results

41 Experimental Setup To better analyze the performanceof our algorithm we select two different datasets The HCIlight field dataset [20] and its printed edition are selected toanalyze the properties of energy ratio It is noticed that theprinted photos are tilted to different degrees in order to better


(a) (b)

Figure 5 (a) is the refocus result by using the correct inclined plane parameters and it is an all-in-focus image (b) is the result by usingwrong parameters and it is noticed that there is only defocus blur in horizontal direction

(a) (b)

(c) (d)

Figure 6 (a) and (c) are two light fields captured from the printed Medieval dataset at different degrees (b) and (d) are the correspondingdepth maps estimated by using our plane function (red represents large depth value and blue represents small depth value) It can be seenthat the depth map is continuous and there are no noticeable ldquostairsrdquo in this result

evaluate our algorithm The experimental environment andprinted light fields can be seen in Figure 7 Apart from thisthe EPFL light field dataset [14] is selected to do a comparisonwith the previous work As this dataset is captured by a Lytrocamera the experimental results on this dataset can betterreflect the pros and cons of the algorithm

We implement the algorithm in the Matlab 2014b on OSX 10111 with 8 gigabytes of RAM and 27GHz of processorThe running time of our implementation for a 9 times 9 times 768 times768 times 3 light field is measured in seconds but does not excelthe time complexity of [14] This time can be accelerated tomicroseconds by using GPU

In the stage of determining line function we divide thesearching space of (119886 119887) into an 11 times 11 grid The target

slope 1198780 we hope to normalize to is not important in ourrectification actually it can be an arbitrary value In the planefitting stage we select 11 EPIs to obtain the function of theplane and select another 15 EPIs in the feature extractingstage The SVM classifier is employed to distinguish the flatsurface from the natural scenes

42 Analysis of the Energy Ratio We compute the energyratio for each EPI in the first dataset and plot their distribu-tion in Figure 9 The horizontal axis is the line number andthe vertical axis is the energy ratio for each EPI It can be seenthat the energy ratio of natural scenes may reach a high valuesometimes but mostly it is very small and far away from theflat surface Apart from this the energy ratio distribution of


Figure 7 The left top one is our experimental environment Others are printed light fields Note that the scene in the left top is just one ofthe different situations and our datasets contain 5 different tilt angles

Table 1 The detection accuracy comparison

Ours Ghasemi and Vetterli [14]Printed (P) 94 92Natural (N) 94 86

flat surface is not only large but also stable in contrast naturalscenes do not meet these properties

It is noticed that there are always some EPIs which havelarge energy ratios in original scenes (the first half of the bluecurve in Figure 8) The main reason for this phenomenon isthat these EPIs are selected to fit the plane function and thefitted plane function ismore suitable for these areasHoweverit may not be suitable for other EPIs (the second half of theblue curve in Figure 8) and this is the reason why we selectother EPIs to combine the feature vectors

43 Further Experiments and Comparison In order to betterevaluate the accuracy of our algorithm in real data capturedby light field camera we run our code in another publiclight field dataset captured by Ghasemi and Vetterli [14] Thedataset consists of 50 light fields of printed photos and 50 lightfields of natural scenes

To test and verify our algorithm in a classification settingwe used a SVM model with cross validation [21] The resultscan be seen in Table 1

Our detection precision is clearly superior to Ghasemiand Vetterlirsquos method [14] This improvement is prominentespecially in the detection of natural scenes where 7 naturalscenes are misclassified as flat surface in [14] and only 3 inour method We further analyze these failed data and find

that most textures of the scene lie on one continuous depthplane and there are a few textures on others The featurepoints which come from this continuous plane play a morevital role and lead to a higher energy ratio In Figure 8(a)most feature points lie on the cardboard and a few pointson the foreground the fitted plane is close to the cardboardplane and it leads to high energy ratio (the blue curve inFigure 8(e)) Then for the flat surface the value of wrongclassified samples is 3 as well By analyzing these samples itis noticed that the number of feature points is too little toestimate the plane function accurately In Figure 8(c) thereare more feature points in the bottom of the scene and afew points in the top which lead to a wrong fitting of theparameters 119887 in Section 33 Figures 8(e) and 8(f) show theenergy ratio distribution of these failed samples

5 Conclusion

In this paper we propose an algorithm to rectify the EPI ofa flat surface which normalizes the mess slope of epipolarlines in EPI into a same slope And this algorithm canbe easily expanded to the inclined plane refocus and thecontinuous depth estimation for flat surface Then we pro-pose a framework for flat surface detection which learns afunction of the flat surface by using several EPIs and tests thisfunction by using another several EPIs The results show thatour algorithm performs well for most scenes and the morecomplex the scene the better the performance

We may continue to study the limits of the algorithmssuch as in terms of low texture scene which leads to theinsufficient feature points


(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)

Figure 8 (a)ndash(d) are four failed samples of our algorithm where (a) and (b) are misclassified as flat surface and (c) and (d) are misclassifiedas natural scenes The green points are the Lisad-1 points (e) and (f) are the corresponding distributions of energy ratio


1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha

OriginalPrint 1Print 2Print 3Print 4Print 5

(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)

Figure 9 The energy ratio distribution for different light fields

Appendix

The proof of two properties mentioned in Section 33 isdescribed here

(1) Assume that the plane function is119860119909 + 119861119910 + 119862119911 + 119863 = 0 (A1)

(2) Substituting any fixed 119910lowast into the function (A1)becomes

119860119909 + 119862119911 = minus119861119910lowast minus 119863 lArrrArr119911 = minus119860119862119909 minus

119861119910lowast + 119863119862 (A2)

(3) The term minus119860119862 is replaced by 119886 and the term minus(119861119910lowast+119863)119862 is replaced by 119887119886 = 119860

119862 119887 = minus119861119862119910lowast minus

119863119862

(A3)

It can be found that the value of 119886 is a constant with differentfixed 119910lowast and the relationship of 119887 and 119910lowast is linear

Disclosure

The manuscript was presented as an abstract in a postersession of ICPR2016

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

The work in the paper is supported by research grant of StateKey Laboratory of Virtual Reality Technology and Systems(BUAA-VR-15KF-10)

References

[1] M Levoy and P Hanrahan ldquoLight field renderingrdquo in Proceed-ings of the 23rd annual conference on Computer graphics andinteractive techniques pp 31ndash42 ACM 1996

[2] S J Gortler R Grzeszczuk R Szeliski and M F CohenldquoLumigraphrdquo in Proceedings of the 23rd annual conference on


Computer graphics and interactive techniques pp 43ndash54 ACMAugust 1996

[3] N g Ren M Levoy M Bredif G Duval M Horowitz and PHanrahan ldquoLight field photography with a hand-held plenopticcamerardquo Computer Science Technical Report CSTR vol 2 no 112005

[4] U Perwass and P Christian ldquoDigital imaging system plenopticoptical device and image data processing methodrdquo US Patent2013

[5] C Birklbauer and O Bimber ldquoPanorama light-field imagingrdquoComputer Graphics Forum vol 33 no 2 pp 43ndash52 2014

[6] T E Bishop and P Favaro ldquoThe light field camera Extendeddepth of field aliasing and superresolutionrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 34 no 5 pp972ndash986 2012

[7] N Li J Ye Y Ji H Ling and J Yu ldquoSaliency detection on lightfieldrdquo in Proceedings of the 27th IEEE Conference on ComputerVision and Pattern Recognition CVPR 2014 pp 2806ndash2813 June2014

[8] P P Srinivasan M W Tao R Ng and R RamamoorthildquoOriented light-field windows for scene flowrdquo in Proceedingsof the 15th IEEE International Conference on Computer VisionICCV 2015 pp 3496ndash3504 December 2015

[9] A EAbdel-HakimandMEl-Saban ldquoFace authentication usinggraph-based low-rank representation of facial local structuresfor mobile vision applicationsrdquo in Proceedings of the 2011 IEEEInternational Conference on Computer Vision Workshops ICCVWorkshops 2011 pp 40ndash47 November 2011

[10] O Kahm and N Damer ldquo2D face liveness detection anoverviewrdquo in Proceedings of the 2012 International Conferenceof the Biometrics Special Interest Group BIOSIG 2012 pp 1ndash12September 2012

[11] J Zhao M Whitty and J Katupitiya ldquoDetection of non-flatground surfaces using V-disparity imagesrdquo in Proceedings of the2009 IEEERSJ International Conference on Intelligent Robotsand Systems IROS 2009 pp 4584ndash4589 October 2009

[12] T Wang A A Efros and R Ramamoorthi ldquoOcclusion-AwareDepth Estimation Using Light-Field Camerasrdquo in Proceedingsof the 2015 IEEE International Conference on Computer Vision(ICCV) pp 3487ndash3495 Santiago Chile December 2015

[13] M W Tao P P Srinivasan J Malik S Rusinkiewicz and RRamamoorthi ldquoDepth from shading defocus and correspon-dence using light-field angular coherencerdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern RecognitionCVPR 2015 pp 1940ndash1948 June 2015

[14] A Ghasemi and M Vetterli ldquoDetecting planar surface using alight-field camera with application to distinguishing real scenesfrom printed photosrdquo in Proceedings of the 2014 IEEE Interna-tional Conference on Acoustics Speech and Signal ProcessingICASSP 2014 pp 4588ndash4592 May 2014

[15] R Raghavendra K B Raja and C Busch ldquoPresentation attackdetection for face recognition using light field camerardquo IEEETransactions on Image Processing vol 24 no 3 pp 1060ndash10752015

[16] T Pingbo B Akinci and D Huber ldquoCharacterization of threealgorithms for detecting surface flatness defects from densepoint cloudsrdquo in Proceedings of the ISTSPIE Electronic Imaging2009

[17] I Tosic and K Berkner ldquo3d keypoint detection by light fieldscale-depth space analysisrdquo in Proceedings of the 2014 IEEEInternational Conference pp 1927ndash1931 2014

[18] R Ng ldquoFourier slice photographyrdquo in Proceedings of the ACMSIGGRAPH 2005 pp 735ndash744 ACM August 2005

[19] R Szeliski R Zabih D Scharstein et al ldquoA comparative studyof energy minimization methods for Markov random fieldswith smoothness-based priorsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 30 no 6 pp 1068ndash10802008

[20] S Wanner S Meister and B Goldluecke ldquoDatasets and bench-marks for densely sampled 4d light fieldsrdquo in Annual Workshopon Vision Modeling and Visualization VMV pp 225-226 2013

[21] A Yuichiro Pattern Recognition amp Machine Learning Elsevier2012

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of


International Journal of

RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal of

Volume 201

Submit your manuscripts athttpswwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



(119909 119910) and (119906 V) describe the distribution of the light inspatial and angular space respectively Under the two-planeparametrization when we fix one spatial dimension and oneangular dimension (119909lowast 119906lowast) (or (119910lowast Vlowast)) the EPI appearsFor each point in the real world there is a correspondingepipolar line in the EPI and the slope of this line has a linearrelationship with the depth of this point

As a basic problem in computer vision flat surfacedetection has been researched for decades but it is not welldeveloped Most of techniques detect flat surface using aprior depth estimation Zhao et al [11] proposed to detectflat surface using the disparity map of the scene however thismethod depends on the accuracy of disparity estimation andis sensitive to the alignment errors of disparity estimationRaghavendra et al [15] proposed a similar strategy Insteadof the disparity map used in [11] they proposed to obtain arough depth map by using the focus measure Undoubtedlythe method also suffers the same problem that is inaccuracyof disparity estimationGhasemi andVetterli [14] proposed toextract energy feature vector based on the change of gradientof EPI and to distinguish the flat surface from nonflat one byusing a Bayes classifier The method computes the slope of allepipolar lines and then takes the variance it is still a depth-based method

Different from the traditional ldquodepthmap to plane fittingrdquostrategy [11 15 16] we detect the Lisad-1 feature point [17]in 4D light field and then fit the function of the flat surfaceby using several feature points in light field directly finallythe robustness of the function which we build is tested byanother several feature points If the scene is a flat surfacethe function which is built from several feature points shouldalso be suitable for other points and vice versa

3 The Proposed Approach forDetecting Flat Surface

It is well observed that for a flat surface which is parallelto the camera plane all epipolar lines in the EPI have thesame slope since they are in the same depth However thisinvariant property went when the flat surface is tilted todifferent angles By analyzing the properties of flat surfacefunction it is noticed that no matter what angle the flatsurface is tilted to the difference of slope stays the same(this property is discussed in Section 33 and its proof can befound in Appendix) Based on this invariant property we firstpropose to rectify the slope of epipolar lines in EPI into a samevalue 1198780 and then to detect the Lisad-1 feature points [17] ofthe rectified EPI (the most important advantage of the Lisad-1 feature points is that it provides the slope of each featurepoint and the extraction of their depth does not suffer anyocclusion or shading problems as they are salient points)Theslope of each feature point ought to be equal to 1198780 if the planefunction is true Finally we combine the energy ratio fromdifferent EPIs as an energy vector and employ a classifier todistinguish flat surface from natural scenes

31 Epipolar Plane Image Rectification On a flat surface thedepth value of a point has a linear relationship with its 2D

Depth

＞ＪＮＢ = ax + b

X axis

Figure 1 The relationship between the horizontal axis 119909 in imageand its depth is linear if the scene is a flat surface

image coordinates When the point is in an EPI with a fixed119910lowast its depth can be expressed by a linear function only on 119909(Figure 1)

119889 (119909) = 119886119909 + 119887 (1)

where 119889(119909) is the depth of the point (119909 119910lowast) And we canderive the slope of each point by

119878 (119909) = 1dis (119909) =

1119891119904119889 (119909) =

119886119909 + 119887119891119904 (2)

where dis(119909) is the disparity of the point (119909 119910lowast) 119878(119909) is theslope of the point 119909 in EPI with the fixed 119910lowast 119891 is the focallength of the lens and 119904 is the baseline between two viewsAs the camera parameters 119891 and 119904 are constants they areignored Equation (2) can be rewritten as

119878 (119909) = 119886119909 + 119887 (3)

If the linear function is determined the slope of eachpoint in EPI can be known and the slope of epipolar lines inEPI can be normalize into a same value Suppose one point(119909 119906) in the original EPI its slope is 119878(119909lowast) where 119909lowast satisfiesthe following function

119909lowast + 119906 1119878 (119909lowast) = 119909 (4)

If the slope we hope to normalize to is 1198780 (it is called targetslope later) the target point of shearing (119909 119906) is (1199091015840 119906) where1199091015840 meets the following function

1199091015840 = 119909lowast + 119906 11198780 (5)

In a brief summary for a point (1199091015840 119906) in the rectified EPIthe corresponding point in the original EPI is

(1199091015840 minus 119906 11198780 + 1199061

119878 (1199091015840 minus 119906 (11198780)) 119906) (6)

We can refer to Figure 2 to understand the procedure ofrectification And Figure 3 gives an example of the originalEPI and the rectified EPI We can see that only one slope isremained in EPI after the rectification


(x u)

xlowast

((x u)


(a) (b)



119886 + 119887 ge 119889min119886 + 119887 le 119889max

119886119873 + 119887 ge 119889min119886119873 + 119887 le 119889max

(7)



119903 = sum|119865|119894=1 119892 (119878 (119865119894) = 1198780)|119865|

119892 (119909) = 1 119909 = 10 119909 = 1

(8)









119871120572 (119909 119906) = 1198710 (119909 + 119906(1 minus 1120572) 119906) (9)






N minus 1)

a


P2 (dＧＲ 0)

P1 (dＧＣＨ 0)



N minus 1)

Na + b = dＧＲ

Na + b = dＧＣＨ

b














119871119886119887 (119909 119906) = 1198710 (119909 + Δ119909 119906) Δ119909 = 119906 1119886119909 + 119887 (10)






dis (119909) = 1119886119909 + 119887 (11)






(a) (b)


(a) (b)

(c) (d)

















5 Conclusion




(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(x u)

xlowast

((x u)


(a) (b)



119886 + 119887 ge 119889min119886 + 119887 le 119889max

119886119873 + 119887 ge 119889min119886119873 + 119887 le 119889max

(7)



119903 = sum|119865|119894=1 119892 (119878 (119865119894) = 1198780)|119865|

119892 (119909) = 1 119909 = 10 119909 = 1

(8)









119871120572 (119909 119906) = 1198710 (119909 + 119906(1 minus 1120572) 119906) (9)






N minus 1)

a


P2 (dＧＲ 0)

P1 (dＧＣＨ 0)



N minus 1)

Na + b = dＧＲ

Na + b = dＧＣＨ

b














119871119886119887 (119909 119906) = 1198710 (119909 + Δ119909 119906) Δ119909 = 119906 1119886119909 + 119887 (10)






dis (119909) = 1119886119909 + 119887 (11)






(a) (b)


(a) (b)

(c) (d)

















5 Conclusion




(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












N minus 1)

a


P2 (dＧＲ 0)

P1 (dＧＣＨ 0)



N minus 1)

Na + b = dＧＲ

Na + b = dＧＣＨ

b














119871119886119887 (119909 119906) = 1198710 (119909 + Δ119909 119906) Δ119909 = 119906 1119886119909 + 119887 (10)






dis (119909) = 1119886119909 + 119887 (11)






(a) (b)


(a) (b)

(c) (d)

















5 Conclusion




(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b)


(a) (b)

(c) (d)

















5 Conclusion




(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation



















5 Conclusion




(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b)

(c) (d)1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(a)(b)

(e)

1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

(c)(d)

(f)



1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60

Buddha


(a)

1

09

Papillon

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(b)

StillLife1

09

08

07

06

05

04

03

02

01

00 10 20 30 40 50 60


(c)


Appendix





119861119910lowast + 119863119862 (A2)



119863119862

(A3)


Disclosure




Acknowledgments


References
























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation






























RoboticsJournal of





Journal of



RotatingMachinery



Journal of

Volume 201


VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









Epipolar Plane Image Rectification and Flat Surface Detection in...

Documents

Transcript of Epipolar Plane Image Rectification and Flat Surface Detection in...