ECE 661 Homework 7:

Instructor: Prof. Avi Kak

TA: Dave Kim

Tommy Chang

November 20, 2012

1 Introduction

The goal of this homework is to apply Zhang’s[2] camera calibration.

2 A Brief Outline of the Calibration Procedure

1. Place the camera at a marked location and record an image of the calibration pattern.The extrinsic pose parameters calculated will be for this location of the camera.

2. Move the camera and turn it so that it assumes different positions and orientation. Recordan image of the calibration pattern for each position/orientation. Note, the calibrationpattern is fixed at the same location on the wall. Record at least 20 images.

3. Extract the corners on the calibration pattern for all images.

4. Use the extracted corners and the known physical dimensions of the calibration patternto calculate the camera calibration parameters.

5. Verify the camera calibration parameters by re-project the physical corners from thecalibration pattern onto the images.

3 Setup

Figure 1 shows the marked location of the camera and its relative pose to the calibration patternon the wall. The placement of the camera is drawn as a cyan box. The camera coordinateframe is marked by the X,Y,Z axes. The image taken at this setup is shown in Figure 2 (Notethat the camera is place on its side.) The world coordinate frame is marked on this image aswell.

Section 7 shows the result of camera external parameters. The external parameters de-scribes how the world frame is located and oriented relative to the camera frame. For example,the external parameter having a translation vector t = [txtytz]

T means that the world frame islocated at (tx, ty, ty) in the camera coordinate. (i.e., with respect to the origin of the cameraframe.) In this setup, we would expect the external parameter to have a positive tz valuebecause the world frame is located in the +z direction of the camera frame.

1

(a) Camera Setup (b) Camera Setup in another view.

Figure 1: The blue color band and its segmented result.

Figure 2: Image taken by the camera placed at the setup position/orientation. The worldframe is shown by the 3 labeled axes at a corner – X is up, Y is to the right and Z is out ofthe picture. All corners are separated by 1.0 inch in both X and Y directions.

2

4 Corner Extraction

1. Apply the Canny edge detector to the image.

2. Use the Hough transform to fit straight lines to the Canny edges.

3. Initialize the corners as the intersections of the fitted straight lines.

4. Refine the initial corners to the actual corner locations if the radial distortion is severe.

4.1 Canny edge detector

I used the OpenCV Canny detector[5]. It has two required parameters:

1. first threshold: This parameters is used to find the initial strong edges. The default rangeof this parameter is from 0 to (9 * 255), ie. 255 times the default 3x3 kernel size. Thehigher the value, the less edges is found. The value I used is (1.5 * 255).

2. second threshold: This is used to find the weaker edges that are continuous from thestrong edges. The value I used is 255.

4.2 Hough Transform to detect lines and initial corners

I used the OpenCV HoughLines2[3] function. It has three required parameters:

1. rho: Distance resolution in pixel-related units. I used 1 pixel for this parameter.

2. theta: Angle resolution measured in radians. I used π/180 radians.

3. threshold: Threshold parameter. A line is returned by the function if the correspondingaccumulator value is greater than threshold. I used 50 pixels.

One common issue with Hough Transform is that the same line may be detected multipletimes. A Non-maxim suppression scheme is needed to achieve a clean result. My approach isthe following:

1. Sort lines detected by Hough Transform by their angles and group them into eithervertical or horizontal lines.

2. For each group, if two lines intersect inside the image, then just keep the line with morepixels (i.e., one with a larger accumulator value).

3. For each group, if two lines are too near each other, then just keep the line with morepixels. Since we know exactly how many lines to expect in each group, this nearnessthreshold can be dynamically set at the run-time to produce the number of expectedlines. Typical values found are < 5 pixels.

More specifically, the line distance in the vertical group is determined by the formulas:

dx = ρ cos(θ),

and similarly for the horizontal group:

dy = ρ sin(θ),

3

where ρ and θ are the line output from Hough Transform. Although OpenCV’s HoughTransform implementation does not return the accumulator values (i.e. number of pix-els on the line), this information can be indirectly obtained from the output ordering.Specifically, the output are sorted by the accumulator value so the first line in the outputhas the most number of pixels on it.

4.3 Refine corners to sub-pixel accuracy

Once the lines are extracted correctly from the Hough Transform, the initial location of thecorners can be computed as the intersections between the lines. However, these initial cornersmay not be exactly on the corners of the calibration pattern due to lens distortion – resultingphysically straight-lines becoming curved when projected onto the image.

OpenCV has an implementation of a sub-pixel corner detector[4]. This detector requiresan initial guess of the corner location. It iteratively solves a set of simultaneous equations topinpoint the exact corner location with sub-pixel accuracy. The other relevant parameter isthe sampling window size, which specifies a region around the initial guess. The true corner isassumed to be inside this sampling region.

At the run-time this window parameter can be dynamically set to the averaged distanceamong adjacent lines in the horizontal and vertical directions.

4.4 Results

Figure 4 shows four results of Canny edge detector, Hough Transform, initial corners, andrefined corners.

5 Intrinsic, Extrinsic, and Radial Distortion

5.1 Intrinsic parameters

The intrinsic parameters can be solved from the IAC (Image of the Absolute Conic) method[2][8].The relevant equation are given below:

ω The Image of the Absolute Conic

K The camera calibration matrix

H The planar homograph: image = H * points on Z=0

P The Projection matrix (transform onto the image)

�I, �J circular points

Ω∞ the absolute conic

�ITΩ∞�I circular points are on the absolute conic

ω = (K R)−TΩ∞(K R)−1 the transform of the absolute conic under P (i.e., to the image plane)

The last equation simplifies to:

ω = K−TK−1

The internal parameters (i.e., the camera calibration matrix), K, can be solved from the trans-formation of the absolute conic under H (i.e., to the image plane):

(H�I)Tω(H�I) = 0

4

To solve the above system of equations, at least 3 H’s are needed.

5.2 Extrinsic parameters

From the definition of the camera projection matrix, P, and a given homography H:

P �X = �x = K[R|�t] �XH �X = �x, where

�X a point in the world

�x the image of �X

R the 3x3 rotation matrix

�t the translation vector

the external parameters, R and �t can be solved.However, the rotation matrix obtained this way may not have the property: RTR = I. An

extra step to condition the rotation matrix is subsequently performed [2].

5.3 Radial distortion

Radial distortion can be modeled by the following process [8]:

1. Assume no skew and no external parameters in the projection matrix:

P = K =

⎡⎣αx 0 xo

0 αx yo0 0 1

⎤⎦

2. Backproject undistorted pixel (x,y) to the corresponding point (Xcam, Ycam, Zcam) in thecamera frame by taking the inverse transform:

⎡⎣Xcam

Ycam

1

⎤⎦ = K−1

⎡⎣xy1

⎤⎦

Note Zcam = 1

3. Apply radial distortion parameters (k1 and k2) to get the distorted coordinate (Xdcam,

Y dcam):

r2 = X2cam + Y 2

cam

Xdcam = Xcam (1 + k1 r2 + k2 r4)

Y dcam = Ycam (1 + k1 r2 + k2 r4)

4. Finally re-project the distorted coordinate onto the pixel (xd, yd):

⎡⎣xd

yd1

⎤⎦ = K

⎡⎣Xd

cam

Y dcam

1

⎤⎦

5

6 Refining the Calibration

The idea is to have a global optimal calibration by minimizing the re-projection error of allthe data taken.

Let �XM,j be the jth corner of the calibration pattern lying on the world Z=0 plane (i.e,

the model plane). And let �xM,i,j be the projected pixel of �XM,j . Let �xi,j be the jth corner onthe ith image.

Then, the goal is to minimize the geometric distance:

d2 =∑i

∑j

‖�xi,j − �XM,i,j‖2

This can be done by using a none-linear least squares optimization method such as theLevenberg-Marquartdt method.[7]

�xM,i,j is a function of all the calibration parameters to be optimized as well as the corneron the model plane:

d2 =∑i

∑j

‖�xi,j − f(K,�ri, ti, k1, k2, �XM,j)‖2,

where �ri is the Rodrigues representation of the rotation matrix Ri. The Rodrigues representa-tion (OpenCV has built-in Rodrigues implementation[6].) is a compact rotation representationand has only 4 parameters consisting of an axis of rotation and angle of rotation. It is im-portant not to use the 3x3 rotation matrix directly as they would be treated as independentvariables during the optimization.

6.1 Results

Below are the result of global optimization.

• On provided 40-images data set:

Interal parameters:K =

⎡⎣727.825389 1.0363175 321.665895

0. 728.94595 240.3882950. 0. 1.

⎤⎦

Radial Distortion : k1 = −0.15726582388, k2 = 0.10701098002

Max re-projection error = 1.59 pixels

• On my 20-images data set:

Interal parameters:K =

⎡⎣705.640229 0.463950353 312.397783

0. 703.551374 235.2003420. 0. 1.

⎤⎦

Radial Distortion : k1 = −0.3390339977, k2 = 0.50399862554

Max re-projection error = 1.63 pixels

6

7 Assessment of Calibration Accuracy

To visually assess the calibration accuracy, the physical calibration pattern (ie., corners) in theworld frame are projected back into the images. The projection uses the globally optimizedcamera parameters as well as the distortion parameters.

7.1 Results

Figures 5 to 8 show 4 reprojected result and the corresponding external parameters.

7

8 Figures

Figure 3: Four images from the provided data set. First column = Canny edge detector.Second column = Hough Transform. Third column = initial corns as intersections of lines.Fourth column = refined corner with sub-pixel accuracy

8

Figure 4: Four images from data set taken with my cheap camera. The columns are orderedthe same way as in Figure 4. A noticeable radial distortion can be seen in the last image.

9

External parameters:[R|�t] =⎡⎣

0.95737211 0.26821168 −0.10724339 | −5.125344940.25537448 −0.95940575 −0.11968494 | 2.11675885−0.13499082 0.0871958 −0.98700272 | 14.33029119

⎤⎦

Figure 5: Reprojected result and the external parameters for frame1 of my data set.

10

External parameters:[R|�t] =⎡⎣0.86638487 0.25469514 0.42954353 | −3.374312830.33312942 −0.93557163 −0.11717729 | 2.313620120.37202425 0.24461422 −0.89541155 | 12.97031149

⎤⎦

Figure 6: Reprojected result and the external parameters for frame2 of my data set.

11

External parameters:[R|�t] =⎡⎣

0.98067968 −0.15187249 −0.12329685 | −2.65356255−0.14402305 −0.98707348 0.07030859 | 5.23780151−0.132381 −0.05119262 −0.98987605 | 18.76736432

⎤⎦

Figure 7: Reprojected result and the external parameters from the provided data set.

12

External parameters:[R|�t] =⎡⎣

0.87925714 −0.18547251 0.43875601 | −2.21539713−0.20586596 −0.97857956 −0.00111792 | 4.407552620.42956501 −0.08934199 −0.89860554 | 18.40556658

⎤⎦

Figure 8: Reprojected result and the external parameters from the provided data set.

13

References

[1] Homework 7 problem set: http://web.ics.purdue.edu/~kim497/ece661/homework/

hw7.pdf

[2] Zhengyou Zhang, “A Flexible New Technique for Camera Calibration”, Technical ReportMSR-TR-98-71, Microsoft Corporation

[3] cv.HoughLines2 OpenCV documentation and tutorial, http://docs.opencv.org/doc/

tutorials/imgproc/imgtrans/hough_lines/hough_lines.html#hough-lines

[4] cv.FindCornerSubPix OpenCV documentation, http://opencv.willowgarage.

com/documentation/python/imgproc_feature_detection.html?highlight=

findcornersubpix#FindCornerSubPix

[5] cv.Canny OpenCV documentation, http://opencv.willowgarage.com/documentation/python/imgproc_feature_detection.html?highlight=canny#Canny

[6] cv.Rodrigues2 OpenCV documentation, http://opencv.willowgarage.com/

documentation/python/calib3d_camera_calibration_and_3d_reconstruction.

html?highlight=rodrigues#Rodrigues2

[7] Numpy and Scipy Documentation: Non-Linear Least Squares http://docs.scipy.org/

doc/scipy/reference/tutorial/optimize.html#least-square-fitting-leastsq

[8] R. Hartley, and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd, Cam-bridge University Press., 2004.

A Source Code Listing

1 #!/ usr /bin /python2 #3 # ECE 661 Homework 74 # In s t r u c t o r : Prof . Avi Kak5 # TA: Dave Kim6 #7 # Tommy Chang8 #9 # 2012−11−06 (Tue)

10 # http ://web . i c s . purdue . edu/˜kim497/ ece661 /homework/hw7 . pdf11

12 from pylab import ∗13 import cv , cv2 , time , os , re14 from s c ipy . opt imize import l e a s t s q15

16 global f on t17 f on t = cv . In i tFont ( cv .CV FONT HERSHEY SIMPLEX, 0 . 5 , 0 . 5 , t h i c kne s s=2)18

19 windowNames = se t ( [ ] )20

21 def CloseWindowNow (name=”window” ) :22 cv . DestroyWindow (name)23 for idx in range (0 , 4 ) : # hack to c l o s e the window proper ly f o r opencv 2 .124 cv .WaitKey (1000)25

26 def CloseAllWindowNow ( ) :

14

27 global windowNames28 nameList = l i s t (windowNames )29 for idx in range ( l en ( nameList ) ) :30 cv . DestroyWindow ( nameList [ idx ] )31 for idx in range (0 , 4 ) : # hack to c l o s e the window proper ly f o r opencv 2 .132 cv .WaitKey (1000)33

34 def DisplayImage ( imgOrMat , name=None ) :35 global windowNames36 i f (name == None ) :37 # func t i onCa l l S t r = in sp e c t . s tack ( ) [ 1 ] [ 4 ] [ 0 ]38 # name = re . sub ( r ’ . ∗ \ ( ( . ∗ ? ) \) . ∗ ’ , r ’\1 ’ , f un c t i onCa l l S t r )39 # name = name . r s t r i p (”\n”)40 name = ”window”41 windowNames . add (name)42 cv . ShowImage (name , imgOrMat)43 cv .WaitKey (1000)44

45 def LoadImage ( imgFileName ) :46 image = cv . LoadImageM ( imgFileName , cv .CV LOAD IMAGE COLOR)47 return image48

49 def GetImageFi les ( dirName , extName ) :50 imageF i l e s = os . l i s t d i r ( dirName )51 imageF i l e s = f i l t e r (lambda x : re . match ( r ’ . ∗ \ . ’+extName , x ) , imageF i l e s )52 imageF i l e s = [ dirName + ”/” + imageFi l e s [ idx ] for53 idx in range ( l en ( imageF i l e s ) ) ]54 return imageF i l e s55

56

57 def OverlayHoughLines ( l i n e s , image ) :58 newImage = cv . CloneMat ( image )59 for idx in xrange ( l en ( l i n e s ) ) :60 # get l i n e in po la r coord inate :61 rho = l i n e s [ idx ] [ 0 ]62 theta = l i n e s [ idx ] [ 1 ]63

64 # the cente r po int on the l i n e :65 cosa = cos ( theta ) ; s i na = s i n ( theta )66 x0 = cosa ∗ rho67 y0 = s ina ∗ rho ;68

69 # Need the two extreme end po in t s :70 # s in c e dot product g i v e s us : x cosa + y s ina = r71 # the s l ope i n t e r s e c t−form i s : y = (−cosa / s ina ) x + ( r / s ina )72 # so l v i n g the s l ope as : de l tay / de l tax = (−cosa / s ina )73 # Note : l i n e can draw out s id e the image border .74 pt1x = in t ( x0 + 1000 ∗ s i na )75 pt1y = in t ( y0 − 1000 ∗ cosa )76 pt2x = in t ( x0 − 1000 ∗ s i na )77 pt2y = in t ( y0 + 1000 ∗ cosa )78 cv . Line ( newImage , ( pt1x , pt1y ) , ( pt2x , pt2y ) , cv .RGB(0 , 0 , 255) , 2)79 return newImage80

81 def OverlayLabelCorners ( corners , image , c o l o r=cv .RGB(0 , 0 , 255) ) :82 image = cv . CloneMat ( image )83 nRows = len ( co rne r s )84 nCols = len ( co rne r s [ 0 ] )85 corne r Id = 086 for r Idx in xrange (nRows) :87 for cIdx in xrange ( nCols ) :88 c en t e r = tup l e (map ( int , c o rne r s [ r Idx ] [ cIdx ] ) )

15

89 cv . C i r c l e ( image , center , 4 , (0 , 0 , 255) , −1)90 cv . PutText ( image , s t r ( co rne r Id ) , center , font , cv .RGB(0 , 255 , 0) )91 corne r Id += 192 return image93

94 def OverlayCorners ( corners , image , s i z e =4, c o l o r=cv .RGB(0 , 0 , 255) ) :95 # here co rne r s i s s imply a l i s t o f co rne r s in (x , y )96 image = cv . CloneMat ( image )97 for idx in xrange ( l en ( co rne r s ) ) :98 c en t e r = tup l e (map ( int , c o rne r s [ idx ] ) )99 cv . C i r c l e ( image , center , s i z e , (0 , 0 , 255) , −1)

100 return image101

102 def GetHoughLineinHC ( houghLine ) :103 rho = houghLine [ 0 ]104 theta = houghLine [ 1 ]105 pt0 = array ( [ rho ∗ cos ( theta ) , rho ∗ s i n ( theta ) , 1 . 0 ] )106 pt1 = array ( [ pt0 [ 0 ] + 100 ∗ s i n ( theta ) ,107 pt0 [ 1 ] − 100 ∗ cos ( theta ) ,108 1 . 0 ] )109 lineHC = c ro s s ( pt0 , pt1 )110 return lineHC111

112 def RealCoord (v ) :113 # convert a image coord ina te to Homogeneous coord ina te :114 i f type (v ) == ndarray :115 i f v . ndim == 1 :116 v [ 0 ] /= v [ 2 ]117 v [ 1 ] /= v [ 2 ]118 v [ 2 ] = 1119 else :120 v [ 0 ] [ 0 ] /= v [ 2 ] [ 0 ]121 v [ 1 ] [ 0 ] /= v [ 2 ] [ 0 ]122 v [ 2 ] [ 0 ] = 1123 e l i f type (v ) == cv2 . cv . cvmat :124 v [ 0 , 0 ] /= v [ 2 , 0 ]125 v [ 1 , 0 ] /= v [ 2 , 0 ]126 v [ 2 , 0 ] = 1127 else :128 a s s e r t (0 )129 return v130

131 def FindCheckerLines ( l i n e s , checkerDim , imageDim) :132 # checkerDim i s (m, n) where m i s the number rows ( eg , 4) o f b lack133 # squares . and n i s number o f columns ( eg 5) o f b lack squares .134 #135 # imageDim i s dimension o f the image ( eg , 480 rows , 640 c o l s )136 #137 # This func t i on r e tu rn s a l i s t o f hough l ine s . Non−maxima138 # suppre s s i on i s done by making sure no l i n e s o f the same group139 # ( v e r t i c a l or ho r i z on t a l ) i n t e r s e c t on the image . The l i n e ’ s140 # qua l i t y i s determined by the order in the l i n e s . The l ong e s t141 # l i n e appears f i r s t . ( i e . low ”badness ” value )142 #143 # Group l i n e s i n to e i t h e r ho r i z on t a l or v e r t i c a l :144 the ta s = array ( [ l i n e s [ idx ] [ 1 ] for idx in range ( l en ( l i n e s ) ) ] )145 the ta s −= pi /2 # ho r i z on t a l l i n e s i s [ −45 . . 4 5 ]146 l ineBadness = range ( l en ( the ta s ) )147 hIdxs = map ( int , where ( abs ( the ta s ) < pi /4) [ 0 ] )148 horzLines = [ l i n e s [ idx ] for idx in hIdxs ]149 hLineBadness = [ l ineBadness [ idx ] for idx in hIdxs ]150 vIdxs = map ( int , where ( abs ( the ta s ) >= pi /4) [ 0 ] )

16

151 ve r tL ine s = [ l i n e s [ idx ] for idx in vIdxs ]152 vLineBadness = [ l ineBadness [ idx ] for idx in vIdxs ]153 a s s e r t ( l en ( hIdxs ) + len ( vIdxs ) == len ( the ta s ) )154

155 # Sort each group by d i s tance , a s l o in c lude the badness measure :156 hLineAndBadness = z ip ( hLineBadness , horzL ines )157 horzLines = sor t ed ( horzLines , key=lambda elm : elm [ 0 ] ∗ s i n ( elm [ 1 ] ) )158 hLineAndBadness = sor t ed ( hLineAndBadness ,159 key=lambda elm : elm [ 1 ] [ 0 ] ∗ s i n ( elm [ 1 ] [ 1 ] ) )160 vLineAndBadness = z ip ( vLineBadness , v e r tL ine s )161 ve r tL ine s = sor t ed ( ver tL ines , key=lambda elm : elm [ 0 ] ∗ cos ( elm [ 1 ] ) )162 vLineAndBadness = sor t ed ( vLineAndBadness ,163 key=lambda elm : elm [ 1 ] [ 0 ] ∗ cos ( elm [ 1 ] [ 1 ] ) )164 #165 # Supress bad l i n e s , group based on the l i n e nearnes s and i n t e r s e c t i o n :166 def GroupLines ( hvLinesWithBadness , sinCos , minSep ) :167 uniqueLines = [ ]168 uniqueLinesBadness = [ ]169 # pr in t ” l i n e s in a group ” , l en ( hvLinesWithBadness )170 hvLine = hvLinesWithBadness [ 0 ] [ 1 ]171 uniqueLines . append ( hvLine )172 badness = hvLinesWithBadness [ 0 ] [ 0 ]173 uniqueLinesBadness . append ( badness )174 for idx in xrange (1 , l en ( hvLinesWithBadness ) ) :175 hvLine = hvLinesWithBadness [ idx ] [ 1 ]176 badness = hvLinesWithBadness [ idx ] [ 0 ]177 lineHC = GetHoughLineinHC ( hvLine )178

179 # check i n t e r s e c t i o n180 curLineHC = GetHoughLineinHC ( hvLine )181 uniqueLineHC = GetHoughLineinHC ( uniqueLines [−1])182 corner = c r o s s ( curLineHC , uniqueLineHC )183 RealCoord ( corner )184 # pr in t ”Corner ” , corner185 i f ( ( ( corner [ 0 ] >=0) and ( corner [ 0 ] < imageDim [ 1 ] ) ) and186 ( ( corner [ 1 ] >=0) and ( corner [ 1 ] < imageDim [ 0 ] ) ) ) :187 # pr in t ” sk ip corner c r o s s ”188 i f ( uniqueLinesBadness [−1] > badness ) :189 uniqueLines [−1] = hvLine190 uniqueLinesBadness [−1] = badness ;191 continue192

193 # check d i s t anc e194 uniqueLineDist = uniqueLines [ −1 ] [ 0 ] ∗ s inCos ( uniqueLines [ − 1 ] [ 1 ] )195 curDis t = hvLine [ 0 ] ∗ s inCos ( hvLine [ 1 ] )196 de l t aD i s t = abs ( uniqueLineDist − curDis t )197 # pr in t ” de l t aD i s t =”, d e l t aD i s t198 i f ( d e l t aD i s t <= minSep ) :199 # pr in t ” sk ip near ”200 i f ( uniqueLinesBadness [−1] > badness ) :201 uniqueLines [−1] = hvLine202 uniqueLinesBadness [−1] = badness ;203 continue204

205 # new group s t a r t s :206 uniqueLines . append ( hvLine )207 uniqueLinesBadness . append ( badness )208 sepDis t = de l t aD i s t209 # pr in t ” newl ine ” , l en ( uniqueLines )210 # pr in t hvLine211 return ( uniqueLines , sepDis t )212

17

213

214 # f ind the best s epa ra t i on d i s t ance :215 minSep = 1 .0216 isDone = False217 while isDone == False :218 ( uniqueHl ines , hSepDist ) = GroupLines ( hLineAndBadness , s in , minSep )219 ( uniqueVl ines , vSepDist ) = GroupLines ( vLineAndBadness , cos , minSep )220 i f ( ( l en ( un iqueHl ines ) == 2 ∗ checkerDim [ 0 ] ) and221 ( l en ( un iqueVl ines ) == 2 ∗ checkerDim [ 1 ] ) ) :222 isDone = True223 e l i f ( ( l en ( un iqueHl ines ) < 2 ∗ checkerDim [ 0 ] ) or224 ( l en ( un iqueVl ines ) < 2 ∗ checkerDim [ 1 ] ) ) :225 print ” f i l t e r too much Can ’ t s epara t e ??? ”226 a s s e r t (0 )227 isDone = True228 else :229 minSep += 1230 print ”minSep” , minSep231

232 a s s e r t ( l en ( un iqueHl ines ) >= 2 ∗ checkerDim [ 0 ] )233 a s s e r t ( l en ( un iqueVl ines ) >= 2 ∗ checkerDim [ 1 ] )234 bes tL ine s = uniqueHl ines + uniqueVl ines235 sepDis t = in t ( ( hSepDist + vSepDist ) / 2 . 0 )236 print ” sepDis t=” , sepDis t237 return ( bestL ines , sepDist )238

239

240 def ReshapeCorners ( corner sSet , dim) :241 # dim = (number or rows , number o f columns )242 #243 # convert 1D to 2D l i s t :244 nRows = dim [ 0 ]245 nCols = dim [ 1 ]246 co rne r s = [ [ c o rne r sSe t [ i ∗nCols+j ] for j in range ( nCols ) ]247 for i in range (nRows) ]248 return co rne r s249

250

251 def FindCheckerCorners ( bestL ines , sepDist , checkerDim ) :252 # checkerDim i s (m, n) where m i s the number rows ( eg , 4) o f b lack253 # squares . and n i s number o f columns ( eg 5) o f b lack squares .254 #255 # This func t i on r e tu rn s a 2D array . Each elment i s a corner in a256 # tup le (x , y ) . Note : x i s column and y i s row . Use257 # ReshapeCorners ( ) to convert to 1D array i f needed .258 #259 # Group l i n e s i n to e i t h e r ho r i z on t a l or v e r t i c a l :260 the ta s = array ( [ be s tL ine s [ idx ] [ 1 ] for idx in range ( l en ( be s tL ine s ) ) ] )261 the ta s −= pi /2262 hIdxs = where ( abs ( the ta s ) < pi /4) [ 0 ]263 horzLines = [ be s tL ine s [ idx ] for idx in hIdxs ]264 vIdxs = where ( abs ( the ta s ) >= pi /4) [ 0 ]265 ve r tL ine s = [ be s tL ine s [ idx ] for idx in vIdxs ]266 a s s e r t ( l en ( hIdxs ) + len ( vIdxs ) == len ( the ta s ) )267

268 nHl ines = len ( horzLines )269 nVl ines = len ( ve r tL ine s )270 print nHlines , nVl ines271 a s s e r t ( nHl ines == 2 ∗ checkerDim [ 0 ] )272 a s s e r t ( nVl ines == 2 ∗ checkerDim [ 1 ] )273

274 # Find the i n t e r s e c t i o n o f l i n e s :

18

275 co rne r s = [ [ ( ) for j in range ( nVl ines ) ] for i in range ( nHl ines ) ]276 horzLines = sor t ed ( horzLines , key=lambda elm : elm [ 0 ] ∗ s i n ( elm [ 1 ] ) ,277 r e v e r s e=True )278 ve r tL ine s = sor t ed ( ver tL ines , key=lambda elm : elm [ 0 ] ∗ cos ( elm [ 1 ] ) ,279 r e v e r s e=False )280 for hIdx in xrange ( nHl ines ) :281 hLineHC = GetHoughLineinHC ( horzLines [ hIdx ] )282 for vIdx in xrange ( nVl ines ) :283 vLineHC = GetHoughLineinHC ( ve r tL ine s [ vIdx ] )284 # do l i n e i n t e r s e c t i o n :285 corner = c r o s s (hLineHC , vLineHC)286 RealCoord ( corner )287 co rne r s [ hIdx ] [ vIdx ] = ( corner [ 0 ] , co rner [ 1 ] )288

289 return co rne r s290

291

292 def Ref ineCorners ( gray , corners , wSize ) :293 corners 1D = [ corner for row in co rne r s for corner in row ]294 corners 1D = cv . FindCornerSubPix ( gray , corners 1D ,295 ( wSize /2 , wSize /2) , (2 , 2) ,296 ( cv .CV TERMCRIT ITER |297 cv .CV TERMCRIT EPS,298 10 , 0 . 0 1 ) )299 corners 2D = ReshapeCorners ( corners 1D , ( l en ( co rne r s ) , l en ( co rne r s [ 0 ] ) ) )300 return corners 2D301

302 def SolveHomographyByDLT V2 ( xyDataSet ) :303 # Solve the Homography by us ing Least Square Method f o r304 # homogeneous equat ions .305 #306 # xyDataSet i s a s e t o f correspondence pa i r s . I t i s a l i s t o f307 # su b l i s t s o f the form : [ (X1 , Y1) , (X2 , Y2) ] Here , (X1 , Y1) , (X2 ,308 # Y2) i s a correspondance pa i r .309 #310 # We want to cons t ruc t a H such that Data2 = H Data1311 nData = len ( xyDataSet )312 a s s e r t ( nData >= 5)313

314 # precond i t i on the data f i r s t to avoid numerica l p r e c i s i o n c u t o f f :315 def Precondit ionData ( xyDataSet ) :316 # xyDataSet i s a s e t o f correspondence pa i r s . I t i s a l i s t o f317 # su b l i s t s o f the form : [ (X1 , Y1) , (X2 , Y2) ] Here , (X1 , Y1) , (X2 ,318 # Y2) i s a correspondance pa i r .319 #320 # 1 . ) f i nd the cente r :321 nData = len ( xyDataSet )322 data1X = [ xyDataSet [ idx ] [ 0 ] [ 0 ] for idx in range ( nData ) ]323 data1Y = [ xyDataSet [ idx ] [ 0 ] [ 1 ] for idx in range ( nData ) ]324 data2X = [ xyDataSet [ idx ] [ 1 ] [ 0 ] for idx in range ( nData ) ]325 data2Y = [ xyDataSet [ idx ] [ 1 ] [ 1 ] for idx in range ( nData ) ]326 al l1X = array ( data1X )327 al l1Y = array ( data1Y )328 mean1X = in t ( a l l1X .mean ( ) )329 mean1Y = in t ( a l l1Y .mean ( ) )330 al l2X = array ( data2X )331 al l2Y = array ( data2Y )332 mean2X = in t ( a l l2X .mean ( ) )333 mean2Y = in t ( a l l2Y .mean ( ) )334 # 2 . ) make data r e l a t i v e to the cen te r :335 def o f f s e tCoord (datum) :336 X1 = datum [ 0 ] [ 0 ]

19

337 Y1 = datum [ 0 ] [ 1 ]338 X2 = datum [ 1 ] [ 0 ]339 Y2 = datum [ 1 ] [ 1 ]340 return [ (X1 − mean1X , Y1 − mean1Y) ,341 (X2 − mean2X , Y2 − mean2Y) ]342 xyDataSet = map ( o f f s e tCoord , xyDataSet )343 # 3 . ) The pre−c ond i t i o n a l t rans fo rmat ion :344 H1 cond = array ( [ [ 1 , 0 , −mean1X ] ,345 [ 0 , 1 , −mean1Y ] ,346 [ 0 , 0 , 1 ] ] , dtype=f l o a t )347 H2 cond = array ( [ [ 1 , 0 , −mean2X ] ,348 [ 0 , 1 , −mean2Y ] ,349 [ 0 , 0 , 1 ] ] , dtype=f l o a t )350 return ( xyDataSet , H1 cond , H2 cond )351 ( xyDataSet cond , H1 cond , H2 cond ) = Precondit ionData ( xyDataSet ) ;352

353 # Generate the des ign matrix A:354 def GenerateArow ( pa i r ) :355 X1 = pa i r [ 0 ] [ 0 ]356 Y1 = pa i r [ 0 ] [ 1 ]357 X2 = pa i r [ 1 ] [ 0 ]358 Y2 = pa i r [ 1 ] [ 1 ]359 return [ 0 , 0 , 0 , −X1 , −Y1 , −1, Y2∗X1 , Y2∗Y1 , Y2 ,360 X1 , Y1 , 1 , 0 , 0 , 0 , −X2∗X1 , −X2∗Y1 , −X2 ]361 A = array (map (GenerateArow , xyDataSet cond ) , dtype=f l o a t )362 A = A. reshape ( nData ∗2 , 9)363

364 # SVD decomposit ion : ( beca r e fu l , i f u se ing e igen decomposit ion ,365 # the e igen ve c t o r s are in columns , not row )366 (U, D, V) = svd (A)367 minSvalIdx = D. argmin ( )368 # (D , U ) = e i g ( dot ( t ranspose (A) , A) )369 # U = transpose (U )370 # minEvalIdx = D . argmin ( )371

372 # get the minimum e igen vec to r :373 a s s e r t (D[ minSvalIdx ] > 0)374 minEigVec = V[ minSvalIdx ]375 H = minEigVec . reshape (3 , 3)376

377 # undo the pre−data cond i t i on ing378 H = dot ( inv (H2 cond ) , dot (H , H1 cond ) )379 return H380

381 def GetCorrespondencePairs ( corners 2D , cornerSep ) :382 corners 1D = [ corner for row in corners 2D for corner in row ]383 nRows = len ( corners 2D )384 nCols = len ( corners 2D [ 0 ] )385 co rne rPa i r s = [ ( ) ] ∗ (nRows ∗ nCols )386 idx = 0387 for r Idx in range (nRows) :388 for cIdx in range ( nCols ) :389 x model = f l o a t ( cIdx ) ∗ cornerSep390 y model = f l o a t ( rIdx ) ∗ cornerSep391 co rne rPa i r s [ idx ] = [ ( x model , y model ) ,392 corners 2D [ rIdx ] [ cIdx ] ]393 idx += 1394 return co rne rPa i r s395

396 def SolveIAC (Hs) :397 # Solve f o r the image o f abo lute con i c (IAC) , aka w.398 # Here Hs i s a l i s t o f homographies , each from a image . image = H ∗ world

20

399 # See Zhengyou Zhang ’ s paper about Camera Ca l i b ra t i on .400 def GenerateTwoRows (H) :401 def GetV ( h i vec , h j v e c ) :402 return array ( [ h i v e c [ 0 ] ∗ h j ve c [ 0 ] ,403 h i v e c [ 0 ] ∗ h j ve c [ 1 ] + h i v e c [ 1 ] ∗ h j ve c [ 0 ] ,404 h i v e c [ 0 ] ∗ h j ve c [ 2 ] + h i v e c [ 2 ] ∗ h j ve c [ 0 ] ,405 h i v e c [ 1 ] ∗ h j ve c [ 1 ] ,406 h i v e c [ 1 ] ∗ h j ve c [ 2 ] + h i v e c [ 2 ] ∗ h j ve c [ 1 ] ,407 h i v e c [ 2 ] ∗ h j ve c [ 2 ] ] )408 h1 vec = H[ : , 0 ]409 h2 vec = H[ : , 1 ]410 v12 vec = GetV ( h1 vec , h2 vec )411 v11 vec = GetV ( h1 vec , h1 vec )412 v22 vec = GetV ( h2 vec , h2 vec )413 return array ( [ v12 vec ,414 v11 vec − v22 vec ] )415 a s s e r t ( l en (Hs) >= 3) # we need at l e a s t 3 images to s o l v e IAC416 V = array (map (GenerateTwoRows , Hs) , dtype=f l o a t )417 V = [ row for twoRows in V for row in twoRows ] # f l a t t e n418

419 # SVD decomposit ion : ( beca r e fu l , i f u se ing e igen decomposit ion ,420 # the e igen ve c t o r s are in columns , not row )421 (U, D, Vt) = svd (V)422 minSvalIdx = D. argmin ( )423

424 # get the minimum e igen vec to r :425 a s s e r t (D [ minSvalIdx ] > 0)426 minEigVec = Vt [ minSvalIdx ]427

428 # recon s t ru c t the IAC ( symmetric )429 w = ze ro s ( ( 3 , 3 ) )430 w[ 0 , 0 ] = minEigVec [ 0 ]431 w[ 0 , 1 ] = minEigVec [ 1 ]432 w[ 0 , 2 ] = minEigVec [ 2 ]433 w[ 1 , 0 ] = w[ 0 , 1 ]434 w[ 1 , 1 ] = minEigVec [ 3 ]435 w[ 1 , 2 ] = minEigVec [ 4 ]436 w[ 2 , 0 ] = w[ 0 , 2 ]437 w[ 2 , 1 ] = w[ 1 , 2 ]438 w[ 2 , 2 ] = minEigVec [ 5 ]439

440 return w441

442 def SolveCameraCal ibrat ionMatr ix (w) :443 # w i s the Image o f the Absolute Conic (IAC)444 # See Zhengyou Zhang ’ s paper about Camera Ca l i b ra t i on .445 yo = (w[ 0 , 1 ] ∗w[ 0 , 2 ] − w[ 0 , 0 ] ∗w[ 1 , 2 ] ) / (w[ 0 , 0 ] ∗w[ 1 , 1 ] − w[ 0 , 1 ] ∗w[ 0 , 1 ] )446 lam = w[ 2 , 2 ] − \447 (w[ 0 , 2 ] ∗w[ 0 , 2 ] + yo ∗ (w[ 0 , 1 ] ∗w[ 0 , 2 ] − w[ 0 , 0 ] ∗w[ 1 , 2 ] ) ) / w[ 0 , 0 ]448 alpha x = sq r t ( lam / w[ 0 , 0 ] )449 alpha y = sq r t ( lam∗w[ 0 , 0 ] / (w[ 0 , 0 ] ∗w[ 1 , 1 ] − w[ 0 , 1 ] ∗w[ 0 , 1 ] ) )450 s = − (w[ 0 , 1 ] ∗ alpha x ∗ alpha x ∗ alpha y ) / lam451 xo = ( s ∗yo ) / a lpha y − (w[ 0 , 2 ] ∗ alpha x ∗ alpha x ) / lam452 K = ze ro s ( ( 3 , 3 ) )453 K[ 0 , 0 ] = alpha x454 K[ 0 , 1 ] = s455 K[ 0 , 2 ] = xo456 K[ 1 , 0 ] = 0457 K[ 1 , 1 ] = alpha y458 K[ 1 , 2 ] = yo459 K[ 2 , 0 ] = 0460 K[ 2 , 1 ] = 0

21

461 K[ 2 , 2 ] = 1462 return K463

464 def SolveExternalParameters (K, Hs) :465 # K i s the i n t r i s i c parameter , Hs i s a l i s t o f p lanar466 # homographies .467 #468 # Return a l i s t o f 3x4 matrix homogeous matrix T that i s the world469 # pose in camera ’ s po int o f view . i e . , T takes world coo rd ina te470 # and trans form to camera coord inate cente red at the camera cente r471 # ( i . e . , c en t e r o f p r o j e c t i o n )472 #473 K inv = inv (K)474 def SolveT (H) :475 h1 vec = H[ : , 0 ]476 h2 vec = H[ : , 1 ]477 h3 vec = H[ : , 2 ]478

479 # Since tz i s invo lved in normal i z ing the homogeous coord inate480 # la t e r , we make the norma l i za t i on d iv id e by a p o s i t i v e va lue .481 # This w i l l make the camera frame and and the ex t e rna l482 # parameters c on s i s t an t .483 t v e c = dot (K inv , h3 vec )484 mag = norm ( dot (K inv , h1 vec ) )485 i f ( t v e c [ 2 ] < 0) :486 mag = −mag487 t v e c /= mag488 r 1 vec = dot (K inv , h1 vec ) / mag489 r 2 vec = dot (K inv , h2 vec ) / mag490 r 3 vec = c r o s s ( r1 vec , r 2 vec )491

492 # cond i t i on the r o t a t i on :493 R = ze ro s ( ( 3 , 3 ) )494 R[ : , 0 ] = r1 vec495 R[ : , 1 ] = r2 vec496 R[ : , 2 ] = r3 vec497 (U, D, Vt) = svd (R)498 R = dot (U, Vt)499

500 # put toge the r a 4x4 t ranso fmrat i on matrix :501 Rt = ze ro s ( ( 3 , 4 ) )502 Rt [ : , 0 ] = R[ : , 0 ]503 Rt [ : , 1 ] = R[ : , 1 ]504 Rt [ : , 2 ] = R[ : , 2 ]505 Rt [ : , 3 ] = t ve c506 # T = append (T, [ [ 0 , 0 , 0 , 1 ] ] , 0)507

508 return Rt509 Rts = map ( SolveT , Hs)510 return Rts511

512 def ApplyRadialDist V2 ( projPtVec , K, k1 , k2 ) :513 # projPtVec i s a 2−D vecto r514 xo = K[ 0 , 2 ]515 yo = K[ 1 , 2 ]516

517 # compute the rad iu s :518 dx = ( projPtVec [ 0 ] − xo )519 dy = ( projPtVec [ 1 ] − yo )520 rSqr = dx∗dx + dy∗dy521 rSqrSqr = rSqr ∗ rSqr522

22

523 # apply d i s t o r t i o n model :524 distModel = (1 + k1∗ rSqr + k2∗ rSqrSqr )525 return [ xo + dx ∗ distModel , yo + dy ∗ distModel ]526

527 def ApplyRadialDist ( projPtVec , K, k1 , k2 ) :528 # projPtVec i s a 2−D vector , make i t 3D529 projPtVec = [ projPtVec [ 0 ] , projPtVec [ 1 ] , 1 . 0 ]530

531 # back−p r o j e c t p i x e l to xyz :532 K inv = array ( [ [ 1 /K[ 0 , 0 ] , 0 , −K[ 0 , 2 ] /K[ 0 , 0 ] ] ,533 [ 0 , 1/K[ 1 , 1 ] , −K[ 1 , 2 ] /K[ 1 , 1 ] ] ,534 [ 0 , 0 , 1 ] ] )535 xyz = dot (K inv , projPtVec )536

537 # compute the rad iu s : ( no cente r )538 rSqr = xyz [ 0 ] ∗ xyz [ 0 ] + xyz [ 1 ] ∗ xyz [ 1 ]539 rSqrSqr = rSqr ∗ rSqr540

541 # apply d i s t o r t i o n model : ( s e t z to 1)542 xy z d i s t = xyz ∗ (1 + k1∗ rSqr + k2∗ rSqrSqr )543 xy z d i s t [ 2 ] = 1 . 0 ;544

545 # rep r o j e c t :546 reprojPtVec = dot (K, xy z d i s t )547

548 return [ reprojPtVec [ 0 ] , reprojPtVec [ 1 ] ]549

550 def Ref ineCalParameters (K, Rts , xyDataSets , modelRadial=True , k1=0, k2=0) :551 # This func t i on uses the non−l i n e a r Least squares mehtod to do a552 # globa l opt imiza t i on on a l l parameters o f the c a l i b r a t i o n by553 # minimizing the re−p r o j e c t i o n e r r o r . The parameters i n c l ud e s the554 # rad i a l d i s t o r t i o n ( k1 and k2 ) .555 #556 # K i s the camera c a l i b r a t i o n matrix . Ts i s a l i s t o f e x t e rna l557 # parameters as a 3x4 homogeous transform , one per image .558 #559 # xyDataSets i s a l i s t o f s u b l i s t s . The s u b l i s t conta in s the560 # po in t s on the model plane and the po in t s on the image .561 #562 # The co s t func t i on i s the r e s i d u a l s s i n c e l e a s t s q w i l l do the563 # square and sum .564 numIntrinsicParams = 7565 def ConvertToParams (K, Rts , k1 , k2 ) :566 r v e c = ze ro s ( ( 1 , 3 ) ) # need to be 2D f o r Rodrigues2567 r t s I dxS t a r t = numIntrinsicParams568 params = ze ro s ( r t s I dxS t a r t + len (Rts ) ∗ 6)569 params [ 0 : 3 ] = K[ 0 , : ]570 params [ 3 ] = K[ 1 , 1 ]571 params [ 4 ] = K[ 1 , 2 ]572 params [ 5 ] = k1573 params [ 6 ] = k2574 for r t Idx in xrange ( l en ( Rts ) ) :575 R = copy (Rts [ r t Idx ] [ 0 : 3 , 0 : 3 ] )576 cv . Rodrigues2 ( cv . fromarray (R) , cv . fromarray ( r v e c ) )577 params [ r t s I dxS t a r t : r t s I dxS t a r t +3] = r ve c [ 0 ] # turn r ve c to 1−D578 r t s I dxS t a r t += 3579 t v e c = Rts [ r t Idx ] [ : , 3 ]580 params [ r t s I dxS t a r t : r t s I dxS t a r t +3] = t ve c581 r t s I dxS t a r t += 3582 return params583 def ConvertFromParams ( params ) :584 k vec = params [ 0 : 5 ]

23

585 K = array ( [ [ k vec [ 0 ] , k vec [ 1 ] , k vec [ 2 ] ] ,586 [ 0 , k vec [ 3 ] , k vec [ 4 ] ] ,587 [ 0 , 0 , 1 ] ] )588 k1 = params [ 5 ]589 k2 = params [ 6 ]590 rtParams = params [ numIntrinsicParams : ]591 a s s e r t (mod ( l en ( rtParams ) , 6) == 0)592 rtParams = rtParams . reshape ( l en ( rtParams ) /6 , 6)593 Rts = [ ]594 r v e c = ze ro s ( ( 1 , 3 ) ) # need to be 2D f o r Rodrigues2595 for r t Idx in xrange ( l en ( rtParams ) ) :596 params = rtParams [ r t Idx ]597 r v e c [ 0 ] = copy ( params [ 0 : 3 ] ) # r ve c i s 2D598 R = ze ro s ( ( 3 , 3 ) ) # the r o t a t i on 3x3 matrix599 cv . Rodrigues2 ( cv . fromarray ( r v e c ) , cv . fromarray (R) )600 t v e c = params [ 3 : 6 ] # the t r a n l s t i o n vec to r601 Rt = ze ro s ( ( 3 , 4 ) )602 Rt [ : , 0 ] = R[ : , 0 ]603 Rt [ : , 1 ] = R[ : , 1 ]604 Rt [ : , 2 ] = R[ : , 2 ]605 Rt [ : , 3 ] = t ve c606 Rts . append (Rt)607 return (K, Rts , k1 , k2 )608 def r e s i d u a l s ( params ) :609 # unpack the c a l i b r a t i o n parameters :610 (K, Rts , k1 , k2 ) = ConvertFromParams ( params )611 # xo = K[ 0 , 2 ] ; yo = K[ 1 , 2 ] # the cente r p i x e l o f image612 a l lRe s = [ ]613 for r t Idx in xrange ( l en ( Rts ) ) :614 # R = Rts [ r t Idx ] [ 0 : 3 , 0 : 3 ]615 # t vec = Rts [ r t Idx ] [ : , 3 ]616 # Rt = append (R, t ranspose ( [ t v e c ] ) , 1)617 Rt = Rts [ r t Idx ]618 P = dot (K, Rt) # the p r o j e c t i o n matrix619

620 # get po in t s on the model plan as 4−vec t o r s and image po in t s as621 # 3−vector , both in homogeous coo rd ina t e s622 xyDataSet = xyDataSets [ r t Idx ]623 wrldPts = [ row [ 0 ] for row in xyDataSet ]624 wrldVecs = array ( [ [ row [ 0 ] , row [ 1 ] , 0 , 1 ] for row in wrldPts ] )625 imgPts = [ row [ 1 ] for row in xyDataSet ]626 imgVecs = array ( [ [ row [ 0 ] , row [ 1 ] , 1 ] for row in imgPts ] )627 imgVecs = imgVecs [ : , 0 : 2 ] # get r i d o f l a s t dim628

629 # compute the re−p r o j e c t i o n r e s i d u a l :630 projdImgVecs = transpose ( dot (P, t ranspose ( wrldVecs ) ) )631 map (RealCoord , projdImgVecs )632 projdImgVecs = projdImgVecs [ : , 0 : 2 ] # get r i d o f l a s t dim633 i f modelRadial :634 projdImgVecs = map (lambda projPtVec :635 ApplyRadialDist ( projPtVec , K, k1 , k2 ) ,636 projdImgVecs )637 r e s = projdImgVecs − imgVecs638 a l lRe s . append ( r e s )639

640 r e s = [ elm for r e sS e t in a l lRe s for row in r e sS e t for elm in row ]641 print max ( r e s )642 return r e s643

644 # Calcu la te the opt imized s o l u t i o n :645 in itParams = ConvertToParams (K, Rts , k1 , k2 )646 p l sq = l e a s t s q ( r e s i dua l s , in itParams )

24

647 f ina lParams = pl sq [ 0 ]648 return ConvertFromParams ( f ina lParams )649

650

651 i f name ==’ ma in ’ :652 #653 # Extract co rne r s :654 #655 cornerSep = 1 .0 # sepa ra t i on between adjacent corner ( eg . 1 inch )656 Hs = [ ]657 xyDataSets = [ ]658 imageF i l e s = GetImageFi les ( ’ t e s t Images ’ , ’ jpg ’ )659 checkerDim = (5 , 4)660 imageF i l e s = GetImageFi les ( ’myImages ’ , ’JPG ’ )661 checkerDim = (4 , 5) # s ide−way662 counter = 1663 for imageFi l e in imageF i l e s :664 print imageFi l e665 image = LoadImage ( imageFi l e )666 imageDim = ( image . rows , image . c o l s )667 gray = cv . CreateMat ( image . he ight , image . width , cv .CV 8UC1)668 cv . CvtColor ( image , gray , cv .CVRGB2GRAY)669 edges = cv . CreateMat ( gray . height , gray . width , cv .CV 8UC1)670 cv . Canny ( gray , edges , 255∗1 .5 , 255)671

672 # Perform hough trans form with 1 degree and 1 p i x e l673 # re s o l u t i o n . Note : the output are so r t ed by the accumulator674 # values . A l i n e must have at l e a s t 50 p i x e l s on i t :675 l i n e s = cv . HoughLines2 ( edges , cv . CreateMemStorage ( ) ,676 cv .CVHOUGHSTANDARD, 1 , p i /180 , 50)677 ( bestL ines , sepDist ) = FindCheckerLines ( l i n e s , checkerDim , imageDim)678 houghResultImg = OverlayHoughLines ( bestL ines , image )679 a l lCorners 2D = FindCheckerCorners ( bestL ines , sepDist , checkerDim )680 in i tCorners Img = OverlayLabelCorners ( a l lCorners 2D , image )681 a l lCorners 2D = Ref ineCorners ( gray , a l lCorners 2D , sepDis t )682 re f inedCorners Img = OverlayLabelCorners ( a l lCorners 2D , image )683

684 #685 # Solve the homography to the model plane :686 #687 xyDataSet = GetCorrespondencePairs ( a l lCorners 2D , cornerSep )688 H = SolveHomographyByDLT V2 ( xyDataSet )689 Hs . append (H)690 xyDataSets . append ( xyDataSet )691

692 #693 # Visua l d i sp l ay and save images :694 #695 # i f ( counter < 5) :696 # DisplayImage ( edges )697 # cv . SaveImage (” edges ” + s t r ( counter ) + ” . png ” , edges )698 # cv . SaveImage (” image ” + s t r ( counter ) + ” . png ” , image )699 # DisplayImage ( houghResultImg )700 # cv . SaveImage (” hough ” + s t r ( counter ) + ” . png ” , houghResultImg )701 # DisplayImage ( in i tCorners Img )702 # cv . SaveImage (” i n i tCo rn e r s ” + s t r ( counter ) + ” . png ” ,703 # ini tCorners Img )704 # DisplayImage ( re f inedCorners Img )705 # cv . SaveImage (” r e f i n edCorne r s ” + s t r ( counter ) + ” . png ” ,706 # ref inedCorners Img )707 counter += 1708 # raw input (” p r e s s ente r to cont inue ”)

25

709

710 #711 # Solve f o r IAC712 #713 w = SolveIAC (Hs)714 K = SolveCameraCal ibrat ionMatr ix (w)715 Rts = SolveExternalParameters (K, Hs)716 (newK, newRts , k1 , k2 ) = Ref ineCalParameters (K, Rts , xyDataSets )717 # (newK, newRts , k1 , k2 ) = (K, Rts , 0 , 0)718

719 #720 # Repro j ec t i on721 #722 for imgIdx in xrange ( l en ( imageF i l e s ) ) :723 imageFi l e = imageFi l e s [ imgIdx ]724 print imageFi l e725 image = LoadImage ( imageFi l e )726

727 P = dot (newK, newRts [ imgIdx ] ) # the p r o j e c t i o n matrix728

729 xyDataSet = xyDataSets [ imgIdx ]730 wrldPts = [ row [ 0 ] for row in xyDataSet ]731 wrldVecs = array ( [ [ row [ 0 ] , row [ 1 ] , 0 , 1 ] for row in wrldPts ] )732 imgPts = [ row [ 1 ] for row in xyDataSet ]733 imgVecs = array ( [ [ row [ 0 ] , row [ 1 ] , 1 ] for row in imgPts ] )734 imgVecs = imgVecs [ : , 0 : 2 ] # get r i d o f l a s t dim735

736 # compute the re−p r o j e c t i o n :737 projdImgVecs = transpose ( dot (P, t ranspose ( wrldVecs ) ) )738 map (RealCoord , projdImgVecs )739 projdImgVecs = projdImgVecs [ : , 0 : 2 ] # get r i d o f l a s t dim740 projdImgVecs = map (lambda projPtVec : ApplyRadialDist ( projPtVec ,741 newK, k1 , k2 ) ,742 projdImgVecs )743 reProjResult Img = OverlayCorners ( projdImgVecs , image , 2)744 r e s = projdImgVecs − imgVecs745 r e s = [ row [ 0 ] for row in r e s ]746 maxRes = max ( r e s ) # max r e p r o j e c t i o n e r r o r747 print maxRes748

749 #750 # Visua l d i sp l ay and save images :751 #752 i f ( imgIdx < 5) :753 DisplayImage ( reProjResultImg )754 cv . SaveImage ( ” r e p r o j ” + s t r ( imgIdx ) + ” . png” , reProjResult Img )755 # raw input (” p r e s s ente r to cont inue ”)

26