Corner Detection via Trilateral Filtering of

Chain Codes

Kemal Ozkan

Department of Computer Engineering

Eskisehir Osmangazi University

kozkan@ogu.edu.tr

Cihan Topal, Cuneyt Akinlar

Department of Computer Engineering

Anadolu University

{cihant, cakinlar}@anadolu.edu.tr

Abstract—Contours are the most common feature used in the

shape detection and recognition applications. Contours in digital

images constitute of consecutive pixels, thus they can be

represented with chain codes by encoding the relative positions of

the pixels forming that contour. Sharp transitions on chain coded

contour information correspond to the corners or notches of the

2D shape enclosed by that contour. Moreover, useful information

about geometric structures and orientations of the shapes can be

gathered by examining the distributions and moments of the

chain codes. Despite their profitable properties, chain codes are

not robust against the rotation. Because contours are exposed to

quantization while they are being digitized, same geometric shape

may have different chain code representations. Another downside

about the chain codes is the difficulty of their extraction from

real images in an efficient manner. In this study, we propose a

novel real-time corner detection method by extracting the chain

codes by Edge Drawing (ED) method and after processing them

with the trilateral filtering.

Keywords-corner detection; edge detection; feature extraction;

chain code; real-time imaging

I. INTRODUCTION

Shape detection, recognition and matching topics are one of the most studied problems in the image processing and computer vision literature. The most appropriate feature to identify a shape should include the contour information enclosing the shape. Studies about representing and coding those contours are one of the very early topics in the computer vision and pattern recognition research. Contours are usually extracted from the derivation of an image by a gradient computation process. By combining the magnitudes of the horizontal and vertical gradients in a vectorial form, a gradient value is obtained. Those gradient values may vary in a range [0, 2π] and constitutes a curvature along a contour path on the image. Chain code is a quantized version of those gradients that lie on the curvature. Freeman proposed chain codes for various numbers of neighborhoods (i.e. 4, 8, 16, 32, etc.) in order to represent the contours in a digitized sequential form [1, 2]. The most common used chain code is the 8-neighborhood chain code which comprise of gradient vectors for 8 adjacent pixels (see Fig. 1).

Not only chain coding is the standard way in representing the contours in a sequential manner, it is also the most commonly used feature for shape description in the literature. The early techniques about the shape representation and detection base their decision on the contour length and

curvature values of the chain coded contours of the shape borders [3, 4]. There are also studies that use the various moments of the contours as features [5, 6]. Furthermore, there are also shape descriptors based on the distribution analysis of chain codes [7]. In [8], various contour based shape descriptors are examined and their performances are compared.

Figure 1. (a) 8-N Freeman chain codes, (b) corresponding angles.

There are many applications that make use of the above properties of the chain coded contours in the literature. Since the chain codes enclose the essential information about the geometry of the contour they are extracted from, they provide quick results on line and curvature detection [9, 10]. In another study [11], authors propose a novel scale invariant shape descriptor for object detection and classification on real images by grouping the adjacent contour segments. In [12], chain coded contour information is used as feature for registration of the satellite images. Analogous to the above examples, character and symbol recognition applications also benefit from chain coded contour information [13, 14].

II. RELATED WORK

In addition to the studies mentioned above about the contours and chain codes, another application which takes advantage of chain coded contour information is corner detection. Since corners are reliable features for images, they are usually substituted as interest point in the object detection and tracking algorithms. Fast and robust corner detection is an essential task especially in the real-time computer vision applications. Sharp transitions on the one-dimensional chain code series extracted from a shape contour correspond to either a corner or a notch on the two-dimensional shape geometry. In this manner, corners of an image can be detected in a fast way by using one-dimensional naïve signal processing techniques.

Although, there are well-known corner detection algorithms based on the two-dimensional image processing

a)

0°

45°

90° 135°

180°

225°

270°

315°

b)

1

0

2

3

4

5

6

7

168

methodologies, we prefer to give a literature digest covering the ones that work on contours like our method. Freeman and Davis scanned the gradient values along the chain coded contour with a moving line segment in order to compute local curvature values [15]. Then they simply differentiate the adjacent local curvature values to locate the corners. Beus and Tiu also followed a similar way in their study, with an addition that limits the length of the line segment which scans the contour [16]. Sarkar proposed the detection of significant points on chain coded contours in his study of polygonal approximation of curves [17]. Those significant points are then used as vertices to be joined with each other in an attempt to achieve the polygonal approximation. In [18], authors proposed computing and comparing local histograms for each point on the chain coded curvature in order to estimate the local curvature value and detect the corners. In [19], authors proposed evaluating the first derivatives of the chain code series in order to find the locations having discontinuities.

These methods may work fine with the given chain coded contour segments; however, extracting the chain coded segments from the real images is a tough problem. The most intuitive way for obtaining chain coded segments is applying a connected component analysis after an edge detection algorithm. Thus edge segments can be found and chain code stream for each segment can be obtained by tracking along the segment. But, in order to achieve the efficiency, edge segments have to be obtained in one-pixel wide, smooth and contiguous manner. Therefore, most of the contour based corner detection methods are not interested in how the chain codes are attained from the real images and work on the contours of the readily segmented shape contours.

The main contributions of this study may be listed in two points. First, we propose an efficient way to extract the edge segments from the real images on the contrary of many other methods. Thus, our method does not work on readily given chain code series. Second contribution of this study is the way that we process the chain codes to find the corner points on the image. In order to find the edge segments and corresponding chain codes from the real images, we propose the use of the recently proposed combined edge/segment detection method, i.e. Edge Drawing (ED) algorithm [20]. Then we apply trilateral filtering to those segments to locate the sharp transitions on the chain code. The rest of the paper is organized as follows: In the 3

rd section, the proposed method is

introduced in detail. In the 4th section, experimental results are

presented, and, the paper is finalized with the concluding remarks in the last section.

III. CORNER DETECTION VIA TRILATERAL FILTERING

A. Problem Definition

Various features can be extracted easily from the chain code series with the time domain signal analysis techniques. But there is an important circumstance that needs to be solved efficiently. Since real curves are quantized as they are being represented by the combinations of chain codes, even a straight line's chain code series can fluctuate with respect to the angle between the horizontal axis. This fact constitutes a problem when one wants to analyze the chain coded curves. In Figure 2, there are four straight lines each with a different angle with the horizontal axis and quantized representations of lines are

indicated having different colors. Note that quantized versions of the lines comprise of various chain code combinations even though all the lines are definitely straight.

Figure 2. Straight lines with various angles and quantized representations.

For a real and more concrete example we can look at Figure 3. There are three squares with the same size but different orientations, and plots of their chain code series obtained with edge drawing algorithm are also presented. Note that the chain code representations fluctuate as the square is rotated. This situation impairs the quality of the information gathered from the chain code series and can prevent attaining effective results.

Figure 3. a) Three squares with different orientations,

b, c, d) chain code series of square 1, 2 and 3, respectively.

B. Proposed Algorithm

The proposed method in this study consists of three essential steps; extraction, smoothing and computing the derivative of chain code series.

1) Extraction of chain code series

The very first idea about obtaining edge segments from a real image is applying an edge detection algorithm, and then substituting a connected component analysis in an attempt to find the segments over the edge image. Thus corresponding chain codes can be computed by processing the edge segments. However, this strategy may turn out inefficient results especially in the noisy cases because noise can cause disconnections and notch-like formations along the edge segment.

1 2 3

(a)

(b)

(c)

(d)

169

In order to get efficiency from the chain code application, the edge segments have to be extracted in such a way that they meet some quality metrics. Those metrics can be listed as smoothness, continuity, thinness and localization, and they indicate how well an edge segment can represent the transitional change on the image. If extracted edge segments do not meet those quality metrics, they cannot contain the morphological and geometric information about that shape.

Edge Drawing (ED) algorithm is a novel detector that

obtains one-pixel wide, smooth and contiguous edges [20]. It

works by computing a set of points most likely to lie on an

edge, and linking them with a smart routing algorithm. In

other words, ED draws the edges from the beginning by using

the information derived from the image, whereas, the

conventional methods obtain the edges by filtering and

eroding the image elements. Obtaining edges in high level

qualifications is not the only competence of ED, but also

getting the edge map in vectorial fashion, i.e. array of

segments instead of a 2D binary edge image. Thus it enables

fast and efficient edge segment extraction for obtaining

corresponding chain codes.

2) Smoothing chain codes with trilateral filtering

As we explain in the Section III.A, straight lines or smooth curves fluctuate when they are being represented with combinations of chain code vectors. This situation complicates the detection of corner points along the contour. Corner points could easily be located by computing first derivatives if chain codes would be smooth like in the Figure 3.a, however, the result of the derivative comprise of useless and meaningless information if the signal looks like in the Figure 3.b and 3.c. So, we need to smooth the chain code series in such a manner that removes the fluctuations but preserve the sharp transitions.

Figure 4. a) Chain code series, b) response of a bilateral filter,

c) first derivative of smoothed signal.

In our previous study, we proposed bilateral filtering to serve this purpose [21]. Bilateral filtering provides good performance on smoothing the chain codes while preserving the corner parts. But, it still fails in some circumstances, especially if outlier chain code values are the case. In Figure 4.a, a chain code series is seen with an outlier around 50

th

sample. Since it is the only chain code vector around its vicinity, we can say that there is no corner and that peak on the chain code is probably consequence of noise. But, when we

inspect the smoothed signal with bilateral filtering (Fig. 4.b), we see that the signal is well flattened except the peak point. And the first derivative of the smoothed signal carries a high variation around the peak's location, which can easily be misinterpreted as a corner.

Under these circumstances, obviously there is a need for improving in such a way that makes the method cover these kinds of situations. The use of trilateral filtering can provide better results since it is an optimized version of bilateral filtering for outliers. Trilateral filtering is widely used in applications where detail preserving smoothing is significantly necessary [22, 23]. A bilateral filter is seen in the Eq. (1);

2

2

2

2

2

||

2

||

),(),(

),(),(),(

R

yx

s

uu

R

yx

S

rs

eyxwandeyxw

whereyxwyxwyxw

σσ

−−−−

==

×=

(1)

In the above equation, wr represents the distance between the samples and ws stands for the intensity differences. Trilateral filters have some improvements over bilateral. Like a bilateral filter, they are claimed to suppress outliers while preserving local characteristics. For this, the value of r is defined and formulated as follows:

/ 2

/ 2 1

n n

i i

i n

n

i

i

x x

r

x

+

−

=

∑ ∑

∑ (2)

In Eq. 2, coefficient n designates the size of the filter and r value can range in [0, 1]. If r value is greater than a user-set threshold (which is taken as 0.5 in this study), j is set to 1; otherwise j is set to 0. Thus final form of the trilateral filter becomes;

1* *j j

r sw w w r−= (3)

Thus trilateral filter on the edges acts as bilateral filter and preserves the important information. On the other hand, it acts like unilateral filter on flat areas and take care of noise. Then we can smooth the chain code series more efficiently that does not fail in the cases including outliers. The trilateral filter output of the chain code series in the Figure 4.a is presented in Figure 5. Note that the false corner caused by an outlier is cured and true corner value preserved, even optimized.

Figure 5. a) response of trilateral filter for the chain code series in Figure 4.a,

b)first derivative of smoothed chain code.

(a)

(b)

(c)

true corner no corner

(a)

(b)

true corner false corner

170

Table 1. Experimental results of the proposed algorithm

Image Edge Drawing result Corners with Bilateral Filtering Corners with Trilateral Filtering

Peppers

number of segments: 165

number of corners: 257

number of corners: 172

Boats

number of segments: 270

number of corners: 410

number of corners: 266

F16

number of segments: 212

number of corners: 461

number of corners: 315

Houses

number of segments: 302

number of corners: 515

number of corners: 261

171

3) Computing the first derivative

After the smoothing operation, we simply compute the first derivative of the smoothed chain code series. Then we simply threshold the first derivative output and find the extrema points of the remaining blobs. In that sense, corner points of the signals in the Fig. 4.c and 5.b correspond to the peak values of the derivative response between the 50

th and 100

th samples.

IV. EXPERIMENTAL RESULTS

We present the results of our previous method [21] and current method for a few number of well-known test images in Table 1. Our method has a computation time around 15~20 milliseconds on an average desktop computer for 512x512 grayscale images. It means that our method can reach a frame rate around 50-65 fps for this resolution. Note that the trilateral filtering results have fewer amounts of corners with the same parameters since it prevents false corners detections.

V. CONCLUDING REMARKS

As reliable features, corners have great importance on

many computer vision and image processing algorithms, i.e.

photo stitching, object detection, shape matching, registration,

etc. Efficient and fast detection of corners is also has a

substantial importance if the realtime applications are the case.

In those applications, 2D image processing methodologies are

commonly substituted. But they may take much time and

sometimes results in enormous number of corners since they

process each pixel.

In this study we introduce a fast corner detection method

that works by locating the sharp changes on angle values of

chain codes. We smooth the extracted chain codes by trilateral

filtering in order to preserve the actual corner locations as we

suppress the fluctuations caused by the quantization process of

real curves. Since we detect the corners only lying on an edge

segment, we achieve adequate results in a very short time

span. Furthermore, we do not need to find and locate the

neighbor corners in order to group them. We still work on

improving our method to obtain more effective corners to be

used as stable key points.

REFERENCES

[1] H. Freeman, ”On the encoding of arbitrary geometric configurations'', IRE Trans. on Electronic Computers, 1961, pp. 260-268.

[2] H. Freeman, "Application of the generalized coding scheme to map data processing," in Proc. IEEE Pattern Recognition and Image Processing Conf., Chicago, IL, May 31-June 2, 1978, pp. 220-226.

[3] S. Marshall, “Review of shape coding techniques”, Image and Vision Computing, 7(4):281–294, November 1989.

[4] M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis and Machine Vision, Chapman & Hall Computing, 1993.

[5] S. A. Dudani, K. J. Breeding, and R. B. McGhee. Aircraft identification by moment invariants, IEEE Trans. on Computers, C-26:39–46, 1977.

[6] L. Gupta and M. D. Srinath, “Contour sequence moments for the classification of closed planar shapes”, Pattern Recognition, 20(3):267–272, 1987.

[7] M. Peura and J. Iivarinen, “Efficiency of simple shape descriptors”, 3rd International Workshop on Visual Form, Capri, Italy, May 28–30 1997.

[8] J. Iivarinen, M. Peura, J. Särelä, and A. Visa, “Comparison of Combined Shape Descriptors for Irregular Objects”, Proc. of the Eight British Machine Vision Conference, UK, 1997.

[9] L. Chao, W. Zhong and L. Lin, "An Improved HT Algorithm on Straight Line Detection Based on Freeman Chain Code", IEEE CISP 2009.

[10] H. Li, H. Zheng, Y. Wang, "Segment Hough Transform – a Novel Hough-based Algorithm for Curve Detection", Int'l Conf. on Image and Graphics, IEEE ICIG 2007.

[11] V. Ferrari, L. Fevrier, F. Jurie, and C. Schmid, "Groups of Adjacent Contour Segments for Object Detection", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 30, no. 1, Jan 2008.

[12] X. Dai and S. Khorram, "A Feature-Based Image Registration Algorithm Using Improved Chain-Code Representation Combined with Invariant Moments", IEEE Trans. on Geoscience and Remote Sensing, vol. 37, no. 5, Sep 1999.

[13] N. Sharma, U. Pal, and F. Kimura, “Recognition of Handwritten Kannada Numerals”, IEEE CIT 2006.

[14] N. A. Jusoh and J. M. Zain, "Application of Freeman Chain Codes: An Alternative Recognition Technique for Malaysian Car Plates", Int’l Journal of Computer Science and Network Security, vol.9(11), Nov 2009.

[15] H. Freeman and L. S. Davis, "A corner finding algorithm for chain coded curves," IEEE Trans. on Computers, vol. C-26, pp. 297-303, 1977.

[16] D. Sarkar, "A simple algorithm for detection of significant vertices for polygonal approximation of chaincoded curves", Pattern Recognition Letters vol.14 pg.959-964, Dec 1993.

[17] F. Arrebola, A. Bandera, P. Camacho and F. Sandoval, "Corner detection by local histograms of contour chain code", Electronics Letters, vol. 33 no. 21, Oct 1997.

[18] N. Nain, V. Laxmi, B. Bhadviya, and N. C. Singh, "Corner Detection using Difference Chain Code as Curvature", IMECS 2008, 19-21 March, Hong Kong.

[19] H. L. Beus and S. S. H. Tiu, “An Improved Corner Detection Based on Chain Coded Plane Curves”, Pattern Recognition Letters vol. 20, pg. 291–296, 1987.

[20] removed for blind review process, “Edge Drawing: An Heuristic Approach to Robust Real-Time Edge Detection”, ICPR2010, 23-26 August, TURKEY.

[21] removed for blind review process, "Corner Detection with Bilateral Filtering of Freeman Codes", 19th IEEE Conf. Signal Processing and Communication Application (SIU2011), Antalya, TURKEY, 2011.

[22] W. W. Albert , A. C. S. Chung , S. H. Yu, "Trilateral Filtering For Biomedical Images", In Proc. of the IEEE Int'l Symposium on Biomedical Imaging, 2004, 820-823.

[23] P. Choudhury, J. Tumblin, "The Trilateral Filter for High Contrast Images and Meshes", Eurographics Symposium on Rendering, pp.1-11, 2003.

172