J.-S. Pan, S.-M. Chen, and N.T. Nguyen (Eds.): ICCCI 2010, Part III, LNAI 6423, pp. 63–70, 2010. © Springer-Verlag Berlin Heidelberg 2010

A Vehicle License Plate Recognition System Based on Spatial/Frequency Domain Filtering and Neural Networks

Mu-Liang Wang1, Yi-Hua Liu1, Bin-Yih Liao2, Yi-Sin Lin2, and Mong-Fong Horng2

1 Department of Computer Science and Information Engineering, Shu-Te University, Kaohsiung, Taiwan

2 Department of Electronics Engineering, National Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan

mulwang@stu.edu.tw, s2nkcaz.wen@msa.hinet.net, jeltsine@gmail.com {byliao,mfhorng}@cc.kuas.edu.tw

Abstract. In this paper, we develop an intelligent application based neural networks and image processing to recognize license plate for car management. Through the license recognition, the car number composed of English alphabets and digitals is readable for computers. Recognition of license is processed in two stages including feature extraction and recognition. The feature extraction contains the image locating, segmentation of the region of interest (ROI). Then the extracted ROIs are fed to a trained neural network for recognition. The neural network is a three-layer feed-forward neural network. Test images are produced from real parking lots. There are 500 images of car plates with tile, zooming and various lighting conditions, for verification. The experiment results show that the ratio of successful locating of license plate is around 96.8%, and the ratio of successful segmentation is 91.1%. The overall successful recognition ratio is 87.5%. Therefore, the experimental result shows that the proposed method works effectively, and simultaneously to improve the accuracy for the recognition. This system improves the performance of automatic license plate recognition for future ITS applications.

Keywords: Neural Network, Plate Recognition, Wavelet Transform, Spatial/ Frequency analysis.

1 Introduction

Intelligent Transportation System (ITS) attracted lots of interest from industrials and academics worldwide in the past decade. The ITS investment in globe keeps growing fast. For examples, according to the report from CCID, ITS development in China becomes the focus of urban transportation improvement. The investment over ITS construction was totaled at CNY19.5 billion in 2008, rising by 39.3% of 2006. In United State, since 1992, Department of Transportation (DOT) has invested $1.5 billion to develop an ITS on limited access highways and local roadways across America. Most of this investment was to construct broadband infrastructure. In 2009,

64 M.-L. Wang et al.

there are at least 22 states in U. S. sought American Recovery and Reinvestment Act funds to invest in intelligent transportation system technologies including traffic cameras, express toll lanes, and improved traffic signals or accident alert systems. In fields of information and communication technology (ICT), data processing and communication are two pillars to develop feasible applications to improve the functionality and quality of the transportation systems. In other words, ITS is expected to offer more automotive services for drivers, riders and walkers on roads. The first and most significant issue of ITS is to identify vehicles in movement. Thus the technique of car license recognition has been key to successful ITS applications. How to identify the car license in a fast, reliable and accurate way has been acknowledged as a critical issue to be explored [1-7]. In this paper, we develop a license plate recognition system with some image processing technologies to automatically detect if there is any car to be recognized in the screen and rapidly filter out suspicious license plates in these images to recognize characters on the license plate.

The rest of this paper is organized as follows. In Section 2, the previous work related to car identification is reviewed to illustrate the state of art. The proposed Vehicle License Plate Recognition System (VLPRS) is presented to describe the architecture and operations for the license recognition in Section 3. The performance of the developed system is evaluated and analyzed in Section 4. Finally, we conclude this work in Section 5.

2 Related Works

There were various approaches proposed to identify moving vehicles. In Taiwan, Electronic Toll Collection (ETC) was installed in highways for toll services for years. The identification of vehicles is realized by an infrared communication between the transceivers installed on vehicles and on the toll stations. Although this approach is reliable and fast enough for moving vehicles, the pre-installation of transceivers is an obstacle of deployment. Radio-Frequency technology is another approach to identify vehicles in recent years due to its low cost of deployment. However, worse reliability of identification degrades the feasibility of the RFID-based approach. Image-based approach is another attractive solution due to the well-deployed cameras on roads and streets. However, how to interpret the image content and to identify the ownership of vehicles is the key issue to be investigated.

Feature extraction is the first step to identify the license plates of vehicle images. In the past, there were some researches focusing on this topic. Mello et. al. proposed a system based on color alternation for acquisition images and fuzzy logic for segmentation of digit images on license plates. Although the reliability and accuracy of that proposal are recognized, the exception conditions in real environments such as lighting dynamic and image tilt are not explored. Lee [2] presented an approach based on neural networks to recognize image patterns and content mining. His neural network simulator is efficient for image data. However, the applications on license identification are rare to evaluate the performance of the simulator. Yu et. al. [5] proposed a vertical edge matching algorithm to detect the edge of characters on license plates. Morphology [6] is also another effective scheme to remove the noise introduced during image

A VLPRS Based on Spatial/Frequency Domain Filtering and Neural Networks 65

captures. Chun et. al.[7] proved that a requirement of vehicle identification system on real-time operation is essential to ITS applications. Lin et. al.[8] overcame the problem of low-resolution printed digit for character recognition. In this paper, we present a VLPRS implementation based on neural networks and dual-domain signal filtering. The developed system features its ability to learn, train and recognize the license plates in various conditions of tilt, zooming, and dim-lighting. Thus the robustness is better than ever.

3 A Vehicle License Plate Recognition System Based on Spatial/Frequency Domain Filtering and Neural Networks

The architecture of the license plate recognition system is shown in Fig. 1, including preprocesses like license plate locating, license plate image character segmentation, and character recognition. In the stage of license plate locating, after inputting of license plate image, we use Wavelet edge detector and apply the morphology technique to quickly locate candidate areas that might be license plates. Projection and plate number format judgment are used to analyze if the located area is a license plate. If it is a license plate, we extract characters from the license plate. In the stage of plate number recognition, we apply the neural learning method to recognize characters on license plates. Here we introduce each stage of entire system below.

Fig. 1. Architecture of Vehicle License Plate Recognition System

In this paper, we present a rapid recognition of license plate, to find suspicious location of license plate and filter out with license plate formats. The function of wavelet transform is to effectively segment the signals by frequencies. In this paper we use the wavelet transform to process the edge detection of the license plate system. If one image had been processed by the wavelet transform, it would be divided into two

66 M.-L. Wang et al.

parts: the high-frequency one and the low frequency one. The high frequency part of an image includes most of the edges of objects that the grayscale changes intensively. Ku presented an approach in Wavelet operators for multi-scale edge and corner detection [4], the approach based on 4-tap Daubechies wavelet transform coefficients and operands combines 2-dimension Discrete Periodic Wavelet Transform to derive the operand mask of wavelet edge detection. The computation is relatively complicated. The original image and the edge detected by wavelet are shown in Fig. 2-3; they allow a wider range of the threshold. If the threshold goes down to 50, the wavelet edge detection gets better result in both the edge of the license plate and the complex street view in background, and it also has a better immunity of noise than the Sobel edge detection.

Fig. 2. Original Image Fig. 3. Edges detected by Wavelet

After the wavelet edge detection, we use the morphology in image process to analyze the shape and the structure of the image to strengthen the structure to locate the license plate. Figure 7-10 show the object we find in contrast with the original image after the morphology process. Closing and opening from morphology can rapidly erase noise and places that do not match the aspect ratio of a license plate. After the morphology process we precede the 8-connected component algorithm. In order to accurately cut out the candidate places for the plates, we set the aspect ratio and the area ratio of object pixels as conditions to filter out things not familiar with a license plate. The figure below shows locations that we get after the license-plate-like condition filtering. Though one of them is not exactly a license plate, but there are still many features to be examined, we put these two objects to the follow-up license plate character procedure.

Fig. 4. Detected edge fed to morphological processing

Fig. 5. Result after morphological processing

A VLPRS Based on Spatial/Frequency Domain Filtering and Neural Networks 67

The second part of the license plate recognition system is character extraction, its main purpose is to extract characters from a license plate object and find the top, right, bottom and left border of each character. Binary threshold is to determine the best threshold. Fig. 6 and 7 below show that the result of the Threshold Selection Method from Gray-Level [9] suffers from the effect of shadow, and fails to divide the characters from background. Before we proceed the character segmentation on the license plate, we must convert the license plate image into objects with single alphabets and digits. The conversion is manipulated by vertical projection of the segmented images. By projection, we detect the positions of characters and the borders of plate. Examples of vertical projections are shown in Fig. 8. Then we use the vertical projection on the horizontal axis in the histogram. We use the vertical-projected aspect ratio as the basis of character and noise prediction. We determine the position of the characters on the license plate by projection.

Fig. 6. Original Image Fig. 7. Binarized image by Otsu[9]

(a) (b) (c)

Fig. 8. Histograms of Vertical projects for various license images

The segmented ROI image blocks are fed to a feed-forward neural network to identify. In the developed neural network, there are two stages, training and testing, for the presented system. In a training stage, a supervised strategy is used to train the network by adapting the connection weights between neurons till the convergence. As depicted in Fig.1, there are three layers to compose the neural network. The input pattern is given as follows

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=×

nkkk

n

n

k

nk

XXX

XXX

XXX

X

X

X

X

,2,1,

,22,21,2

,12,11,1

2

1

(1)

68 M.-L. Wang et al.

where k, n are the pattern number and pattern length, respectively. Beside the output of the neural network are given as

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=×

mkkk

m

m

k

mk

TTT

TTT

TTT

T

T

T

T

,2,1,

,22,21,2

,12,11,1

2

1 (2)

where m is the number of alphabets and digits to recognize. The elements of Tk, m indicate the recognition results generated by the neural network. When Tk, m is set to 1, the k-th input pattern is recognized as the m-th alphabets or digits. The initial weight matrix are depicted as

)(/ nknknk XnormXX ××× = (3)

mkT

nkmn TXW ××× ⋅= (4)

During the training stage, the weight matrix is updated according to Eq. 5-8

mnnkmk WXY ××× ⋅= (5)

mkmkmk YTdY ××× −= (6)

mkT

nkmn dYXdW ××× ⋅= (7)

mnmnmn dWWW ××× += (8)

where Y, dY and dW are the actual output, the output error and the adjustments of the connection weights. And the actual output of the neural network is shown in Eq. (9)

( ) *WXfY mk =× (9)

where the activation function of this neural network are indicated as follows,:

( ))exp(1

)exp(1

x

xxf

σσ

−+−−= (10)

Clearly, the activation function depicted in Eq. 10 produces a bipolar output Y, ranging between [-1, 1]. In a stabilized network, the neuron outputs converge to either 1 or -1. If Yi converges to 1, the i-th alphabet is identified, otherwise the i-th alphabet is denied. Thus, after the training stage, the trained neural network is used to recognize the license plate through the manipulation of the extracted features. This process is denoted as testing stage. In the following section, we will use 500 vehicle images from real environments to verify the performance of the developed system.

4 Experiment Results

There are four test cases for various conditions such as tilt, zooming, dim-lighting and integrated recognition. In a real application, tilt images are encountered unavoidably.

A VLPRS Based on Spatial/Frequency Domain Filtering and Neural Networks 69

This image tilt usually is caused by relative positions of camera and vehicles. The width-to-height ratio (WHR) of license plate image is the index to measure the tilt. The images of with no tilt have the WHR around 2.9. The tested image with the WHR of 3.9 is as shown in Fig. 9. The segmentation result is shown in Fig. 10. The second test is to verify the recognition in the conditions of zoomed images. The original images as shown in the first row, Fig. 11. These two images with significant difference are captured in various distances. The second row in Fig. 11 shows the extracted context of license plates. Clearly, the developed system is able to successfully extract the ROI of alphabets and digits.

Fig. 9. Tilted image Fig. 10. Segmentation results

The third test is to verify the performance of the developed system in a dim-lighting condition. In Fig. 12 (a), the captured original image is shown. The detected edges are depicted in Fig. 12 (b). The extracted ROI of plate image is shown in Fig. 12 (c).

(a) (b)

Fig. 11. Zoomed images and the extractions

(a) (b) (c)

Fig. 12. Performance evaluation in the dim-lighting condition

70 M.-L. Wang et al.

The accuracy and reliability are two performance indexes to be verified in this test. There are totally 500 vehicle medium-resolution images in the testing of the license plate recognition system. The identification procedure is divided into two parts including the internal testing and the external testing. In internal testing, we use 250 same images to train the neural network and to test the training performance. In an external test, the extra 250 vehicle images are fed to the system to verify the identification accuracy. These images are taken from various environments including day, night, distances between the camera and the car. In this paper we take characters in the 250 license plates as the learning sample of neural network. The recognition rate of the internal testing is 100%. And we use another 250 images which are taken in a worse situation for the external testing.

5 Conclusion

In this paper, we apply wavelet edge detection to extract the edges in the image during the preprocess stage. Wavelet edge detection owns the characteristic of high noise-immunity, and it allows a wider range in threshold we set. Then we apply morphology, to record the top, right, bottom and left border of each object. With projection, we can detect the position of plates and record the height and the top and bottom border of characters in the process of character segmentation. Afterwards, we check the number of characters and determine whether the aspect ratio of characters meets the format of plate numbers. Finally we normalize the character images and apply SimNet to recognize characters.

References

1. Chang, S.L., Chen, L.S., Chung, Y.C., Chen, S.W.: Automatic License Plate Recognition. IEEE Transactions on Intelligent Transportation Systems 5(1), 42–53 (2004)

2. Mello, C.A.B., Costa, D.C.: A Complete System for Vehicle License Plate Recognition. In: 16th International Conference on Signals and Image Processing (IWSSIP 2009), pp. 1–4 (2009)

3. Lee, H.C.: SimNet: A neural network architecture for pattern recognition and data mining, University of Missouri-Rolla (2003)

4. Ku, C.T.: Wavelet operators for multi-scale edge and corner detection. Department of Electrical Engineering, I-Shou University, Taiwan (1998)

5. Yu, M., Kim, Y.D.: An Approach to Korean License Plate Recognition Based on Vertical Edge Matching. In: IEEE Conference on Systems, Man, and Cybernetics, pp. 2975–2980. IEEE Press, New York (2000)

6. Hsieh, J.W., Yu, S.H., Chen, Y.S.: Morphology-based license plate detection from complex scenes. In: 16th International Conference on Pattern Recognition, pp. 176–179. IEEE Press, New York (2002)

7. Chun, B.T., Soh, Y.S., Yoon, H.S.: Design of Real Time Vehicle Identification System. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 3, pp. 2147–2152. IEEE Press, New York (1994)

8. Lin, H.H., Chen, C.Y., Chuang, J.H.: Recognition of Printed Digits of Low Resolution. Pattern Recognition and Image Analysis 10(2), 265–272 (2000)

9. Otsu, N.: A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on System, Man, and Cybernetics SMC-9, 62–66 (1979)

10. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, Hoboken (2000)