3rd International IEEE Conference Intelligent Systems, September 2006

Thai Vehicle License Plate Recognition Using

the Hierarchical Cross-correlation ARTMAP

Pruegsa Duangphasuk and Arit Thammano

Abstract- This paper proposes a Thai vehicle license platerecognition system, called the hierarchical cross-correlationARTMAP. An ability to separately train each segment of thenetwork gives the hierarchical cross-correlation ARTMAP anadvantage over the other approaches. The experimentalresults show that the hierarchical cross-correlation ARTMAPoutperforms the other approaches by a wide margin.

Index Terms-Cross-correlation, Hierarchical neuralnetwork, Pattern Recognition, Vehicle license platerecognition.

I. INTRODUCTION

During the past decade, a variety of techniques have beenapplied to deal with the car license plate recognition. Themethods widely used are the neural networks [1, 2, 3, 4, 5],fuzzy logic [6], and the template matching based approach[7, 8, 9, 10]. Even though many researches have succeededin recognizing the standard Thai characters for a long time,the Thai license plate recognition still has many problemsto overcome. One of the major problems is the possibilityof wrong recognition due to the similarity of the charactersbetween Thai vowels, Thai consonants and Arabic

numerals, e.g. O-0-0, 1-1, 5-5-5, 7-e1- 1, 3-1, w-W, fl-n-n, v-

v-zi-zi, - -u-in,- n-n-n, 1-1-1, uJ-u-u-U, vJ-1. This ismainly because the Thai characters used in the licenseplates are not standard. It typically contains no head. Thehead is the loop at the beginning of the characters. It is oneof the important features used to differentiate one characterfrom another. Without the head, the characters are evenmore similar as shown in Fig. 1. In this paper, thehierarchical cross-correlation ARTMAP is proposed torecognize the no-head Thai characters as well as the Thaivehicle license plates.

Following this introduction, section II provides anoverview of Thai vehicle license plates. Section IIIdescribes the architecture of the proposed model and itslearning algorithms. Section IV presents the preprocessingprocess. In section V, the experimental results aredemonstrated and discussed. Finally, section VI is theconclusion.

P. Duangphasuk is with the Faculty of Information Science andTechnology, Mahanakorn University of Technology, Bangkok, 10530Thailand (telephone: 662-988-3655 ext. 214, email: pruegsaAmut.ac.th).

A. Thammano is with the Computational Intelligence Laboratory,Faculty of Information Technology, King Mongkut's Institute ofTechnology Ladkrabang, Bangkok, 10520 Thailand (telephone: 662-737-2551 to 4, e-mail: aritAit.kmitl.ac.th).

Standard Characters No-head Characters

5 a X 5 5 5

qAfl l 0 01QH~ l flm 'In n n nfl fi D n n n

Fig. 1. Examples of the standard characters and no-head characters

II. THAI VEHICLE LICENSE PLATES

In Thailand, the characters on vehicle license plates consistof Thai consonants, vowels, tone symbols, other symbolsand Arabic numerals. Fig. 2 shows some examples of Thailicense plates; all four images, (a) to (d), were taken whilethe vehicles were immobile. The license plates shown inFig. 2(a), 2(b), and 2(d) are white with black text. 2(a) isfor private cars; 2(b) is for diplomatic cars; and 2(d) is formotorcycles. The license plate shown in Fig. 2(c) is whitewith green text; it is for commercial trucks. Typically, Thailicense plates, except diplomatic cars' and motorcycles',consist of two parts: the upper part and the lower part. Theupper part is the serial number, while the lower part is thename of the city in which the vehicle is registered. To havea clearer view on how Thai license plates are recognized inthis research, it would be better to understand the Thaiwritten language beforehand.

(a) (b)

(c) (d)Fig. 2. Examples of the Thai motor vehicle license plates

The text line image of a Thai document is composed ofthree levels (upper level, central level, and lower level) asshown in Fig. 3. All Thai consonants and Arabic numeralsare located in the central level of the line. Vowels and othersymbols generally appear in all three levels. Tone symbolscan only be seen in the upper level. Table I shows allcharacters used in Thai written language and the levels theyappear in.

d -* Upper Level

o) cJj na 19 h 14U >- Central Levelower evel

Fig. 3. An example of a line in Thai document

1-4244-0195-X/06/$20.00 ©2006 IEEE 652

TABLE IALL CHARACTERS USED IN THAI WRITTEN LANGUAGE AND THE LEVELS

THEY APPEAR IN

Upper Level

fl6Ufl Hf91 IMOX J q JM1U q 11

Central Level

w1 t tO lli10 123456789

Lower Level

III. THE PROPOSED SYSTEM

The proposed model is composed of three components:Upper, Central, and Lower as shown in Fig. 4. Uppercomponent is used to recognize the characters in the upper

level; while Central and Lower components are used torecognize the characters in the central level and lower levelrespectively.

2. For the rest of the input pattern, calculate the maximumof the normalized cross-correlation between the inputvector of the hth segment (lh) and each weight vector ofthe same segment (wj) at any shifted period p. Then

the output of the jth node, Tj (Ih), in the hth segment of

the hidden layer is determined. For each segment, thehidden node J with the highest output value is selectedas the winning node. In case of a tie, the node with thesmallest index is chosen.

T (Ih) =f(Ih wh) (3)

f(x, (4)

J =argmax{T> (I )} (5)

Fig. 4. The proposed model

Each component is a four-layer feedforward neuralnetwork as shown in Fig. 5. The first layer is the inputlayer, which consists of nine segments. Each segmentcontains N input nodes, which is the same size as thenumber of feature components extracted from eachsegment of the character image. The second layer is thehidden layer. Each hidden node in the hth segment is fullyconnected to all input nodes of the same segment via the

hconnections w I. The third layer is the cluster layer. The

nodes in this layer and the hidden layer are constructed

during the learning phase. wkh is the weights of the

connections between the kth node in cluster layer and the hthsegment. The fourth layer is the output layer. Each noderepresents a class of character. During the training process,

the input vector, {.h }N),...h h1(I.1N),...,(I19,..9), is presented to the model, together

with its associated class of character. Then the followinglearning algorithm begins:1. For the first input pattern, initialize the reference

weights of the first hidden node (wh) and the firstcluster node (wlh) according to the following equations:

h h(1

Wil = Ii(1Wlh =w h (2)1

where i = 1, 2, ..., N.h

j= 1,2, ...,MI

h= 1,2, ...,9.N is the number of feature components in each

segment.Mh is the number of hidden nodes in the hth

segment.f(x, y) is the maximum of the normalized cross-

correlation at any shifted period p.

p is the lag variable, which is between -(N-1) to(N-1).

3. Examine the winning node J whether it passes thevigilance criterion in (6). If the winning node meets thevigilance criterion, the weight vector of the winningnode (wyh ) will be updated according to equation 7.

TJ (I ) Phidden

w h(t + 1) = f(lh A Wh(t)) + (1 _)Wh(t)(6)

(7)

where Phidden is the vigilance parameter at the hiddenlayer. It has a value between 0 and 1. However, if thecondition in (6) is not satisfied, a new hidden node isrecruited to code the input pattern. The weight of thisnew node is initialized to be equal to the input pattern.

h hwi =I (8)

4. Transmit the weight vector of the winning node to thenext layer (cluster layer). Then the choice function ofeach kth node in the cluster layer is computed as

follows:Ck = f(wJ, Wk) (9)

where w = 1 2 h 9

Wk = {Wkl, Wk2,..., Wkh,---, Wk9}

f(wJ, Wk) is calculated according to equation 4.5. Select the winning node K with maximum choice

function value among all the nodes k in the clusterlayer.

653

Input Characters

Upper Central Lower

Hierarchical Hierarchical HierarchicalCross-Correlation Cross-Correlation Cross-Correlation

ARTMAP ARTMAP ARTMAP

Hidden Layer (j)

Input Layer (i)

Input Vector = 11 Input Vector = 12 Input Vector = I3 Input Vector = I9

Fig. 5. Architecture of the hierarchical cross-correlation ARTMAP

K = arg max{Ck} (10)k

6. Compare CK to the vigilance parameter at the clusterlayer (Pcluster). If the criterion in (11) is respected and thewinning cluster node K belongs to the correct class, theweight vector wkh will be updated according to wherethe weight vector wh is originated. If the weight vector

wh is transmitted from the newly recruited J node, wkhwill be updated based on equation 12; otherwise,equation 13 is employed.

algorithm is applied to each character image to reduce thethickness of the character image to its skeleton. Next, eachcharacter block is divided into 9 equal segments and thefeature is extracted from the thinned character image. Inthis paper, the merely feature is a list of directional codes.There are 9 directional codes (Fig. 6): 0.000, 0.125, 0.250,0.375, 0.500, 0.625, 0.750, 0.875, and 1.000.

In the feature extraction process, the starting point of astroke must be identified first, then the directional code isused to traverse along the contour of the character. Fig. 7

shows the directional codes for the Character "m."

CK > Pcluster (1 1)wkht +1) =D(h A Wkh (t)) + (I1- )Wkh (t) (12)

wkh(t + 1) = Wkh(t) (13)

However, if the condition in (11) is not satisfied or thewinning cluster node K does not belong to the correctclass, a new cluster node will be recruited. Then theweight of this new node is initialized according to (14)and the connection between a new cluster node and thetarget output is created.

Wk =WJ (14)

During the recognition process, the class whose clusternode returns the maximum output value is the result of theprediction.

0.250

0.375 0.125

*0 or 1

0.625

Fig. 6. Nine directional codes

IV. PREPROCESSING

The photos of the license plates were taken from manydifferent angles by using 02 Xda II pocket PC. Then eachimage is segmented into individual character blocks. Oncethe image is turned into character blocks, the thinning

[0.000 0.000 0.875 0.750 0.750 1.000 1.000 0.750 0.750 0.750 1.000 0.125],[0.125 0.1250.125 0.125 0.000 0.125 0.000],[0.000 0.000 0.875 0.875 0.875 0.750 0.750 0.7500.750],[0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.675],[1.000 0.750 .0.750 0.750 0.875 0.675 0.750 0.750],[0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.7500.750 0.675 0.750],[0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.750 0.750],[-],[-]

Fig. 7. Directional codes for the character "mn"

654

V. THE EXPERIMENTAL RESULTS

To test the performance of the Hierarchical Cross-correlation ARTMAP, Thirty-two experiments have beenconducted on 2 databases. The first database contains 39no-head fonts of each of 80 characters presently used in aThai document. The thirty-nine no-head fonts are JsBoaboon, Js Chalit, Js Chanok, DB Private, Jasmine, Js 75Pumpu, Kodchiang, Lily, DB Pat Ex, DB Patpong, DBSathom, DB Silom, DB Erawan, DB Chusri, DBChalewhieng, DB Toomtam, Js Amphan, Js Chalewhiang,Js Chusri, Js Jetarin, Js Karabow, Js Khunwai, JsLaongdao, Js Neeno, Js Ninja, Js Noklae, Js Oobboon, JsPranee, Js Prapakom, Js Puchong, Js Pumphuang, JsSamurai, Js Sangravee, Js Sarunya, Js Setha, Js Sirium, JsThanapom, Js Tina, and Js Toomtam. Sixteen experimentshave been conducted on this database. In each experiment,ten fonts are randomly chosen to be used as the trainingdata, while the remaining twenty-nine fonts are used as thetesting data. The second database contains 506 Thai vehiclelicense plates. Another sixteen experiments have beenconducted on this database. These experiments use thesame training sets as the first sixteen experiments, while thetesting data consists of all 506 Thai vehicle license plates.

Fig. 8. Examples of the license plates in the second database

The performance of the proposed neural network iscompared with the fuzzy ARTMAP neural network andtwo of the best commercial OCR software available in themarket, "ArnThai version 2.5" and "ThaiOCR version1.5b." Table II and III show the experimental results for thefirst database and the second database respectively.

TABLE IIEXPERIMENTAL RESULTS FOR THE FIRST DATABASE

Recognition Rate (0/O)Proposed Fuzzy ArnThai ThaiModel ARTMAP OCR

1 86.38 78.45 47.59 41.512 93.41 80.39 43.23 39.013 87.16 75.09 48.10 43.714 93.66 80.47 45.34 39.875 92.03 80.78 45.00 39.226 83.97 75.47 42.41 35.867 94.96 84.57 49.22 43.888 88.19 76.42 44.35 40.569 90.60 76.25 50.39 43.4110 93.97 82.03 49.22 44.0111 93.06 81.72 47.54 43.1912 91.81 80.95 47.84 42.3313 89.91 79.01 48.62 43.6614 87.59 78.06 45.30 42.0315 85.99 77.50 46.25 42.4116 94.40 82.46 48.15 44.87

TABLE IIIEXPERIMENTAL RESULTS FOR THE SECOND DATABASE

Recognition Rate (0/O)Proposed Fuzzy AmThai ThaiModel ARTMAP OCR

1 85.12 74.462 85.85 75.593 85.47 74.204 85.79 75.155 86.09 76.106 86.45 76.537 87.55 77.688 89.04 79.23 48.91 41.369 84.34 74.1810 85.27 74.6111 86.16 75.5612 86.71 78.0813 86.23 76.2814 88.43 78.7115 87.50 77.7916 88.71 78.51

VI. CONCLUSION

The experiments show that the hierarchical cross-correlation ARTMAP can be used successfully to recognizethe characters on Thai vehicle license plates. With itsability to separately train each segment of the network, thehierarchical cross-correlation ARTMAP can achieve muchhigher performance on the experimental databases incomparison to the fuzzy ARTMAP neural network and twoof the best commercial software available in the market.

REFERENCES[1] Y. Cheng, J. Lu and T. Yahagi, "Car license plate recognition based

on the combination of Principal Components Analysis and RadialBasis Function Networks," Proceedings of the 7th InternationalConference on Signal Processing, vol. 2, 2004, pp. 1455-1458.

[2] R. Parisi, E. D. Di Claudio, G. Lucarelli and G. Orlandi, "Car platerecognition by neural networks and image processing," Proceedingsof the 1998 IEEE International Symposium on Circuits and Systems,vol. 3, 1998, pp. 195-198.

[3] X. Jianfeng, L. Shaofa and C. Zhibin, "Color analysis for Chinesecar plate recognition," Proceedings of the 2003 IEEE InternationalConference on Robotics, Intelligent Systems and Signal Processing,vol. 2, 2003, pp. 1312-1316.

[4] K. K. Kim, K. I. Kim, J. B. Kim and H. J. Kim, "Learning-basedapproach for license plate recognition," Proceedings of the 2000IEEE Signal Processing Society Workshop, 2000, pp. 614-623.

[5] M. Raus and L. Kreft, "Reading car license plates by the use ofartificial neural networks," Proceedings of the 38th MidwestSymposium on Circuits and Systems, vol. 1, 1995, pp. 538-541.

[6] J. A. G. Nijhuis, M. H. ter Brugge, K. A. Helmholt, J. P. W. Pluim,L. Spaanenburg, R. S. Venema and M. A. Westenberg, "Car licenseplate recognition with neural networks and fuzzy logic,"Proceedings ofIEEE International Conference on Neural Network,vol. 5, 1995, pp. 2232-2236.

[7] M. Sarfraz, M. J. Ahmed, and S. A. Ghazi, "Saudi Arabian licenseplate recognition system," Proceedings of the 2003 InternationalConference on Geometric Modeling and Graphics, 2003, pp. 36-41.

[8] M. Ko and Y. Kim, "License plate surveillance system usingweighted template matching," Proceedings of the 32nd AppliedImagery Pattern Recognition Workshop, 2003, pp. 269-274.

[9] S. Yohimori, Y. Mitsukura, M. Fukumi, N. Akamatsu, and W.Pedrycz, "License plate detection system by using threshold functionand improved template matching method," Proceedings of the NorthAmerican Fuzzy Information Processing Society, vol. 1, 2004, pp.357-362.

[10] P. Comelli, P. Ferragina, M. N. Granieri, and F. Stabile, "Opticalrecognition of motor vehicle license plates," IEEE Transactions onVehicular Technology, vol. 44, no. 4, November 1995, pp. 790-799.

655