Data-driven Recognition of Self-dual Binary Symbols on ... · Johannes Herwig and Josef Pauli,...

7
Data-driven Recognition of Self-dual Binary Symbols on Curved and Reflective Surfaces Johannes Herwig and Josef Pauli Intelligent Systems Group University of Duisburg-Essen 47057 Duisburg, Germany Email: {firstname.lastname}@uni-due.de Abstract—A robust and effective two-dimensional symbol code is developed that is decodable in a bottom-up fashion imposing minimal constraints from a priori models. For binary encoding of arbitrary data two equally sized, lattice-like arranged, quadratic symbols are proposed which only differ in their complementary brightness distribution. For symbol code recognition the orien- tation parameter of the symbols is estimated from their own higher level texture. Then, normalized matching with a single correlation filter concurrently detects both symbol types, whereby dependently the response is either strongly positive or negative. The filter output is the correlation score whose local maxima are adaptively extracted by morphological dilation. Finally, a fuzzy region-merging approach based on four-neighborhoods restores the encoded bit matrix. I. I NTRODUCTION Most state of the art machine-readable symbol codes were invented for document management systems that support the electronic tracking of paper documents through imprinted data labels [1]. Nowadays, different applications in automated manufacturing have emerged where physical objects of a diverse range of materials and geometries are directly marked with symbol codes in order to control intelligent production lines [2]. Current symbol codes are originally unable to cope with the changed requirements, especially regarding reflective and non-flat component surfaces. Furthermore, minimal size constraints prevent current symbol codes from being applied onto tiny parts hindering the quest for miniaturization. Figure 1 outlines the problem and shows how the herein developed symbol code integrates into an automated part assembly. A. Previous Symbol Code Designs Generally [3], [4], a symbol code is made up of one symbol that represents a binary ”1” and another distinctive symbol for a binary ”0”. The rectangular tag region with the symbol matrix is horizontally and vertically divided into quadratic cells. A binary symbol is drawn into every cell following some binary coding and error correction scheme. Beside an encoding area that is composed of data cells, the tag region contains additional graphical symbols or patterns that guide the recognition and decoding process. These are a finder pattern 0 Copyright 2011 IEEE. Johannes Herwig and Josef Pauli, ”Data-driven recognition of self-dual binary symbols on curved and reflective surfaces”, Im- age and Signal Processing and Analysis (ISPA), 2011 7th International Sym- posium on, S. Loncaric, G. Ramponi, D. Seric, pp. 355-360, 4-6 Sept. 2011, http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6046632&isnumber=6046561 Part-ID: 1234567 Charge: 897654 Date: 01.01.2008 Storage Data Symbol Matrix Laser Marking Registration Code Reading Station Scanner Conveyor Belt Imaged Region Extraction Symbol Grid 0 0 1 0 0 1 0 0 1 0 1 0 1 0 1 0 0 1 0 0 Bit Matrix 0 0 1 0 0 1 0 0 1 0 1 0 1 0 1 0 0 1 0 0 Bit Matrix Fig. 1. Schematic overview of the workflow of symbol codes featuring the proposed symbology. The forward problem starts with the binary encoding of the storage data. Then, the encoded bit matrix is topologically transformed into the symbol matrix, and finally the part is laser marked. The inverse problem begins during component assembly at the code reading station where a video camera scans the symbol codes of randomly rotated cylinder parts. From the imaged region of interest the geometrically and perspectively distorted symbol code is extracted and registered onto an orthogonal but unknown symbol grid from which the encoded bit matrix is deduced. After binary decoding the storage data is restored which triggers the further handling of the part. [5] that localizes the symbol code, an alignment pattern that assists in image rectification and rotation, and a timing pattern that determines the size and position of data cells. Popular examples are the Data Matrix and its siblings and QR Code [6]. Whereas error correction bits absorb reading failures in the encoding area, the failed extraction of a finder, alignment or timing pattern [7], [8] prevents the decoding of all the data cells at once. For example, at the code reading station in figure 1, the symbol code may be translated so that the finder pattern is damaged by a specular reflection which breaks the symbol code recognition completely. Data Glyph [9] is a self- clocking [10] symbol code which abandons finder and timing patterns, because its distinctive symbol shapes are easily identified and separate themselves. With 2 n distinctive shapes, n bits are graphically encoded per symbol. These shapes are consecutively detected by 2 n symbol matching filters. Data Glyph softens the single point of failure problem by integrating typical building blocks into a superficially homogeneous tag region that consists of binary symbols only and wherein encoding and guiding areas are spatially distributed [11].

Transcript of Data-driven Recognition of Self-dual Binary Symbols on ... · Johannes Herwig and Josef Pauli,...

Data-driven Recognition of Self-dual BinarySymbols on Curved and Reflective Surfaces

Johannes Herwig and Josef PauliIntelligent Systems Group

University of Duisburg-Essen47057 Duisburg, Germany

Email: {firstname.lastname}@uni-due.de

Abstract—A robust and effective two-dimensional symbol codeis developed that is decodable in a bottom-up fashion imposingminimal constraints from a priori models. For binary encoding ofarbitrary data two equally sized, lattice-like arranged, quadraticsymbols are proposed which only differ in their complementarybrightness distribution. For symbol code recognition the orien-tation parameter of the symbols is estimated from their ownhigher level texture. Then, normalized matching with a singlecorrelation filter concurrently detects both symbol types, wherebydependently the response is either strongly positive or negative.The filter output is the correlation score whose local maxima areadaptively extracted by morphological dilation. Finally, a fuzzyregion-merging approach based on four-neighborhoods restoresthe encoded bit matrix.

I. INTRODUCTION

Most state of the art machine-readable symbol codes wereinvented for document management systems that support theelectronic tracking of paper documents through imprinteddata labels [1]. Nowadays, different applications in automatedmanufacturing have emerged where physical objects of adiverse range of materials and geometries are directly markedwith symbol codes in order to control intelligent productionlines [2]. Current symbol codes are originally unable to copewith the changed requirements, especially regarding reflectiveand non-flat component surfaces. Furthermore, minimal sizeconstraints prevent current symbol codes from being appliedonto tiny parts hindering the quest for miniaturization. Figure1 outlines the problem and shows how the herein developedsymbol code integrates into an automated part assembly.

A. Previous Symbol Code Designs

Generally [3], [4], a symbol code is made up of one symbolthat represents a binary ”1” and another distinctive symbolfor a binary ”0”. The rectangular tag region with the symbolmatrix is horizontally and vertically divided into quadraticcells. A binary symbol is drawn into every cell followingsome binary coding and error correction scheme. Beside anencoding area that is composed of data cells, the tag regioncontains additional graphical symbols or patterns that guide therecognition and decoding process. These are a finder pattern

0 Copyright 2011 IEEE. Johannes Herwig and Josef Pauli, ”Data-drivenrecognition of self-dual binary symbols on curved and reflective surfaces”, Im-age and Signal Processing and Analysis (ISPA), 2011 7th International Sym-posium on, S. Loncaric, G. Ramponi, D. Seric, pp. 355-360, 4-6 Sept. 2011,http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6046632&isnumber=6046561

Part-ID: 1234567Charge: 897654Date: 01.01.2008

Storage Data Symbol Matrix

Laser MarkingRegistration Code Reading Station

Scanner Conveyor Belt

Imag

ed R

egio

n

Extraction

Symbol Grid 0 0 1 0 01 0 0 1 01 0 1 0 10 0 1 0 0

Bit Matrix0 0 1 0 01 0 0 1 01 0 1 0 10 0 1 0 0

Bit Matrix

Fig. 1. Schematic overview of the workflow of symbol codes featuring theproposed symbology. The forward problem starts with the binary encoding ofthe storage data. Then, the encoded bit matrix is topologically transformed intothe symbol matrix, and finally the part is laser marked. The inverse problembegins during component assembly at the code reading station where a videocamera scans the symbol codes of randomly rotated cylinder parts. From theimaged region of interest the geometrically and perspectively distorted symbolcode is extracted and registered onto an orthogonal but unknown symbol gridfrom which the encoded bit matrix is deduced. After binary decoding thestorage data is restored which triggers the further handling of the part.

[5] that localizes the symbol code, an alignment pattern thatassists in image rectification and rotation, and a timing patternthat determines the size and position of data cells. Popularexamples are the Data Matrix and its siblings and QR Code[6]. Whereas error correction bits absorb reading failures inthe encoding area, the failed extraction of a finder, alignmentor timing pattern [7], [8] prevents the decoding of all thedata cells at once. For example, at the code reading station infigure 1, the symbol code may be translated so that the finderpattern is damaged by a specular reflection which breaks thesymbol code recognition completely. Data Glyph [9] is a self-clocking [10] symbol code which abandons finder and timingpatterns, because its distinctive symbol shapes are easilyidentified and separate themselves. With 2n distinctive shapes,n bits are graphically encoded per symbol. These shapes areconsecutively detected by 2n symbol matching filters. DataGlyph softens the single point of failure problem by integratingtypical building blocks into a superficially homogeneous tagregion that consists of binary symbols only and whereinencoding and guiding areas are spatially distributed [11].

(a) Binary ”1” (b) Binary ”0” (c) Symbols with relative scales

Fig. 2. The proposed binary ”1” and binary ”0” symbols are inverse to eachother regarding their spatial light flux distribution, which 2(c) makes obvious.Black areas represent imprints and white denotes untreated surface areas.

II. PROPOSED SELF-DUAL AND SYMMETRIC SYMBOLS

The herein proposed binary symbols are shown in figure2. Both symbols 2(a) and 2(b) are quadratic which facilitatestheir grid-like arrangement. The internal shape of each symbolconsists of two adjacent, connected and homogeneous regionsof different brightnesses, whereby one of them is shapedlike a right angle and the other is quadratic as figure 2(c)indicates. The binary ”0” symbol is the inverse of the binary”1” symbol regarding their spatial light flux distribution.Hence, the proposition that interchanging brightnesses resultsin the other binary symbol is self-dual as figure 2(c) makesobvious. The binary value is not encoded in the brightnessesthemselves but rather in the orientation of contrast whosedetection is illumination-invariant and reliable, because firstlythe two brightnesses that make up the contrast are redundantlymeasured at every pixel per homogeneous region, and secondlyboth regions comprise roughly the same area with a 5 : 4 ratio.Moreover, because the binary symbols are self-dual, there isonly a single matching filter necessary that simultaneouslydetects both symbols, whereas with the previous art of self-clocking symbol codes two filters (2n with n = 1 for onebit per symbol) are needed. The symbol design behavesnicely towards out-of-focus blur and geometric distortionsdue to curved and dome-shaped surfaces or slight perspectivewarping, because the black imprinting areas expose symmetricproperties. Effects of blurring and perspective warping aredamped and on curved surfaces the symbols appear locallyflat. Other approaches [12], [13] model these separately.

A. Industrial Prototypes and Data Densities

Figure 3 shows component parts featuring the proposedsymbol code. Figure 3(a) shows a cylinder with a symbolcode area of 7mm2 which contains a 21× 21 symbol matrixwith each symbol cell measuring approximately 33µm2. In asecond version the same code was burned into a 5mm2 areawith symbol cells of approximately 25µm2. Figures 3(b) and3(c) show the heads of two different types of valves. Bothvalves have a diameter of only 6mm. The symbol cells are70µm2 and less than 40µm2, respectively.

Next, the practically achieved data densities are discussed.Roughly speaking, the underlying bit matrix which corre-sponds to the 7mm2 symbol code shown in figure 3(a) isdivided into 3 × 3 blocks of which 3 bits are reserved foraddressing, and another two are parity bits, so that only4 bits encode real data. Here, a symbol block fits exactly

(a) Cylinder part

(b) Head of valve with notch

(c) Dome-shaped head of valve

Fig. 3. Industrial prototypes featuring different symbol code layouts.

TABLE ICOMPARISON OF DATA DENSITIES WITH OTHER POPULAR SYMBOLOGIES.

Proposed5mm2

Proposed7mm2

DataMatrix

QR Code DataGlyph

Bytes per square inch 490 220 366 241 311Error correction in % 20 20 20 25 20Code size/version - - 72× 72 69× 69 -Resolution in dpi 900 900 300 300 380Symbol size in dots 9 12 4 4 5Symbol size in µm ≈ 25 ≈ 33 ≈ 33 ≈ 33 ≈ 33

into a 1mm2 area, and hence the data density is 4 bits permm2. However, in the real world the effective bit rate islower due to the presence of error correction bits. Table Icompares the density of the proposed symbol code with otherpopular symbologies at a redundancy of 20% - 25%. Foreach symbology the printing resolution is adjusted, so thatthe symbol cells measure about 33µm at their recommendedminimal dot size. For the proposed symbol code the actuallyused laser marking resolution and symbol size in dots aregiven. The data density of the proposed symbology reachesthe QR Code but cannot catch up with Data Matrix and DataGlyph. The lower data rates are due to the relatively highshare of addressing and parity bits in the proposed encodingscheme. However, these add to robust decoding capabilitiesand circumvent the global registration of geometric distortions.Anyway, the figures in table I are theoretical because only theproposed symbology was practically evaluated. The markingquality of the 5mm2 code as seen in the top two rows of figure4(a) provides evidence that decoding Data Matrix or QR Codeat the same resolution seems tough at least.

III. PROPOSED DATA-DRIVEN SYMBOL RECOGNITION

The robustness of the distortion-tolerant symbol designallows for reliable bottom-up processing. Thereby the imposedconstraints of models are kept at a minimum to allow for localadaptivity. In the data-driven approach only minor a prioriknowledge about the pattern is needed. Model parameters aresubsequently derived from the data itself through hierarchicalmodeling of robust entity domains that expose sufficientsimilarities, regularities and redundancies.

A. Locating the Supposed Midpoints of Symbol Cells

Firstly, the symbols are detected in the input image I(x, y).Because the symbols are distortion-tolerant and appear locallyflat on curved surfaces, the input image needs not to beregistered or pre-processed. The rotation parameters and thesize s of the symbols in the input image are assumed fixedfor most applications in industrial automation. However, laterin this paper section III-C suggests how to dynamically derivethese parameters from the input images. A template Ts(x′, y′)with running indices x′ = 0, . . . , s−1 and y′ = 0, . . . , s−1 iscreated according to the qualitative layout from figure 2. Then,normalized template matching Cs(x, y) = I(x, y) ? Ts(x

′, y′)is applied, which is implemented [14] as the cross-correlationof the input image I(x, y) with the template Ts(x′, y′):

Cs(x, y) =

∑x′,y′ (T ′s(x

′, y′) · I ′s(x+ x′, y + y′))√∑x′,y′ T ′s(x

′, y′)2 ·∑

x′,y′ I ′s(x+ x′, y + y′)2.

Thereby the symbol template Ts(x′, y′) is initially normalized:

T ′s(x′, y′) := Ts(x

′, y′)− s−2 ·∑x′′,y′′

Ts(x′′, y′′)

and also every pixel of the input image I(x, y) is normalized:

I ′s(x+x′, y+y′) := I(x+x′, y+y′)−s−2·

∑x′′,y′′

I(x+x′′, y+y′′),

whereby both normalizations are according to the mean in theemployed template windows.

Data Matrix [7] and QR Code [8] require homogeneouslighting for reliable binarization, but here the normalizedmatching is invariant to local lighting changes, and sincethe binary symbols are self-dual a single application of thecorrelation filter gives the scores for both symbols at once.Therefore, the matching result readily defines the correlationscore without having to develop reasonable fusion strategiesthat would integrate different error measures from different fil-ter outputs. The correlation result Cs(x, y) has values between{−1.0, ..., 1.0}, whereby scores > 0.0 indicate a binary ”1”symbol and scores < 0.0 indicate a binary ”0” symbol.

For example, figure 4(a) shows three rows of inputs I(x, y)of symbol codes which belong to different prototypes of thecylinder part from figure 3(a). The symbol cells in the top rowof test inputs are 6× 6 pixels, 8× 8 pixels in the second row,and 16×16 pixels in the bottom row. The different sizes s aredue to different optics and working distances, and because the

(a) Input I (b) Matching |Cs| (c) Dilation Ds (d) Midpoints Ms

Fig. 4. Image processing chain for the extraction of symbol cell midpoints.

first two rows feature the 5mm2 but the bottom row showsthe 7mm2 symbol code. The absolute values |Cs(x, y)| ofthe normalized template matching are shown in figure 4(b).In figure 5 the processing results of a cropped part of thebottom row of figure 4 are shown. The input 5(a) and thecorrelation result 5(b) show darker and brighter dot-shapedareas at the symbol midpoints with sharp cut-offs betweenthe symbol cells. These properties are further clarified byfigure 5(c) which zooms into two adjacent symbol cells forwhich real correlation scores are exemplarily given. In orderto simplify the problem of midpoint extraction to local maximadetection, an image with absolute correlation values |Cs(x, y)|is derived, wherein the symbol midpoints can be identified asbright spots (figures 4(b) and 5(d)).

The values of |Cs| are morphologically dilated using arectangular mask Rs of the same size s as the symbol template,so that local correlation maxima are flattened over the symbolcell areas. Figures 4(c) and 5(e) show the dilation resultsDs(x, y) = |Cs(x, y)|⊕Rs(x

′, y′). The input image 4(a) of thebottom row was captured under homogeneous lighting. Notethe regularity of the tiles and their high overall correlationscores in 4(c) which easily distinguishes the symbol code areafrom the background. The image quality of the above two rowsis lower due to inhomogeneous lighting, specular reflection,slightly out-of-focus capture, and poor quality of the lasermarkings. Therefore, a more irregular tiling pattern emergeswith generally lower correlation scores and parts of the symbolcode area already merge into the background texture.

In order to extract the supposed midpoints from the dilatedimage Ds(x, y), it is compared pixelwise with the originallyobtained absolute correlation values |Cs(x, y)|, and only thosepixels that have the same value in both images are labeled asmidpoints. Examples of the resulting midpoint image Ms(x, y)are shown in figures 4(d) and 5(f). The advantages of this

(a) Input I (b) Matching Cs (c) Zoomed Cs (d) Absolute |Cs|

(e) Dilation Ds (f) Midpoints Ms (g) Thresh. = 0.8 (h) Thresh. = 0.9

Fig. 5. This is a crop of the bottom row of figure 4. Figure 5(c) shows amagnified part of the correlation map 5(b) featuring two adjacent binary ”0”and binary ”1” symbols. The values of the marked pixel locations in 5(c)are from left to right: −0.02, −0.89, −0.75, −0.01, −0.25, −0.54, +0.20,+0.85, +0.72, +0.06, and −0.41. In the dilated image 5(e) the correlationscores lie between−0.85 and−0.98 for binary ”0” and +0.85 and +0.95 forbinary ”1” symbols, respectively. Figures 5(g) and 5(h) show the superiorityof the proposed midpoint detection 5(f) over global thresholding.

approach to local maxima detection are that it is parameter freeand that it is locally adaptive. The alternative results of a globalbinarization of the absolute correlation values |Cs(x, y)| withdifferent threshold parameters are shown in figures 5(g) and5(h). Whereas these results only give over-segmented or under-segmented point clouds, the morphological maxima detectiongives exactly localized midpoints.

In summary, the finally extracted binary image Ms(x, y)can be written as the Kronecker Delta in the form δ = 1A :X × Y → {0, 1} with the indicator function 1A and the setA = {(i, j) ∈ X × Y |i = j} where X and Y are images:

δ = 1A : Ds × |Cs| −→ {0, 1} :=Ms

with Cs = I ? Ts and Ds = |Cs| ⊕ Rs. This processingchain, where the results of a contextualizing operation Ds

which involves |Cs| are back-projected onto the original |Cs|,realizes adaptive behavior in a data-driven approach.

B. Restoring the Encoded Bit Matrix from Distorted Midpoints

The original orthogonal layout of the internal lattice-likestructure is geometrically distorted due to perspective warp-ing, curved component surfaces or noisy localization of theextracted midpoints. However, an a priori assumption is thatlocally the orthogonal structure remains loosely intact. There-fore, the undistorted symbol grid can be restored in a bottom-up fashion without the need for global warping. Bottom-up processing is suspectible to noise and leads to unreliableresults, but the distortion-tolerant design of the symbols andthe correlation scores allow for data-driven regularization.

Figure 6 schematically depicts the approach. The detectedmidpoints Ms(x, y) are copied into a list structure which issorted by their absolute correlation scores. The first element islabeled as a true midpoint and it is pointed to the correspond-ing location L1 in the midpoint image Ms(x, y) as figure 6(a)shows. The a priori assumption is that the orientation of paral-lel rows and orthogonal columns of the symbol code remains

(a) At the start (b) . . . running . . .

(c) . . . running . . . (d) Reject outliers

Fig. 6. Subsequently connecting local four-neighborhoods of midpoints.

locally intact. Hence, starting from the current location thepointer is subsequently pointed to its four-neighborhood ofstep-size s as Pw, Ps, Pe, and Pn in figure 6(a) indicate. Ifthe pointer P? does not reach a neighboring midpoint , then itis searched in a window around the guessed location which is20% of the step-size. If the midpoint is found, then it is addedto its parent and otherwise a new midpoint is placed there.However, if one of the neighbors already has a parent, thenthe current midpoint is discarded as a false midpoint (figure6(d)). In this manner the list is traversed from top to bottom. Inthe end the symbol grid can be two-dimensionally registeredand restored simultaneously by traversing the finally mergedfour-neighborhoods starting from the midpoint with the highestcorrelation score. In order to restore the bit matrix the plus orminus sign of the correlation scores Cs(x, y) at the extractedmidpoint locations indicate the bit value. Restored bit matricescorresponding to the extracted midpoints Ms from figure 4 areshown in figure 7 whereby binary ”1” and ”0” are labeled bywhite black squares, and false detections are marked red andblue. The symbol recognition rates are 94%, 84% and 100%,respectively. The running times for the algorithm depend onthe image size and noise but were between 100ms and 250mson a 2GHz Pentium IV machine.

An alternative approach is the zig-zag method which walksthe midpoints row by row. But walking rows accumulateslocation errors and if adjacent rows are close to each otheror if there are false midpoints, then the logical rows of thesymbol matrix easily interleave into each other. By mergingindividual four-neighborhoods no errors are accumulated andareas without symbols, like the notch of the valve from figure3(b), are treated with the proposed approach.

C. Estimating Symbol Size and Rotation Parameters

In machine vision applications the working distances areusually fixed and hence the symbol size is predetermined. Aslong as the parts do not rotate in the sensor plane, there isonly a translation of the symbol code as is depicted withthe cylinder parts at the code reading station in figure 1.

However, when the part randomly rotates in the image plane,like the heads of valves in figures 3(b) and 3(c), then therotation is different for every image. In order to estimate theorientation the predominant set of discontinuous line edges,which originate from the symbol imprints, is chosen as a robustfeature, because it redundantly emerges from the regularity ofthe orthogonally arranged symbols and appears as a strongcharacteristic of the symbol code. Line segments are extractedvia the Hough Transform whose input is a binary image ofline edges that is obtained as follows. The Sobel operatorfilters gradients in the y-direction. Its absolute values arenormalized into an 8-bit image, which is then automaticallythresholded using Otsu’s method [15], [14]. This approachis chosen over the Canny detector because it is parameterfree and because Canny tends to link false diagonal edgesbetween adjacent symbols which only appear due to inaccuratelaser markings. For the same reason, the Sobel operator isapplied in the direction of one axis, only. In the binaryimage the probabilistic Hough Transform [16] implementedby [14] finds line segments which are represented by startingand ending points. Apart from angle resolution (= 10◦) andaccumulator threshold (= 50), there are also parameters forthe minimum line length (= 180 pixels) and the maximumgap (= 35 pixels) between line segments lying on the sameline. Thereby the minimum line length is roughly 2/3 of theexpected symbol code size, and the maximum gap is 2.5 timesthe expected symbol size in order to successfully bridge thediscontinuities between adjacent symbols. Results with thesame fixed parameters are shown in figure 8. The first 50 linesegments are extracted, and their orientations are transferedinto a histogram and the bin with the most counts gives thepredominant orientation of line segments. Finally, the symbolsize is estimated by brute-force which matches smaller parts ofthe image with a set of properly rotated templates of predefinedsizes s, and s with the highest correlation score is selected.

IV. EXPERIMENTS

In this section the robustness of the proposed symbols andtheir recognition likelihoods are evaluated.

A. Scale-invariance

The effect of a wrong template size estimate on the recog-nition result is analyzed. The symbol code image from thebottom row of figure 4 was repeatedly downscaled in orderto generate a sufficiently large data set containing symbols

(a) First row (b) Second row (c) Third row

Fig. 7. The resulting bit matrices corresponding to the rows of figure 4.

(a) Cylinder edges and line segments

(b) Valve edges and line segments

Fig. 8. Estimation of rotation parameters from large scale structures ofdiscontinuous edge lines originating from the symbol imprints. The left figureshows the binarization of 3(a) and the extracted Hough lines. The results forthe valve 3(c) are on the right and were obtained with the same parameters.

of different sizes from about 6.4 synthetic pixels up to ap-proximately 16 original pixels. These seven images containing7×(21×21) symbols were processed with fixed template sizesof 12, 16, and 20 pixels. The results are shown in figure 9. Forevery curve the standard deviation from the mean correlationscore is indicated by error bars. From the blue and purplecurves it can be concluded that over-estimating the symbol sizeby up to 20% results in significantly lower correlation scoreswhen compared with the correct estimate (purple and bluecurves at 16 pixels, and also the blue curve could be comparedat 12.8 and 16 pixels), but still the scores are greater than 0.6which is regarded as sufficient for reliable symbol recognition.However, when the symbol size is under-estimated (greencurve) the correlation score equals that of the correct templateestimate (green and blue curves at 16 pixels), and also symbolsizes down to 9.6 pixels can be matched reliably. Hence, witha template size of 12 pixels real symbol sizes between about10 and 16 pixels can be recognized without the need of a scale-space approach. Therefore, under-estimating the real symbolsize is a preferable strategy when symbol sizes are expectedto vary due to different working distances. It behaves nicelybecause the symbol design is center-symmetric.

Fig. 9. Evaluating the effect of wrong template size estimates.

B. Rotation-invariance

The effect of a wrong rotation estimate on the symbolrecognition is analyzed. The test data was generated similarlyas previously. Three original images of flat symbol codesunder zero degrees rotation were captured, whereby one ofprototype was imaged directly after the laser marking when itwas still covered with dirt particles that are usually removedusing pressure-air. These images were rotated syntheticallyby steps of one degree. Then the proposed image processingchain was applied for each series with the correct templatesize parameter but the false assumption of zero degreesrotation. The results are presented in figure 10. Generally,the correlation score gradually decreases with the error inthe rotation estimate. However, the larger symbols can berecognized with correlation scores well above 0.6 even at anerror of 20 degrees, and even the dirty symbol code has itsmidpoints reliably detected up to 15 degrees rotation error.Both the symbol size and the quality of the markings seem tobe directly related to the rotation-invariance property, since allthe curves are qualitatively similar but have their correlationscores degraded by some constant factor. Hence, improvingthe quality of the markings and enlarging their size directlyenhances the rotation invariant capabilities of the approach.

C. Borderline Cases

Figure 11 shows borderline cases. Although still decodablethese images suffer from a combination of degradations likeperspective distortion, limited depth-of-field, strong lightingchanges and marking errors. Symbol codes with smaller per-spective distortion but other severe damages like dirt (see sec-tion above) or out-of-focus blur (figure 12) remain decodable.

V. CONCLUSION

In this paper a self-clocking and self-dual binary symbolcode has been proposed. Both symbols can be detected by asingle normalized template matching operation without pre-processing the input image. The lighting invariant encodingof the binary value as the contrast of the symbol combined

Fig. 10. Evaluating the effect of wrong rotation estimates.

(a) Input I (b) Dilation Ds (c) Midpoints Ms

Fig. 11. Borderline cases where only about 50% of a bit matrix were restored.

with normalized detection handles specularities and unevenillumination. Because the symbols are distortion-tolerant noglobal warping is necessary, but the registration is donebottom-up by subsequently merging robust symbol neighbor-hoods. The rotation parameter of the symbol code can beautomatically estimated via the Hough Transform. The pattern

(a) Input I (b) Overlayed Ds (c) Bit matrix

Fig. 12. Out-of-focus captures without severe perspective distortion remaindecodable. Figure 12(b) shows the dilation Ds overlaid with the connectedsymbol locations, and 12(c) proves that 94% of the bit matrix were restored.

recognition method copes with different sources of systemnoise like inaccurate laser marking, non-homogeneous lightingand misfocused images. The a priori model only consists ofqualitative information about the symbols and their topologicallayout. The symbol size can be estimated by brute-force, sothat the presented approach is essentially non-parametric.

REFERENCES

[1] J. Z. Gao, L. Prakash, and R. Jagatesan, “Understanding 2D-barcodetechnology and applications in m-commerce - design and implementa-tion of a 2D barcode processing solution,” in 31st Int. Comp. Softw. andApp. Conf., vol. 2, 2007, pp. 49–56.

[2] W. van Gils, “Two-dimensional dot codes for product identification,”IEEE Transactions on Information Theory, vol. 33, no. 5, pp. 620–631,1987.

[3] E. Ouaviani, A. Pavan, M. Bottazzi, E. Brunelli, F. Caselli, and M. Guer-rero, “A common image processing framework for 2D barcode reading,”in 7th Int. Conf. on Image Process. and Its App. (Conf. Publ. No. 465/2).IEEE, 1999, pp. 652–655.

[4] T. Pavlidis, “A new paper/computer interface: two-dimensional symbol-ogies,” in Proc. 15th Int Pattern Recogn Conf, vol. 2, 2000, pp. 145–151.

[5] H. Kato, K. T. Tan, and D. Chai, “Development of a novel finder patternfor effective color 2d-barcode detection,” in Proc. Int. Symp. Parallel andDistributed Processing with Applications, 2008, pp. 1006–1013.

[6] H. Kato and K. T. Tan, “Pervasive 2D barcodes for camera phoneapplications,” IEEE Pervasive Computing, vol. 6, no. 4, pp. 76–85, 2007.

[7] Y.-F. Cheng, “A scanning method for dotted data matrix,” in Proc. EighthInt. Conf. Intell. Systems Design and App., vol. 2, 2008, pp. 179–183.

[8] Y. Liu and M. Liu, “Automatic recognition algorithm of quick responsecode based on embedded system,” in Proc. Sixth Int. Conf. IntelligentSystems Design and Applications ISDA ’06, vol. 2, 2006, pp. 783–788.

[9] D. L. Hecht, “Printed embedded data graphical user interfaces,” IEEEComputer, vol. 34, no. 3, pp. 47–55, 2001.

[10] D. S. Bloomberg, D. L. Hecht, R. F. Tow, and L. P. Flores, “Self-clockingglyph shape codes,” US Patent RE 38 758, Jul. 19, 2005.

[11] D. L. Hecht and R. G. Stearns, “Explicit synchronization for self-clocking glyph codes,” US Patent 5 449 895, Sep. 12, 1995.

[12] H. Wang and Y. Zou, “Camera readable 2d bar codes design anddecoding for mobile phones,” in Proc. IEEE Int Image Processing Conf,2006, pp. 469–472.

[13] E. Ohbuchi, H. Hanaizumi, and L. A. Hock, “Barcode readers using thecamera device in mobile phones,” in Proc. Int Cyberworlds Conf, 2004,pp. 260–265.

[14] (2010) OpenCV software library. [Online]. Available: http://opencv.willowgarage.com

[15] N. Otsu, “A threshold selection method from gray-level histograms,”IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp.62–66, 1979.

[16] C. Galamhos, J. Matas, and J. Kittler, “Progressive probabilistic houghtransform for line detection,” in Proc. IEEE Computer Society ConfComputer Vision and Pattern Recognition, vol. 1, 1999.