Instant Segmentation and Feature Extraction for Recognitionof Simple Objects on Mobile Phones

Andreas Hartl, Clemens Arth, Dieter SchmalstiegInstitute for Computer Graphics and Vision

Graz University of Technology, Austria{hartl,arth,schmalstieg}@icg.tugraz.at

Abstract

Object detection and recognition algorithms are an inte-gral part of the architecture of many modern image process-ing systems employing Computer Vision (CV) techniques.In this paper we describe our work in the area of segmen-tation and recognition of simple objects in mobile phoneimagery. Given an image of several objects on a structuredbackground, we show how these objects can be segmentedefficiently and how features can be extracted efficiently forfurther object recognition and classification. We prove thealgorithms presented are useful given a set of test cases,and we show that the algorithms discussed can be used forinstant object segmentation and recognition in a real-worldapplication on ordinary off-the-shelf smartphones.

1. Introduction

Computer vision (CV) based object detection and recog-nition has become an integral part of many modern imageprocessing systems. Especially in the area of CV for indus-trial applications, many of these systems are based on al-gorithms that put heavy requirements on the hardware usedand the image capturing conditions. This observation is stillvalid, even if the objects to be detected or recognized are ofrelatively simple shape, e.g. screws on a conveyor belt, sim-ple workparts for manufacturing, medical pills, etc.

As a result of the complexity of the task and the needfor significant amounts of computational resources, the useof CV based detection and recognition systems on mobilephones has not been widely investigated yet. Nevertheless,the deployment of such algorithms in mobile scenarios isstill an important task, and the steadily increasing powerof mobile devices opens up a huge field of applications forthese types of algorithms in mobile phone scenarios. Thereis a strong motivation to push forward the development ofalgorithms suitable for performing these tasks on commonoff-the-shelf mobile phones.

In this work we discuss an approach to efficiently use ob-ject detection and recognition algorithms together on mo-bile phones. We present a way for efficient detection andsegmentation of simple objects on a structured background.Based on the result of segmentation, a set of features is es-timated which can be used for object classification, eitheron site with an on-board database, or by using an onlinedatabase on the Internet. This setup allows for the genera-tion of powerful mobile applications.

The rest of the paper is structured as follows. In Sec-tion 2 we give a short overview of CV on mobile phones.In Section 3 we discuss our approach for object segmenta-tion on a structured background. Section 4 contains a closerdescription of the feature extraction and classification ap-proach, followed by an extensive experimental evaluationin Section 5. Final remarks and an outlook on future workis finally given in 6.

2. Related WorkSince mobile phones have only recently become compu-

tationally powerful devices, the use of CV algorithms onmobile phones does not have a long history.

There is, however, some related work from a number ofdifferent research areas. Also, since mobile phone hardwareis essentially embedded hardware, many algorithms havebeen developed in the embedded CV community1.

Face detection for example is a popular research topic.Some camera phones have this feature nowadays, sincemost ordinary cameras have face detection algorithms al-ready built in hardware. Theocharides et al. [18] proposethe hardware implementation of a face recognition systembased on neural network algorithm originally developed byRowley et al. [14]. Rahman et al. presented an algorithmrunning in software on an TI OMAP3430 platform [13]. Thealgorithm performs color classification using Gaussian mix-

1Note that in the following we will not discuss approaches which of-fload the CV work to a server, but only refer to related work performingthe CV task on-site right on the mobile device.

117978-1-4244-7030-3/10/$26.00 ©2010 IEEE

ture models and basically detects skin-like areas as faces.The authors also present a hybrid system combining theiroriginal approach with the very popular Viola-and-Jones al-gorithm [12]. Ng et al. describes a system for face verifica-tion on mobile phones based on minimum average correla-tion filters [10]. A study on iris detection and verification isconducted by Park et al. [11].

In the field of Augmented Reality (AR) on mobilephones, Wagner et al. present and evaluate natural fea-tures for object tracking in real-time on mobile phones[20]. Klein and Murray recently presented a version oftheir PTAM software on an iPhoneTMfor small workspaceself-localization and mapping (SLAM) [8]. For the task ofoutdoor AR, Takacs et al. suggest an approach using fea-tures subdivided into cells called Loxels [17]. An approachfor self-localization in large-scale environments was pre-sented recently by Arth et al. using point cloud reconstruc-tions of city areas [1]. A system for guiding visually im-paired people in traffic situations was presented recently byIvanchenko et al. [7]. The system is running on a Nokia N95mobile phone, detects two-striped crosswalks and guidesthe user across streets.

The collectivity of related work mentioned above is notexhaustive. Other approaches include OCR [9, 22] or mu-seum guidance [5] for example. However, it is apparent thatCV has started to conquer the field of mobile phone devel-opment in recent years. Although there are applications forthe most obvious tasks, to the best of our knowledge there isno basic study in the area of object segmentation and recog-nition on mobile phones available yet, which is comparableto our work presented in this paper.

3. Object SegmentationWhen using mobile phones as image capturing devices, a

non-fixed acquisition setup is typically the case. Segmenta-tion methods, thus, should be invariant to changes in objectposition, orientation and scale. Naturally, perspective dis-tortion, varying lighting conditions and blurred input im-ages pose additional problems. Thus the goal is to developan approach which is able to work despite adverse environ-mental conditions, and deliver results reliably and instantlyon a mobile phone.

For solving the task successfully and instantly on mo-bile phones, we propose a method for robust segmentationbased on a marker target with a structured background. Acheckerboard background is chosen to allow segmentationof arbitrarily colored objects in a robust manner (see Figure1). The marker target itself can be detected efficiently us-ing readily available marker tracking software, like Studier-stube or ARToolkitPlus2. Moreover, the marker itself has afixed known size, which enables us to accurately measure

2http://studierstube.icg.tu-graz.ac.at

Figure 1. A frame marker with a generic object. The detectedobject contour is marked with green color.

the object dimensions. During the marker detection step,a homography H is calculated which essentially maps themarker in the image to the undistorted view of the marker (aplane-to-plane mapping [6]). This homography H is usedto define a rectangular region of interest (ROI) within theoriginal image to reduce the amount of data for segmenta-tion and to facilitate parameter selection. Note that we donot rectify the image prior to segmentation. The final resultsare acquired by local adaptive thresholding and applying aset of morphological operations on a grayscale input image:

Mseg = (¬(MTh1• SE1) ◦ SE2) • SE1, (1)

In Equation 1, MTh1denotes a mask obtained by local

adaptive thresholding with a neighborhood size of (2·wh)+1, and SE1 and SE2 denote square-shaped and disk-shapedstructuring elements of length (2 ·wh) + 1 and (2 ·wh) + 3,respectively. The symbol ¬ is used for mathematical inver-sion and the symbols ◦ and • denote morphological openingand closing. For local adaptive thresholding we use an effi-cient method originally proposed by Shafait et al. [15]. Toextract contours, region labeling with integrated boundarycomputation is carried out following the linear-time methodproposed by Chang et al. [3].

The choice of the single parameter wh has turned out tobe noncritical. It may be automatically selected by usingprior knowledge about the image position of the checker-board area and its amount of squares in each direction, ifnecessary. However, the length of the checkerboard patternmust be chosen with care, since it influences the quality ofsegmentation to a large extent. In turn, this choice is de-pendent on the desired image resolution, because a suffi-ciently sharp image of the background pattern is required3.

3As a good tradeoff between different image resolutions, segmentationaccuracy, runtime and usability, setting sqw to 0.6mm is suggested.

18

Needless to say that the approach only works under the as-sumption that objects do not contain a similar pattern to thechosen background.

We have found the proposed method to be sufficiently in-variant to perspective distortion and variable lighting. Thealgorithm relies on a series of relatively simple process-ing steps and the amount of input data can be consider-ably reduced. These properties make the approach partic-ularly interesting for non-fixed setups using mobile deviceswith limited resources. Although the use of a marker withcheckerboard background seems to be prohibitive, there aremultiple justifications for this. For measuring object size,some reference measurements are necessary which can sim-ply be provided by knowledge about the real marker di-mensions. Preceding tests have shown that even popularand powerful segmentation algorithms like Graph Cuts orMSER are not able to deliver satisfying segmentation re-sults for arbitrarily colored objects on, for example, uniformwhite background. Moreover, these algorithms can hardlybe used as efficiently as our approach in terms of computa-tional and memory requirements.

4. Feature Extraction and Classification

For object classification, several features and propertieshave to be estimated from the segmented object. Typi-cal features include object size, color and shape, but alsomaterial related properties like surface glossiness or objecttransparency. However, the latter properties are hard to esti-mate accurately, so the former ones must serve sufficientlyfor reliable classification. For calculation of each feature,efficient solutions are needed to allow instant computationon mobile devices.

4.1. Object Size

Correct measurements of the object size can only beacquired if invariance to perspective distortion is ensured.This can be facilitated by rectification of the object (or itsboundary) according to the previously calculated homogra-phy H . Still, some assumptions have to be made accordingto the specific object category. While it is relatively simpleto calculate the length and width of circular or rectangularobjects, a more sophisticated method is needed for takingsuch measurements on more general objects.

First, we determine the direction of maximum variancewithin the segmented region. Because the boundary can bedistorted due to errors during segmentation, a subset of allregion points is taken for increased robustness. This sub-set of points is selected and rectified using the homographyH . Subsequently, the direction of maximum variance v1

= [v1; v2] is determined by analysis of the covariance ma-trix C [21]. Generally, we define the length of an objectas the extension of its boundary along the major axis of its

Figure 2. Variance estimation for a generic object (beige color) isshown on the left, the general definition of length and width isdepicted on the right side.

projection. The extension in the perpendicular direction isthen defined as width. Both measurements are acquired byrecording the minimum and maximum values of the corre-sponding projection. As mentioned above, the definition oflength and width might vary according to the given objectcategory (see Figure 2 for an illustration of the measurementprocess).

The square size sqw of the checkerboard pattern, thatlimits the accuracy of segmentation, also limits the accu-racy of the measurements. Under the assumption that thetarget entirely fills the viewable area and does not exhibitother kinds of distortion, this is basically the only factor in-fluencing the expected error. If we assume that objects maybe correctly segmented under the previous conditions, themaximum error emax can be estimated to be:

emax = 2 · sqw (2)

Using an ordinary ruler as a measurement tool, it is ratherdifficult to measure lengths below 1 mm, and arbitrary ob-ject shapes further aggravate measurements. For these rea-sons, the accuracy of the proposed method can be expectedto be better than what can be obtained with a ruler.

The most expensive step in the proposed method for sizeestimation is the determination of the direction of the majoraxis. By using a suitable spacing for sampling region points,which in turn is dependent on the current region size, the re-quired effort can be considerably reduced with little degra-dation in accuracy. So, the problem can still be computed ina reasonable amount of time for larger region sizes. Due tothe low amount of memory resources needed and the pos-sibility to do all calculations using fixed-point arithmetics,if necessary, the algorithm is ideally suitable for mobile de-vices.

4.2. Object Color

Methods for object color estimation have to be tolerantto varying lighting conditions, and the results should corre-spond to human perception as closely as possible. In ourapproach, as a first step we reduce the influence of varyinglighting conditions by applying a method for local whitebalance. This method is based on reference measurements

19

on the white border of the marker-based target and apply-ing a subsequent correction to each segmented pixel accord-ingly. For information on the algorithm used, the interestedreader is referred to the work of Susstrunk et al. [16].

Depending on the given task, usually a restricted dis-crete subset of colors is assumed to cover the entire rangeof possible object colors. Facilitating this assumption andfor reasons of efficiency, we perform color estimation usingan sRGB lookup table containing the set of possible objectcolors. This lookup table is generated by defining a set ofsuitable colors manually, and partitioning the color spacewith heuristic thresholds accordingly (including a class forall colors not covered by any manually chosen one). Al-ternatively, suitable color classes can be generated by cal-culation from a set of training samples in an offline stage.For assigning a discrete color to each pixel, we evaluate the∆ECIE00 color distance metric in CIE LAB space (for de-tails refer to the work of Vik [19]). Through assigning eachpixel the most likely color, we get a per-pixel classificationresult for a segmented object.

The classification results are aggregated in a histogram,and the final color estimation result is drawn from subse-quent analysis of this histogram information. Currently, weassume objects to be of one single or two dominant colors,and do not expose color gradients. We sort the histogram hcin descending order and apply the four measurements:

single color (i) coverage: ci =hc(0)∑hc

(3)

two color (i, j) coverage: ci,j =(hc(0) + hc(1))∑

hc(4)

single color (i) significance: si =hc(0)− hc(1)

hc(0)(5)

two color (i, j) significance: si,j =1

si(6)

An object is assumed to be of a specific color, if one of thecoverages is above a threshold thc. The products

pi = ci · si (7)pi,j = ci,j · si,j (8)

are used to decide, whether one or two dominant colors arepresent. If pi > pi,j , the result is the corresponding labeli of the maximum entry in hc. Otherwise two colors areassumed and the labels i, j corresponding to the first twoentries in hc are reported.

The proposed method is particularly suited for non-fixedsetups, because it is able to reduce the influence of ambi-ent lighting and the results correspond to human perceptionof color. The approach is rather efficient and thus instantlycomputable on mobile phones, since costly color conver-sions and distance computations are avoided. The size of

the lookup table (sLT entries per channel) can be chosendepending on the memory resources available, thus the ap-proach scales well for different hardware setups. Note thatcolor classification for more than two colors can easily beperformed using a more evolved histogram analysis.

4.3. Object Shape

The boundary of the segmented region can serve as a ba-sis for the estimation of object shape. As for determiningobject size, shape estimation should be invariant to changesin translation, rotation, scale, and should also tolerate a cer-tain degree of perspective distortion. In this respect, we pro-pose a modified pairwise geometric histogram (PGH), orig-inally proposed by Evans et al. [4] for this task. The PGHdescriptor features good descriptive power and also allowsfor efficient shape matching.

In the PGH descriptor, oriented line segments are investi-gated. Their relative orientation and perpendicular distanceis analyzed and this information is collected in a 2D his-togram of size D · A. The set of possible angles and dis-tances is mapped onto the histogram by accumulation ofoccurrences. During the process, each line is used as areference line and the angle, as well as the perpendiculardistance, is computed to all the other lines. Every line isrepresented as a histogram and the accumulation of all his-tograms represents the shape of the object (see Figure 3).The boundary of a segmented object has to be approximatedby a polygonal representation. An efficient implementationof Critical Point Detection (CPD) proposed by Zhu et al.[23] can be used therefor.

The PGH is not scale invariant in its original form. How-ever, limited scale invariance may be achieved by applica-tion of a suitable and stable similarity metric [2]. For effi-ciency reasons we assume that the maximum distance for aPGH can be estimated from the rectified contour. Thus, thescale is estimated by searching for the maximum distanceδmax of the boundary from its centroid, and the maximumPGH distance is computed as:

δPGH = 2 · δmax (9)

Shape matching is finally done by using the Euclidean dis-tance as a similarity metric. Note that for simple convexobjects the PGH can be constructed solely by the points ofthe object’s convex hull (no CPD required).

The proposed method for shape estimation can be in-stantly computed on mobile devices, because the numberof input points needed for PGH computation can be scaledon demand. For simple convex objects, information alreadyavailable about the convex hull can be used to avoid sam-pling critical boundary points. For concave objects, the useof the CPD only introduces negligible computation over-head. Shape matching using the proposed similarity metriccan be further optimized by using suitable data structures.

20

Figure 3. Line j is described in the pairwise geometric histogramwith reference to line i. The collectivity of all descriptions formsthe final shape descriptor of the object.

ObjectsCoins Med. Pills Screws/Nuts Misc

Number 8 13 13 6Convex true true false false

Imag

eSi

ze 320x240 1 13 0 0422x318 8 13 6 5506x380 8 13 10 5576x432 8 13 11 6640x480 8 13 13 6

Table 1. The five different object categories used in our experi-mental evaluation. The lower part of the table lists the numberof successfully segmented objects for varying image resolutions.Note that we varied the image resolution linearly in the number ofimage pixels.

This makes the algorithm ideally suited to mobile devicesdue to its low computational complexity and the moderatememory requirements.

5. ExperimentsIn the following, we evaluate the individual parts of our

approach in detail on different object categories. Finally wedemonstrate the application of our approach on the task ofmedical pill recognition on a modern phone. All parts ofour algorithm were implemented in C/C++. As our mobileplatform we choose an Asus M530w smartphone4 runningWindows Mobile 6. The device features a 2 mega-pixel aut-ofocus camera, a 416MHz fixed-point CPU and 64 MB ofRAM, as well as 256 MB Flash memory. At the time ofwriting, this phones hardware is already a little outdated, soour approach is expected to run at a higher performance onfuture smartphones.

5.1. Object Segmentation

First, we evaluate our segmentation method for varyingimage resolutions. Table 1 lists the details about the dif-ferent sets of object categories used. The categories aresorted by increasing complexity, from relatively simple ob-

4http://www.asus.com

Figure 4. Sample results from the object segmentation step. Thetwo images in the last row show erroneous and inaccurate segmen-tation results.

jects such as coins to arbitrary objects like keys. We gener-ated ground-truth data by segmenting the object in the testimages manually.

Figure 4 shows some sample segmentation results. Quiteaccurate segmentation results are generated in most cases,but as can be seen from the last two images, in some casesreflections or bad lighting effects can cause inaccurate or er-roneous results. The lower part of Table 1 gives an overviewabout the number of successfully segmented objects in theindividual categories for varying image resolutions. For aconsiderable number of objects, the algorithm is not ableto generate a reasonable segmentation result at lower im-age resolutions. This could be improved by using differentmarker-based targets that match specifically to the currentresolution (modified square length sqw).

Figure 5 depicts the segmentation accuracy for varyingimage resolutions and the runtime performance on the mo-bile phone. As expected, the segmentation accuracy in-creases with increasing image resolution, but saturates at

21

Figure 5. Segmentation accuracy and runtime for varying imageresolutions on the mobile device. Accuracy is measured as relativedifference between the segmentation result and the ground truth,compared to the total object size. The runtime performance isaveraged over all segmentation runs for all object categories in thetest set.

Figure 6. Accuracy and runtime performance for varying imageresolutions on the mobile device. In the upper graph, the numberof objects is also plotted, which is delivered to the feature extrac-tion algorithm from the segmentation step. Accuracy is measuredin terms of difference in object width/height measured, comparedto the ground truth object size. For shape and color, accuracy ismeasured in terms of percentage of correct votes for shape andcolor categories, compared to the overall number of objects.

a medium resolution level. Also the time spent in the seg-mentation step on the mobile device increases accordingly,so the fictional segmentation frame rate drops from about 6fps for the lowest resolution to about 1.4 fps for the highesttested resolution.

5.2. Feature Extraction

In the second experiment, we investigate the feature ex-traction algorithms on our mobile device for varying imageresolutions. Figure 6 shows the accuracy and the runtime

special

special special special special

oval oval oblong oblong special

special special special special special

circular

specialspecial

Figure 7. Four different shape classes with differently coloredboundaries and 13 color classes used in our experiments.

performance plotted against varying image resolutions. Ad-ditionally, in the upper plot the number of objects deliveredby the previous segmentation step is given for each individ-ual resolution. As can be seen, the accuracy for the indi-vidual features stays approximately the same for all testedresolutions. As an important aspect, for low resolutionsthe number of objects passing the previous segmentationstage is significantly lower than for images with high reso-lution. This result directly implies that the limiting factor inour approach is basically the segmentation step. Segmen-tation does not work successfully on most objects for lowimage resolutions, although the subsequent feature extrac-tion would lead to accurate results. Thus, an improvementto the previous segmentation algorithm is advised. In thelower plot, it’s easy to see that the runtime for the size andcolor estimation algorithm increases linearly with increas-ing image resolution, while the shape estimation runtimestays constant. The reason for this is that we chose to lin-early increase the number of sample pixels used for the es-timation of the former features. In contrast, the number ofinvestigated line segments for the shape feature stays almostconstant.

5.3. Application to Medical Pill Recognition

Finally we propose the use of our segmentation and fea-ture extraction approach in the task of medical pill recog-nition on mobile phones. Often, the process of drug iden-tification must be carried out in a mobile scenario, wherefor example a person is found unconscious. In the absenceof other tools, only a visual inspection is possible. In thisapplication scenario the proposed segmentation and featureestimation methods are used to estimate the correspondingproperties of a pharmaceutical pill and to query a database.

We have generated a reference set of medical pills forthis experimental evaluation, where we have restricted our-selves to use four different shape classes and 13 colorclasses, as illustrated in Figure 7. Single pills were allowed

22

Shapes circular oval oblong specialInstances 41 26 33 8Colors single multiInstances 98 10Sizes [mm] min. length min width max length max width

5.68 5.68 18.07 18.07

Table 2. Reference testset of medical pills for our experimentalevaluation. The testset exhibits reasonable variation in shape andpill size.

Color black white blue beige brownInstances 1 29 6 11 4Color gray green ocher pink/violet orangeInstances 1 5 6 3 6Color rose/peach red yellowInstances 11 5 10

Table 3. Distribution of single-colored pills over the 13 colorclasses used.

to be of one or two dominant colors. Note that collectinga large amount of different medical pills is a tedious task,especially when the aim is to collect samples from a largernumber of different color and shape classes. Because pillswith rarely used shapes are especially hard to get, we col-lected them in a large class for special shapes. In Table 2details about the testset are given, Table 3 describes the dis-tribution over the 13 color classes for single-colored pills.

For the experiment we fixed the image resolution to640x480 pixels. All samples were captured under two dif-ferent lighting conditions, fluorescent light and daylight.From initial evaluation of this data, optimal parameters forshape estimation (D = 12, A = 12) and color estimation(sLT = 36) were determined. In this step, the best recogni-tion rate for shape is 89% and that for color is 84%. In sizeestimation an average deviation of 0.43 mm and a maximumdeviation of 1.37 mm is achieved.

We retrieved pill information from the reference set de-scribed in Tables 2 and 3 by querying, either using singlefeatures or simple AND-combinations of several features.Querying for object size, results in a range of ±0.7 mmfrom the measured value were allowed. From the final listof possible candidates, we counted a reported pill as suc-cessfully recognized if it is among the first N candidates.

Retrieval performance for single features gives informa-tion about the descriptive power of a feature (see Figure 8).Size is determined to be most powerful feature, followed bycolor and shape. Size and color give better results with lesscandidates than the feature shape. Performance with thefeature size reaches a recognition rate of 84.26 % (N = 7candidates). This rate can be significantly improved, if theresults are initially ordered by the minimum sum of devia-tions in length and width (indicated as size-sorted result inTable 4).

Figure 8. Pill retrieval on the reference set.

Size Size (sorted) Shape Color RRmax #X 84.26 % 7

X 100.00 % 8X 48.15 % 10

X 65.74 % 10X X 76.39 % 7X X 66.70 % 3X X X 59.72 % 3

X X 90.28 % 6X X 81.48 % 3X X X 72.69 % 3

Table 4. Pill retrieval on the reference set with single features andfeature combinations (Nmax = 10). RRmax denotes the highestrecognition rate achieved.

Peak performance drops when using feature combina-tions. The reason can be seen in mutual reactions of errorsfrom the individual feature estimators and our use of a strictAND combination. Although this way of combining fea-tures seems to be a disadvantageous choice at first, this iswhat online available databases are most likely to offer (seefor example the Identa5 online database). Without doubt,using a probabilistic combination of features accounting forestimation uncertainty of individual features, the recogni-tion performance could be increased significantly. Whenusing size-sorted results, considerable gains in recogni-tion rate are possible. With subsequent querying for colormatches, a recognition rate of 90.28 % (N = 6 candidates)is achieved. The peak performance for several feature com-binations is listed in Table 4.

6. ConclusionIn this paper we presented our approach to instant object

segmentation and recognition of simple objects on modern

5http://www.gelbe-liste.de/pharmindex/identa

23

mobile phones. After describing the basic principles of ouralgorithms in detail, the applicability and usefulness of ourapproach was demonstrated on the real-world application ofmedical pill recognition. The results of our extensive evalu-ation prove the performance and accuracy of our approach,and its suitability for several tasks. This was demonstratedon the application of pill recognition on mobile phones. Theslim design of our algorithms was shown to be ideally suitedto modern mobile phone architectures despite existing hard-ware limitations.

Although a lot of effort is put into generating an accurateand fast object segmentation, under several circumstancesthe results were still not satisfying. Difficult lighting condi-tions can generate artifacts or highlights on objects, whichcause segmentation errors or, in the worst case, algorithmfailure. Thus, an important part of future work must bededicated to further improve the quality of the segmenta-tion result. Another interesting topic of upcoming researchwill be the estimation of additional object features. Forknown categories, additional properties can probably be es-timated, e.g. object transparency or surface glossiness, tofurther draw conclusions on the object material from. Need-less to mention that further optimization of our algorithmsto reach real-time performance on modern mobile phones isleft as an open issue.

AcknowledgementsThis research work was partially funded by the ChristianDoppler Research Association.

References[1] C. Arth, D. Wagner, M. Klopschitz, A. Irschara, and

D. Schmalstieg. Wide Area Localization on Mobile Phones.In ISMAR, pages 73–82, Oct. 2009. 2

[2] A. Ashbrook, N. Thacker, P. Rockett, and C. Brown. RobustRecognition of Scaled Shapes using Pairwise Geometric His-tograms. In BMVC, pages 503–512, 1995. 4

[3] F. Chang, C.-J. Chen, and C.-J. Lu. A Linear-TimeComponent-Labeling Algorithm using Contour TracingTechnique. CVIU, 93(2):206–220, 2004. 2

[4] A. Evans, N. Thacker, and J. Mayhew. Pairwise Represe-nations of Shape. In ICPR, pages 133–136. Los Alamitos,Calif. u.a. IEEE Computer Soc. Pr., 1992. 4

[5] P. Fockler, T. Zeidler, B. Brombach, E. Bruns, and O. Bim-ber. PhoneGuide: Museum Guidance supported by On-device Object Recognition on Mobile Phones. In Proc. Int.Conf. on Mobile and Ubiquitous Multimedia (MUM), pages3–10, 2005. 2

[6] R. Hartley and A. Zisserman. Multiple View Geometry inComputer Vision. Cambridge University Press, 2nd edition,2008. 2

[7] V. Ivanchenko, J. Coughlan, and H. Shen. Staying in theCrosswalk: A System for Guiding Visually Impaired Pedes-

trians at Traffic Intersections. Assoc. for the Advancement ofAssistive Technology in Europe (AAATE), Sep. 2009. 2

[8] G. Klein and D. Murray. Parallel Tracking and Mapping ona Camera Phone. In ISMAR, pages 83–86, Oct. 2009. 2

[9] M. Koga, R. Mine, T. Kameyama, T. Takahashi, M. Ya-mazaki, and T. Yamaguchi. Camera-based Kanji OCR forMobile-phones: Practical Issues. In ICDAR, volume 2, pages635 – 639, Aug. 2005. 2

[10] C. Ng, M. Savvides, and P. Khosla. Real-time Face Verifi-cation System on a Cell-phone using Advanced CorrelationFilters. In IEEE Workshop on Automatic Identification Ad-vanced Technologies, pages 57 – 62, Oct. 2005. 2

[11] K. R. Park, H.-A. Park, B. J. Kang, E. C. Lee, and D. S.Jeong. A Study on Iris Localization and Recognition on Mo-bile Phones. EURASIP Journal on Advances in Signal Pro-cessing, 2008. 2

[12] M. Rahman, N. Kehtarnavaz, and J. Ren. A Hybrid FaceDetection Approach for Real-time Deployment on MobileDevices. In ICIP, pages 3233 –3236, Nov. 2009. 2

[13] M. Rahman, J. Ren, and N. Kehtarnavaz. Real-time Imple-mentation of Robust Face Detection on Mobile Platforms.IEEE Int. Conf. on Acoustics, Speech, and Signal Processing(ICASSP), 0:1353–1356, 2009. 1

[14] H. Rowley, S. Baluja, and T. Kanade. Rotation InvariantNeural Network-based Face Detection. In CVPR, pages 38–44, Jun. 1998. 1

[15] F. Shafait, D. Keysers, and T. Breuel. Efficient Implemen-tation of Local Adaptive Thresholding Techniques Using In-tegral Images. In Proc. Int. Conf. on Document Recognitionand Retrieval (DRR), volume 6815. SPIE, 1 2008. 2

[16] S. Susstrunk, J. Holm, and G. D. Finlayson. ChromaticAdaptation Performance of Different RGB Sensors. Proc.of SPIE, the Int. Society for Optical Engineering, 4300:172–183, 2001. 4

[17] G. Takacs, V. Chandrasekhar, N. Gelfand, Y. Xiong, W. C.Chen, T. Bismpigiannis, R. Grzeszczuk, K. Pulli, andB. Girod. Outdoors Augmented Reality on Mobile Phone us-ing Loxel-based Visual Feature Organization. In Proc. ACMInt. Conf. on Multimedia Information Retrieval (MIR), pages427–434, 2008. 2

[18] T. Theocharides, G. Link, N. Vijaykrishnan, M. Irwin, andW. Wolf. Embedded Hardware Face Detection. In Proc. Int.Conference on VLSI Design, pages 133 – 138, 2004. 1

[19] M. Vik. Industrial Colour Difference Evaluation: LCAMTextile Data. In Proceedings of the AIC Color and Paints (In-terim Meeting of the International Color Association), pages138–142, 2004. 4

[20] D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, andD. Schmalstieg. Pose Tracking from Natural Features onMobile Phones. In ISMAR, pages 125–134, Sep. 2008. 2

[21] S. Wijewickrema and A. P. Paplinski. Principal Compo-nent Analysis for the Approximation of a Fruit as an Ellipse,2004. 3

[22] J. Zhang, X. Chen, J. Yang, and A. Waibel. A PDA-basedSign Translator. In Proc. IEEE Int. Conf. on MultimodalInterfaces (ICMI), pages 217 –222, 2002. 2

[23] P. Zhu and P. M. Chirlian. On Critical Point Detection ofDigital Shapes. PAMI, 17(8):737–748, 1995. 4

24