Mean Shift Based Clustering in High Dimensions_ a Texture Classification Example

Mean Shift Based Clustering in High Dimensions:A Texture Classification Example

Bogdan Georgescu�� Ilan Shimshoni�� Peter Meer��

Computer Science �� Industrial Engineering and Management��

Electrical and Computer Engineering�� Technion - Israel Institute of TechnologyRutgers University, Piscataway, NJ 08854, USA Haifa 32000, ISRAELgeorgesc, [email protected] [email protected]

Abstract

Feature space analysis is the main module in many com-puter vision tasks. The most popular technique, k-meansclustering, however, has two inherent limitations: the clus-ters are constrained to be spherically symmetric and theirnumber has to be known a priori. In nonparametric clus-tering methods, like the one based on mean shift, theselimitations are eliminated but the amount of computationbecomes prohibitively large as the dimension of the spaceincreases. We exploit a recently proposed approximationtechnique, locality-sensitive hashing (LSH), to reduce thecomputational complexity of adaptive mean shift. In ourimplementation of LSH the optimal parameters of the datastructure are determined by a pilot learning procedure, andthe partitions are data driven. As an application, the per-formance of mode and k-means based textons are comparedin a texture classification study.

1. IntroductionRepresentation of visual information through feature spaceanalysis received renewed interest in recent years, moti-vated by content based image retrieval applications. Theincrease in the available computational power allows todaythe handling of feature spaces which are high dimensionaland contain millions of data points.

The structure of high dimensional spaces, however, de-fies our three dimensional geometric intuition. Such spacesare extremely sparse with the data points far away from eachother [17, Sec.4.5.1]. Thus, to infer about the local struc-ture of the space only a small number of data points maybe available, which can yield erroneous results. The phe-nomenon is known in the statistical literature as the curse ofdimensionality, and its effect increases exponentially withthe dimension. The curse of dimensionality can be avoidedonly by imposing a fully parametric model over the data [6,p.203], an approach which is not feasible for a high dimen-sional feature space with a complex structure.

The goal of feature space analysis is to reduce the datato a few significant features through a procedure known un-

der many different names, clustering, unsupervised learn-ing, or vector quantization. Most often different variantsof k-means clustering are employed, in which the featurespace is represented as a mixture of normal distributions [6,Sec.10.4.3]. The number of mixture components � is usu-ally set by the user.

The popularity of the k-means algorithm is due to itslow computational complexity of ��, where � is thenumber of data points, � the dimension of the space, and �the number of iterations which is always small relative to �.However, since it imposes a rigid delineation over the fea-ture space and requires a reasonable guess for the numberof clusters present, the k-means clustering can return erro-neous results when the embedded assumptions are not satis-fied. Moreover, the k-means algorithm is not robust, pointswhich do not belong to any of the � clusters can move theestimated means away from the densest regions.

A robust clustering technique which does not requireprior knowledge of the number of clusters, and does notconstrain the shape of the clusters, is the mean shift basedclustering. This is also an iterative technique, but instead ofthe means, it estimates the modes of the multivariate distri-bution underlying the feature space. The number of clustersis obtained automatically by finding the centers of the dens-est regions in the space (the modes). See [1] for details. Un-der its original implementation the mean shift based cluster-ing cannot be used in high dimensional spaces. Already for� � �, in a video sequence segmentation application, a fine-to-coarse hierarchical approach had to be introduced [5].

The most expensive operation of the mean shift methodis finding the closest neighbors of a point in the space. Theproblem is known in computational geometry as multidi-mensional range searching [4, Chap.5]. The goal of therange searching algorithms is to represent the data in astructure in which proximity relations can be determinedin less than �� time. One of the most popular struc-tures, the kD-tree, is built in �� operations, wherethe proportionality constant increases with the dimensionof the space. A query selects the points within a rectangu-lar region delimited by an interval on each coordinate axis,

Proceedings of the Ninth IEEE International Conference on Computer Vision (ICCV’03) 0-7695-1950-4/03 $ 17.00 © 2003 IEEE

and the query time for kD-trees has complexity bounded by

��

��

�, where � is the number of points found.

Thus, for high dimensions the complexity of a query is prac-tically linear, yielding the computational curse of dimen-sionality. Recently, several probabilistic algorithms havebeen proposed for approximate nearest neighbor search.The algorithms yield sublinear complexity with a speedupwhich depends on the desired accuracy [7, 10, 11].

In this paper we have adapted the algorithm in [7] formean shift based clustering in high dimensions. Work-ing with data in high dimensions also required that we ex-tend the adaptive mean shift procedure introduced in [2].All computer vision applications of mean shift until now,such as image segmentation, object recognition and track-ing, were in relatively low-dimensional spaces. Our imple-mentation opens the door to use mean shift in tasks basedon high-dimensional features.

In Section 2 we present a short review of the adaptivemean-shift technique. Locality-sensitive hashing, the tech-nique for approximate nearest neighbor search is describedin Section 3, where we have also introduced refinements tohandle data with complex structure. In Section 4 the per-formance of adaptive mean shift (AMS) in high dimensionsis investigated, and in Section 5 AMS is used for textureclassification based on textons.

2. Adaptive Mean ShiftHere we only review some of the results described in [2]which should be consulted for the details.

Assume that each data point �� , � � �� isassociated with a bandwidth value �� . The samplepoint estimator

��

�

��

�

��

��

��

(1)

based on a spherically symmetric kernel with boundedsupport satisfying

�� (2)

is an adaptive nonparametric estimator of the density at lo-cation � in the feature space. The function ��, � � � � �,is called the profile of the kernel, and the normalization con-stant �� assures that �� integrates to one. The function �� can always be defined when the derivativeof the kernel profile �� exists. Using �� as the profile,the kernel �� is defined as ��

��.By taking the gradient of (1) the following property can

be proven

��

��(3)

where � is a positive constant and

��

��

�

��

��

��

��

��

��

��

��

��

�� (4)

is called the mean shift vector. The expression (3) showsthat at location � the weighted mean of the data points se-lected with kernel � is proportional to the normalized den-sity gradient estimate obtained with kernel . The meanshift vector thus points toward the direction of maximumincrease in the density. The implication of the mean shiftproperty is that the iterative procedure

��

��

��

��

��

��

��

��

��

��

��

�� (5)

is a hill climbing technique to the nearest stationary pointof the density, i.e., a point in which the density gradientvanishes. The initial position of the kernel, the starting pointof the procedure �� can be chosen as one of the data points��. Most often the points of convergence of the iterativeprocedure are the modes (local maxima) of the density.

There are numerous methods described in the statisti-cal literature to define ��, the bandwidth values associatedwith the data points, most of which use a pilot density es-timate [17, Sec.5.3.1]. The simplest way to obtain the pilotdensity estimate is by nearest neighbors [6, Sec.4.5]. Let�� be the -nearest neighbor of the point ��. Then, wetake

�� (6)

where �� norm is used since it is the most suitable for thedata structure to be introduced in the next section. Thechoice of the norm does not have a major effect on the per-formance. The number of neighbors should be chosenlarge enough to assure that there is an increase in densitywithin the support of most kernels having bandwidths ��.While the value of should increase with � the dimensionof the feature space, the dependence is not critical for theperformance of the mean shift procedure, as will be seenin Section 4. When all �� , i.e., a single global band-width value is used, the adaptive mean shift (AMS) pro-cedure becomes the fixed bandwidth mean shift (MS) dis-cussed in [1].

A robust nonparametric clustering of the data is achievedby applying the mean shift procedure to a representativesubset of the data points. After convergence, the detectedmodes are the cluster centers, and the shape of the clustersis determined by the basins of attraction. See [1] for details.


3. Locality-Sensitive HashingThe bottleneck of mean shift in high dimensions is the needfor a fast algorithm to perform neighborhood queries whencomputing (5). The problem has been addressed before inthe vision community by sorting the data according to eachof the � coordinates [13], but a significant speedup wasachieved only when the data is close to a low-dimensionalmanifold.

Recently new algorithms using tools from probabilisticapproximation theory were suggested for performing ap-proximate nearest neighbor search in high dimensions forgeneral datasets [10, 11] and for clustering data [9, 14]. Weuse the approximate nearest neighbor algorithm based onlocality-sensitive hashing (LSH) [7] and adapted it to han-dle the complex data met in computer vision applications.In a task of estimating the pose of articulated objects, de-scribed in these proceedings [16], the LSH technique wasextended to accommodate distances in the parameter space.

3.1. High Dimensional Neighborhood QueriesGiven � points in �� the mean shift iterations (5) requirea neighborhood query around the current location �� . Thenaive method is to scan the whole dataset and test whetherthe kernel of the point �� covers �� . Thus, for each meancomputation the complexity is ��. Assuming that forevery point in the dataset this operation is performed �times (a value which depends on the ��’s and the distribu-tion of the data), the complexity of the mean shift algorithmis ��.

To improve the efficiency of the neighborhood queriesthe following data structure is constructed. The data is tes-sellated � times with random partitions, each defined by �inequalities (Figure 1). In each partition � pairs of randomnumbers, �� and �� are used. First ��, an integer between 1and � is chosen, followed by ��, a value within the range ofthe data along the ��-th coordinate.

The pair �� partitions the data according to the in-equality

�� (7)

where �� is the selected coordinate for the data point��. Thus, for each point �� each partition yields a �-dimensional boolean vector (inequality true/false). Pointswhich have the same vector lie in the same cell of the parti-tion. Using a hash function, all the points belonging to thesame cell are placed in the same bucket of a hash table. Aswe have � such partitions, each point belongs simultane-ously to � cells (hash table buckets).

To find the neighborhood of radius � around a querypoint �, � boolean vectors are computed using (7). Thesevectors index � cells ��, � �� in the hash table.

The points in their union ��

��

�� are the ones returned

by the query (Figure 1). Note that any � in the intersection

��

��

�� will return the same result. Thus �� deter-

mines the resolution of the data structure, whereas �� de-termines the set of the points returned by the query. The de-scribed technique is called locality-sensitive hashing (LSH)and was introduced in [10].

Points close in�� have a higher probability for collisionin the hash table. Since �� lies close to the center of ��,the query will return most of the nearest neighbors of �. Theexample in Figure 1 illustrates the approximate nature of thequery. Parts of an �� neighborhood centered on � are notcovered by�� which has a different shape. The approxima-tion errors can be reduced by building data structures withlarger ��’s, however, this will increase the running time ofa query.

L

Figure 1: The locality-sensitive hashing data structure. Forthe query point � the overlap of � cells yields the region��which approximates the desired neighborhood.

3.2. Optimal Selection of � and �The values for � and � determine the expected volumes of�� and ��. The average number of inequalities used foreach coordinate is ��, partitioning the data into �� regions. The average number of points ��

in a cell �� and�� in their union �� is

��

��

� (8)

Note that in estimating �� we disregard that the pointsbelong to several ��’s.

Qualitatively, the larger the value for �, the number ofcuts in a partition, the smaller the average volume of thecells ��. Similarly, as the number of partitions � increases,the volume of �� decreases and of �� increases. For a


given �, only values of � below a certain bound are ofinterest. Indeed, once � exceeds this bound all the neigh-borhood of radius � around � has been already covered by��. Thus, larger values of � will only increase the querytime with no improvement in the quality of the results.

The optimal values of � and � can be derived from thedata. A subset of data points �� , isselected by random sampling. For each of these data points,the �� distance �� (6) to its k-nearest neighbor is deter-mined accurately by the traditional linear algorithm.

In the approximate nearest neighbor algorithm based onLSH, for any pair of � and �, we define for each of the� points �� , the distance to the k-nearest neighbor re-turned by the query. When the query does not return thecorrect k-nearest neighbors �� . The total runningtime of the � queries is ��. The optimal �� isthen chosen such that

��

�� (9)

subject to:�

�

��

��

��

��

where is the LSH approximation threshold set by the user.The optimization is performed as a numerical search pro-

cedure. For a given � we compute, as a function of �, theapproximation error of the � queries. This is shown in Fig-ure 2a for a thirteen-dimensional real data set. By thresh-olding the family of graphs at � ��, the function ��is obtained (Figure 2b). The running time can now be ex-pressed as ��, i.e., a one-dimensional function in�, the number of employed cuts (Figure 2c). Its minimumis �� which together with �� are the optimal pa-rameters of the LSH data structure.

0 20 40 600

0.5

1

1.5

L

erro

r

K=30

K=20

K=10

0 10 20 30 400

100

200

300

400

K

L

0 10 20 30 400

5

10

15

20

K

t[K,L

(K)]

(a) (b) (c)

Figure 2: Determining the optimal � and �. (a) Depen-dence of the approximation error on � for � � �� .The curves are thresholded at � ��, dashed line. (b)Dependence of � on � for � ��. (c) The running time ��. The minimum is marked ‘�’.

The family of error curves can be efficiently generated.The number of partitions � is bounded by the availablecomputer memory. Let �� be that bound. Similarly,we can set a maximum on the number of cuts, ��.Next, the LSH data structure is built with �� .

Then, the approximation error is computed incrementallyfor � � �� by adding one partition at a time.This yields �� which is subsequently used as ��

for �� , etc.

3.3. Data Driven PartitionsThe strategy of generating the� random tessellations has animportant influence on the performance of locality-sensitivehashing. In [7] the coordinates � have equal chance to beselected and the values are uniformly distributed overthe range of the corresponding coordinate. This partitioningstrategy works well only when the density of the data is ap-proximately uniform in the entire space. However, featurespaces associated with vision applications are often multi-modal. In [10, 11] the problem of nonuniformly distributeddata was dealt with by building several data structures withdifferent values of � and � to accommodate the differentlocal densities. The query is performed first under the as-sumption of a high density, and when it fails it is repeatedfor lower densities. The process terminates when the near-est neighbors are found.

Our approach is to sample according to the marginal dis-tributions along each coordinate. We use a few points ��chosen at random from the data set. For each point one ofits coordinates is selected at random to define a cut. Usingmore than one coordinate from a point would imply sam-pling from partial joint densities, but does not seem to bemore advantageous. Our adaptive, data driven strategy as-sures that in denser regions more cuts will be made yieldingsmaller cells, while in sparser regions there will be less cuts.On average all cells will contain a similar number of points.

The two-dimensional data in Figure 3b and 3b comprisedof four clusters and uniformly distributed background, isused to demonstrate the two sampling strategies. In bothcases the same number of cuts were used but the data drivenmethod places most of the cuts over the clusters (Figure 3b).For a quantitative performance assessment a data set often normal distributions with arbitrary shapes (5000 pointseach) were defined in 50 dimensions. When the data drivenstrategy is used, the distribution of the number of pointsin a cell is much more compact and their average value ismuch lower (Figure 3c). As a consequence, the data drivenstrategy yields more efficient k-nearest neighbor queries forcomplex data sets.

4. Mean Shift in High DimensionsGiven �� , the current location in the iterations, an LSHbased query retrieves the approximate set of neighborsneeded to compute the next location (5). The resolutionof the data analysis is controlled by the user. In the fixedbandwidth MS method the user provides the bandwidth pa-rameter �. In the AMS method, the user sets the number


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

X1

X2

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

X1

X2

(a) (b)

0 5000 10000 15000 200000

1000

2000

3000

4000

5000

6000

Number of Neighbors

Num

ber

of C

ells

Uniform DistributionData−Driven Distribution

(c)Figure 3: Uniform vs. data driven partitions. Typical re-sult for two-dimensional data obtained with (a) uniform; (b)data driven strategy. (c) Distribution of points-per-cell for a50-dimensional data set.

of neighbors � used in the pilot density procedure. The pa-rameters� and� of the LSH data structure are selected em-ploying the technique discussed in Section 3.2. The band-widths �� associated with the data points are obtained byperforming � neighborhood queries. Once the bandwidthsare set, the adaptive mean shift procedure runs at approx-imately the same cost as the fixed bandwidth mean shift.Thus, the difference between MS and AMS is only one ad-ditional query per point.

An ad-hoc procedure provides further speedup. Sincethe resolution of the data structure is ��, with high prob-ability one can assume that all the points within �� willconverge to the same mode. Thus, once any point from a�� is associated with a mode, the subsequent queries to ��automatically return this mode and the mean shift iterationsstop. The modes are stored in a separate hash table whosekeys are the � boolean vectors associated with ��.

4.1. Adaptive vs. Fixed Bandwidth Mean ShiftTo illustrate the advantage of adaptive mean shift, a dataset containing 125,000 points in a 50-dimensional cube wasgenerated. From these �� ,�� points belonged to tenspherical normal distributions (clusters) whose means werepositioned on a line through the origin. The standard devi-ation increases as the mean becomes more distant from theorigin. For an adjacent pair of clusters, the ratio of the sumof standard deviations to the distance between the meanswas kept constant. The remaining 100,000 points were uni-formly distributed in the 50-dimensional cube. Plotting the

distances of the data points from the origin yields a graphvery similar to the one in Figure 4a. Note that the pointsfarther from the origin have a larger spread.

The performance of the fixed bandwidth (MS) and theadaptive mean shift (AMS) procedures is compared for var-ious parameter values in Figure 4. The experiments wereperformed for 500 points chosen at random from each clus-ter, a total of 5000 points. The location associated with eachselected point after the mean shift procedure, is the em-ployed performance measure. Ideally this location shouldbe near the center of the cluster to which the point belongs.

In the MS strategy, when the bandwidth � is small, dueto the sparseness of the high-dimensional space very fewpoints have neighbors within distance �. The mean shiftprocedure does not start and the allocation of the points is tothemselves (Figure 4a). On the other hand as � increases thewindows become too large for some of the local structuresand points may converge incorrectly to the center (mode) ofan adjacent cluster (Figures 4b to 4d).

The pilot density estimation in the AMS strategy auto-matically adapts the bandwidth to the local structure. Theparameter �, the number of neighbors used for the pilot es-timation does not have a strong influence. The data is pro-cessed correctly for � � �� to 500, except for a few points(Figures 4e to 4g), and even for � � �� only some of thepoints in the cluster with the largest spread converge to theadjacent mode (Figure 4h). The superiority of the adaptivemean shift in high dimensions is clearly visible. Due to thesparseness of the 50-dimensional space, the 100,000 pointsin the background did not interfere with the mean shift pro-cesses under either strategy, proving its robustness.

The use of the LSH data structure in the mean shift pro-cedure assures a significant speedup. We have derived fourdifferent features spaces from a texture image with the filterbanks discussed in the next section. The spaces had dimen-sion � � �� and 48, and contained � � �� points.An AMS procedure was run both with linear and approxi-mate queries for 1638 points. The number of neighbors inthe pilot density estimation was � � ��. The approxima-tion error of the LSH was � � ��. The running times (inseconds) in Table 1 show the achieved speedups.

Table 1: Running Times of AMS Implementations� Traditional LSH Speedup

4 1,507 80 18.88 1,888 206 9.2

13 2,546 110 23.148 5,877 276 21.3

The speedup will increase with the number of data points�, and will decrease with the number of neighbors �. There-fore in the mean shift procedure the speedup is not as highas in applications in which only a small number of neigh-bors are required.


0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

h=100

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

h=1400

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

h=2700

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

h=4000

Points

Dis

tanc

e

(a) (b) (c) (d)

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

k=100

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

k=300

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

k=500

Points

Dis

tanc

e

0 1000 2000 3000 4000 50001

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3x 10

5

k=700

Points

Dis

tanc

e

(e) (f) (g) (h)

Figure 4: Distance from the origin of 5000 points from ten 50-dimensional clusters after fixed bandwidth mean shift (MS):(a) to (d); and adaptive mean shift (AMS) : (e) to (h). The parameters: MS – bandwidth �; AMS – number of neighbors �.

5. Texture ClassificationEfficient methods exist for texture classification under vary-ing illumination and viewing direction [3],[12], [15], [18].In the state-of-the-art approaches a texture is characterizedthrough textons, which are cluster centers in a feature spacederived from the input. Following [12] this feature space isbuilt from the output of a filter bank applied at every pixel.However, as was shown recently [19], neighborhood infor-mation in the spatial domain may also suffice.

The approaches differ in the employed filter bank.– LM: A combination of 48 anisotropic and isotropic fil-

ters were used by Leung and Malik [12] and Cula andDana [3]. The filters are Gaussian masks, their firstderivative and Laplacian, defined at three scales. Be-cause of the oriented filters, the representation is sen-sitive to texture rotations. The feature space is 48 di-mensional.

– S: A set of 13 circular symmetric filters was used bySchmid [15] to obtain a rotationally invariant featureset. The feature space is 13 dimensional.

– M4, M8: Both representations were proposed byVarma and Zissermann [18]. The first one (M4) isbased on 2 rotationally symmetric and 12 oriented fil-ters. The second set is an extension of the first one at3 different scales. The feature vector is computed byretaining only the maximum response for the orientedfilters (2 out of 12 for M4 and 6 out of 36 for M8), thusreducing the dependence on the global texture orienta-tion. The feature space is 4 respectively 8 dimensional.

To find the textons, usually the standard k-means clus-tering algorithm is used, which as was discussed in Sec-

tion 1 has several limitations. The shape of the clusters isrestricted to be spherical and their number has to be set priorto the processing.

The most significant textons are aggregated into the tex-ton library. This serves as a dictionary of representativelocal structural features and must be general enough to char-acterize a large variety of texture classes. A texture is thenmodeled through its texton histogram. The histogram iscomputed by defining at every pixel a feature vector, re-placing it with the closest texton from the library (vectorquantization) and accumulating the results over the entireimage.

Let two textures � and � be characterized by the his-tograms �� and �� built from � textons. As in [12] the�� distance between these two texton distributions

��

��

��

��

�� (10)

is used to measure similarity, although note the absence ofthe factor �� to take into account that the comparison is be-tween two histograms derived from data. In a texture clas-sification task the training image with the smallest distancefrom the test image determines the class of the latter.

In our experiments we substituted the k-means basedclustering module with the adaptive mean shift (AMS)based robust nonparametric clustering. Thus, the textonsinstead of being mean based are now mode based, and thenumber of the significant ones is determined automatically.

The complete Brodatz database containing 112 textureswith varying degrees of complexity was used in the experi-ments. Classification of the Brodatz database is challenging


because it contains many nonhomogeneous textures. The�� images were divided into four �� subim-ages with half of the subimages being used for training (224models) and the other half for testing (224 queries). Thenormalizations recommended in [18] (both in the image andfilter domains) were also performed.

The number of significant textons detected with the AMSprocedure depends on the texture. We have limited the num-ber of mode textons extracted from a texture class to five.The same number was used for the mean textons. Thus, byadding the textons to the library, a texton histogram has atmost � � �� bins.

Table 2: Classification Results for the Brodatz DatabaseFilter M4 M8 S LM

RND 84.82% 88.39% 89.73% 92.41%k-means 85.71% 94.64% 93.30% 97.32%AMS 85.27% 93.75% 93.30% 98.66%

The classification results using the different filter banksare presented in Table 2. The best result was obtained withthe LM mode textons, an additional three correct classifica-tions out of the six errors with the mean textons. However,there is no clear advantage in using the mode textons withthe other filter banks.

The classification performance is close to its upperbound defined by the texture inhomogeneity, due to whichthe test and training images of a class can be very different.This observation is supported by the performance degrada-tion obtained when the database images were divided intosixteen �� subimages and the same half/half partitionyielded 896 models and 896 queries. The recognition ratedecreased for all the filter banks. The best result of 94%,was again obtained with the LM filters for both the meanand mode textons. In [8], with the same setup but employ-ing a different texture representation, and using only 109textures from the Brodatz database the recognition rate was��.

A texture class is characterized by the histogram of thetextons, an approximation of the feature space distribution.The histogram is constructed from a Voronoi diagram with� cells. The vertices of the diagram are the textons, andeach histogram bin contains the number of feature points ina cell. Thus, variations in textons translate in approximat-ing the distribution by a different diagram, but it appearsto have a weak influence on the classification performance.When by uniform sampling five random vectors were cho-sen as textons, the classification performance (RND) de-creased only between 1% to 6%.

The k-means clustering imposes rigidly a given numberof identical spherical clusters over the feature space. Thus,it is expected that when this structure is not adequate, themode based textons will provide a more meaningful decom-position of the texture image. This is proven in the follow-

5 10 15 200

0.2

0.4

0.6

0.8

1

Texton No.

Per

cent

age

texture classified pixels

mode mean

Figure 5: Mode (�) vs. mean (Æ) based textons. The lo-cal structure is better captured by the mode textons. D001texture, LM filter bank.

ing two examples.

In Figure 5 the LM filter bank was applied to a regulartexture. The AMS procedure extracted 21 textons, the num-ber also used in the k-means clustering. However, whenordered by size, the first few mode textons are associatedwith more pixels in the image than the mean textons, whichalways account for a similar number of pixels. The differ-ence between the mode and mean textons can be seen bymarking the pixels associated with textons of the same lo-cal structure (Figure 5, bottom). The advantage of the modebased representation is more evident for the nonregular tex-ture in Figure 6, where the cumulative distribution of themode textons classified pixels is has a sharper increase.

Since textons capture local spatial configurations, we be-lieve that combining the mode textons with the representa-tion proposed in [19] can offer more insight into why thetexton approach is superior to previous techniques.


2 4 6 8 10 12 14 160

0.2

0.4

0.6

0.8

1

Texton No.

Per

cent

age

texture classified pixels

mode mean

Figure 6: Mode (�) vs. mean (Æ) based textons. The lo-cal structure is better captured by the mode textons. D040texture, S filter bank.

6. ConclusionWe have introduced a computationally efficient method thatmakes possible in high dimensional spaces the detection ofthe modes of distributions. By employing a data structurebased on locality-sensitive hashing, a significant decrease inthe running time was obtained while maintaining the qual-ity of the results. The new implementation of the mean shiftprocedure opens the door to the development of vision algo-rithms exploiting feature space analysis – including learningtechniques – in high dimensions. The C++ source code ofthis implememtation of mean shift can be downloaded from

http://www.caip.rutgers.edu/riul.AcknowledgmentsWe thank Bogdan Matei of the Sarnoff Corporation, Prince-ton, NJ, for calling our attention to the LSH data structure.This work was done during the sabbatical of I.S. at RutgersUniversity. The support of the National Science Foundationunder the grant IRI 99-87695 is gratefully acknowledged.

References[1] D. Comaniciu and P. Meer. Mean shift: A robust approach

toward feature space analysis. IEEE Trans. Pattern Anal.Machine Intell., 24(5):603–619, 2002.

[2] D. Comaniciu, V. Ramesh, and P. Meer. The variable band-width mean shift and data-driven scale selection. In Proc.8th Intl. Conf. on Computer Vision, Vancouver, Canada, vol-ume I, pages 438–445, July 2001.

[3] O. G. Cula and K. J. Dana. Compact representation of bidi-rectional texture functions. In Proc. IEEE Conf. on ComputerVision and Pattern Recognition, Kauai, Hawaii, volume 1,pages 1041–1047, 2001.

[4] M. de Berg, M. van Kreveld, M. Overmars, andO. Schwartzkopf. Computational Geometry. Algorithms andApplications. Springer, second edition, 1998.

[5] D. DeMenthon. Spatio-temporal segmentation of video byhierarchical mean shift analysis. In Proc. Statistical Meth-ods in Video Processing Workshop, Copenhagen, Denmark,2002. Also CAR-TR-978 Center for Automat. Res., U. ofMd, College Park.

[6] R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification.Wiley, second edition, 2001.

[7] A. Gionis, P. Indyk, and R. Motwani. Similarity search inhigh dimensions via hashing. In Proc. Int. Conf. on VeryLarge Data Bases, pages 518–529, 1999.

[8] G. M. Haley and B. S. Manjunath. Rotation-invariant tex-ture classification using a complete space-frequency model.IEEE Trans. Image Process., 8(2):255–269, 1999.

[9] P. Indyk. A sublinear time approximation scheme for clus-tering in metric spaces. In Proc. IEEE Symp. on Foundationsof Computer Science, pages 154–159, 1999.

[10] P. Indyk and R. Motwani. Approximate nearest neighbors:towards removing the curse of dimensionality. In Proc.Symp. on Theory of Computing, pages 604–613, 1998.

[11] E. Kushilevitz, R. Ostrovsky, and Y. Rabani. Efficientsearch for approximate nearest neighbor in high dimensionalspaces. In Proc. Symp. on Theory of Computing, pages 614–623, 1998.

[12] T. Leung and J. Malik. Representing and recognizing thevisual appearance of materials using three-dimensional tex-tons. Intl. J. of Computer Vision, 43(1):29–44, 2001.

[13] S.A. Nene and S.K. Nayar. A simple algorithm for nearest-neighbor search in high dimensions. IEEE Trans. PatternAnal. Machine Intell., 19(9):989–1003, 1997.

[14] R. Ostrovsky and Y. Rabani. Polynomial time approximationschemes for geometric k-clustering. In Proc. IEEE Symp. onFoundations of Computer Science, pages 349–358, 2000.

[15] C. Schmid. Constructing models for content-based image re-trieval. In Proc. IEEE Conf. on Computer Vision and PatternRecognition, Kauai, Hawaii, volume 2, pages 39–45, 2001.

[16] G. Shakhnarovich, P. Viola, and T. Darrell. Fast pose esti-mation with parameter-sensitive hashing. In Proc. 9th Intl.Conf. on Computer Vision, Nice, France, 2003.

[17] B. W. Silverman. Density Estimation for Statistics and DataAnalysis. Chapman & Hall, 1986.

[18] M. Varma and A. Zisserman. Classifying images of materi-als. In Proc. European Conf. on Computer Vision, Copen-hagen, Denmark, volume III, pages 255–271, 2002.

[19] M. Varma and A. Zisserman. Texture classification: Arefilter banks necessary? In Proc. IEEE Conf. on ComputerVision and Pattern Recognition, Madison, Wisconsin, vol-ume II, pages 691–698, 2003.


Mean Shift Based Clustering in High Dimensions_ a Texture Classification Example

Documents

Transcript of Mean Shift Based Clustering in High Dimensions_ a Texture Classification Example