Modeling Spatial-Modeling Spatial-Chromatic Distribution Chromatic Distribution

for CBIRfor CBIR

NTUT CSIE D.W. Lin

2004.3.19

Outlines

Review– Incorporating shape into color information– Geometry enhanced color histogram

Modeling spatial-chromatic distribution– Nakagami-m distribution– Refining the modeling efficiency

Experiment results at present Feature works

The integration of color and shape

Color histogram– Global color histogram– Global color histogram + spatial info.– Partition + local color histogram or color momnets

Dominant color– Extracting the representative colors of image via VQ

or clustering (e.g. k-means algorithm)– Spatial info. can be attained

• Histogram refinement• Specific-color pixel distribution (single, pair, triple …)• Edge histogram …

Spatial or frequency

Color Correlograms

For a nn with m colors image I, – The histogram is:

– The correlogram is:

– The autocorrelogram is:

ii c

Ipc IpnIh

Pr2)(

kppIpIj

ic

ji cIpIp

kcc

212

,

)(, |)( Pr

21

)()( )(,

)( II kcc

kc

Geometry-enhanced color histogram

For image I1 and I2, the similarity between autocorrelograms with color j and distance k is:

GECH uses first distance moment to decorrelate the spatial information from autocorrelogram

))]](),(([[))(),((sim 2121 IISEEII kj

kjKJ

,))(,)(min(),( 2121 jjj

jWSI WIpIpIIS

max

21 |)()(|1

d

IdIdW jj

j

Autocorrelogram and GECH

color

distance

i

531

Color moment

GECH, O(m)

Autorrelogram, O(md)

modeling

Modeled histogram , O(m)

Modeling spatial-chromatic distribution

Complexity of feature vector– High dimension for bearing more info.

Nakagami-m distribution– Adequate pixels for a perceptible color region– Clustering phenomenon for meaningful color

region (thus the beginning may not be zero)– Variety of distribution curves capture the

spatial information well

Nakagami-m distribution

Ω=E[R2], second moment

, fading figure

0 ,2 /12 2

rerm

mrP mrm

m

R

2

1 ,

22

2

m

REm

Modeling spatial-chromatic distribution – cont.

Parameters estimation for Nakagami-m– based on the maximum-likelihood – Using second order approximation

24

48366ˆ 2MLm

n

ii

N

i

rN

rN

1

2

1

21 ln

11ln

Modeling efficiency

Testset Berkeley14838 Berkeley150 Stanford

th= 3, L287.51%(61.91%)

87.74%(60.91%)

Pixel_th = 1%, L2

87.68%(62.11%)

88.55%(61.85%)

Pixel_th = 2%, L2

88.22%(62.81%)

88.98%(62.52%)

Th=3, L188.72%(58.20%)

90.95%(57.55%)

90.18%(56.27%)

Th=1%, L188.93%(58.37%)

91.14%(57.75%)

91.06%(57.17%)

Th=2%, L189.71%(59.06%)

91.74%(58.43%)

91.51%(57.79%)

Metric: intersection, compared with uniform dist.

Modeling efficiency – cont.

Testset Nor Berkeley Stanford

Pixel_threshold = 3

72.47% (0.39%)

86.51% (0.73%)

74.64% (0.27%)

Pixel_th = 1% 79.79% (2.8%)87.32% (1.13%)

82.85% (2.8%)

Pixel_th = 2% 84.51% (7.2%)89.26% (2.91%)

86.44% (6.1%)

Threshold: percentage of total DC image pixelsEntriey: rule out colors (pixels) in percentage

Refining the modeling

At the first glance: – Remove the insignificant pixel– Find out the dominant cluster

Segmentation via MED (maximum entropy discrimination)

MED

Max. entropy discrimination– discretization, classification, method(MEM)– Power spectrum estimation

For segmentation– b: SPMF– c: PMF (for max. entropy)– d: likelihood ratio

MED – cont.

20 observed values: 0.1, 0.9, 1.5, 2.0, 2.8, 3.2, 3.3, 3.5, 3.7, 3.8, 4.0, 4.5, 4.9, 5.5, 6.0, 7.3, 8.5, 8.8, 9.1, 9.5

Equal-width-interval MED

Interval width = 9.5/4 = 2.375p(I1) = (4/20)/2.375 = 0.084

Elements of Interval = 20/4 = 5p(I1) = 0.25/(2.8-0.1) = 0.084 For uniform distribution

Algorithm for refining modeling

Eta = 0.8183 Eta = 0.9563

Remarks for the algorithm

Considerations:– Choice of parameters: number of intervals,

constraints while merging the neighbor intervals

– Sparse data, or scene with texture may be in vain

– Concave region

Conclusions and feature works

Other features that may satisfy Nagakami-m modeling (avoid biased by correlogram)

Similarity measure: Battachaya distance

References

J. Cheng, N.C. Beaulieu, “Maximum-likelihood based estimation of the Nakagami m parameter,” IEEE Communications Letters, Vol. 5, No. 3, pp. 101-103,

2001 S.-H. Yang and D.-W. Lin, “A geometry-enhanced color

histogram,” IEEE Int’l Conf. Information: Research and

Education, New Jersey, USA, Aug. 2003.

References - MED

J.B. Jordan and L.C. Ludeman, “Image segmentation using maximum entropy technique,” Intl’ Conf. Acoustics, Speech and Signal Processing (ICASSP’84), Vol. 9, pp. 674-677, 1984

Hsu , T.S. Chua, and H.K. Pung, “An integrated color-spatial approach to content-based image retrieval,” Proc. Of ACM Multimedia Conf., pp. 305-313, Nov. 1995

T. Jaakkola, M. Meila, and T. Jebara, “Maximum entropy discrimination,” In Advance in Neural Information Processing Systems 12 MIT Press, 1999