Proceedings of the International Conference on Computer and Communication Engineering 2008 May 13-15, 2008 Kuala Lumpur, Malaysia

978-1-4244-1692-9/08/$25.00 ©2008 IEEE

Effectiveness of Relevance Feedback for Content Based Image Retrieval Using Gustafson-Kessel Algorithm

Ali Selamat, Muhammad Khairi Ismail

Faculty of Computer Science and Information Systems, Universiti Technologi Malaysia, 81310 Skudai, Johor, Malaysia

Email (aselamat@utm.my1, muhammad_khairi@yahoo.com2)

Abstract

The performance of the Content Based Image

Retrieval (CBIR) can compute using similarity of the images where user can retrieve from the image database. The term similarity in the mind of the user may different depends on the search query and the experience of the user which has been using the similar applications. When the users are not satisfied with their search results, the relevance feedback (RF) retrieval is one of the solutions for this critical problem. The user needs to use this feedback on the next retrieval process in order to increase the retrieval performance. In this paper, we have used a relevant feedback approach based on Gustafson-Kessel (GK) clustering approach in order to evaluate the effectiveness of the image retrieval results from the users. From the experiments, we have found that the RF method using Gustafson-Kessel (GK) clustering can improve the retrieval performance of the CBIR system even if we are using a large set of image datasets with a variety of images..

I. INTRODUCTION Content-based image retrieval (CBIR) is a system

that a user demands for the visual contents from the image in a color, shape, texture, and spatial layout of the search images from large scale image databases based on users’ interests and the content of images itself. The CBIR systems can be classified into two categories, which are low-level feature based system, and high-level/semantic feature based system. Low-level features are general features which have been computed from pixel values. The images are generally represented by numeric features or attributes, such as texture, color and shape [1].

In this paper we focus on the idea of using the GK clustering and relevance feedback algorithm in order to improve the CBIR system performance. We have used

the Gustafson-Kessel (GK) clustering and relevant feedback (RF) methods in order to learn human desires by using a variety of knowledge extracted from previous experience of the system. We capture the user interests from the CBIR system based on a user profile.

The first part of this paper is focusing on brief idea of GK clustering problem in image retrieval and also the relevance feedback method. The second part of the paper is focusing on the experiments and analysis of the CBIR results using GK and RF methods.

The paper is organized as follows: The methodology of the research is described in Section 3. In Section 4, we discussed about images pre-processing and then for Section 5, we analyzed the experimental results of GK clustering. The testing analysis is also described in Section 5. Last part of the section, we concludes the paper.

II. RELATED WORK In the CBIR system, the image retrieval process is

still far from user’s expectations. They still have a gap between low-level features and high-level concepts [4, 5]. One of the interactive learning techniques is “relevance feedback” (RF). It has initially been developed to narrow down this semantic gap between the users expections from the system and the results presented by the system. User interacts with the system and rates the relevance of the retrieved images according to his or her perception judgment. With this additional information, the system will dynamically learns the user’s intention, and progressively presents better results [5]. The most important thing to be learned in relevance feedback learning is the weights of different features. The feedback, provided by different users in the form of “similar” (positive) images and “dissimilar” (negative) images, is an important part of the experience [4]. We have created a user profile in the CBIR system in order to capture the

455

preferences of the images searched from the image database.

III. CONTENT BASED IMAGE RETRIEVAL OF MALAYSIA TOURISM IMAGES

We have developed a CBIR system based on the tourism images. The tourism images have been acquired based on 20 Malaysia’s travel destinations such as Eye Of Malaysia, Batu Caves, Tugu Negara, A Famosa, Komtar and including nine states of Malaysia like Perlis, Kedah, Perak, etc.. About 1000 images have been used for testing and evaluation. Our CBIR system has provide a user with a number of categories which represents a variety images in the database. Our database consists of images with size of 600 x 400 pixels contains of 500 tourism structure images, 500 varieties of other types of images. For each class of images we have employed a total of 50 images for each group. There are five main phases involved in this research for the tourism content based image retrieval (TCBIR) system as shown in Fig. 1.

Figure 1. A general framework of the research conducted on

tourism content based image retrieval (TCBIR) system

IV. IMAGES PRE-PROCESSING AND FEATURES EXTRACTION

All images must have gone through the median filtering for noise reduction and then edge detection process by using Sobel operator [12]. This process is used to find the approximate absolute gradient magnitude at each point in an input grayscale image. The data on input pixel generated from this operation

will be stored for feature extraction by the geometrical moments [6].

Figure 2. Detailed of TCBIR System Diagram

Figure 3. The sample of tourism image datasets

A. Median filter We have used Median filters in order to reduce the

noise on the images. It is particularly effective in the presence of both bipolar and unipolar impulse noise. Median filter controls the strength of the function by specifying the size of the nearest neighbourhood of surrounding pixels. We used that value to calculate the median value and this operation will minimize the blurring of the image.

B. Sobel operator Sobel Operator is a discrete differentiation operator

that performs a 2D spatial gradient measurement on an image. Typically it is used to find the approximate absolute gradient magnitude at each point in an input grayscale image. Basically, the operator consists of a pair of 3×3 convolution kernels as shown in Fig. 2. One kernel is simply the other rotated by 90°. This is very similar to the Roberts Cross operator [7].

C. Geometrical moments TCBIR images with geometrical moments

commonly used as image features. Many researches on

456

CBIR have used moments for images recognition [3] and the researcher will use the geometrical moments which provide an alternative series of expansions in representing shapes of images objects. Moments are derived from raw measurements and can be used to achieve rotation {R}, scale {S}, and translation {T} (or position) invariant [6].

The geometrical moment invariant involved a series of calculation, which is based on pixel values in whole image. We need to find the width and the height of the image to perform this operation. After that, the beginning location of the store pixel values will be determined and the system can start reading the pixel values by each row and column till it cover all resolution of the current images.

V. GUSTAFSON -KESSEL CLUSTERING The Gustafson-Kessel algorithm assigned each

cluster with both a point and a matrix and this algorithm will cluster to its centre and covariance. The original Fuzzy C-Means make the implicit hypothesis that clusters are spherical, but Gustafson- Kessel algorithm is not subject to this constraint and can identify ellipsoidal clusters. The GK algorithm is also data scale independent, i.e., if the data in any dimension are multiplied by a constant than the relative coordinates of the cluster centers and the matrix of membership degrees are identical. Moreover the convergence of this algorithm is ensured, but locally only [2].

VI. RELEVANCE FEEDBACK LEARNING The Relevance Feedback (RF) framework is less or

more to the same all CBIR current systems. A query of images is given to the system and then a set of images will be retrieved. The user will comment on or indicates which images are in the set are relevant and irrelevant. The system then takes the user's suggestion and tries achieving optimal retrieval performance [9]. The RF techniques allow the user to grade the retrieved images by their relevance to answer a given query. The relevance weights features are the basis to redo queries, taking advantage of the feedback aiming at adjusting the most influential attributes concerning the user’s view. The weights are updated during the RF iterations. [8].

VII. EXPERIMENT AND RESULT

A. Analysis of the image retrieval Results using Gustafson-Kessel Clustering

Image retrieval performance by its precision and recall to demonstrate the reliability of image clustering

and relevance feedback which has been described in Table 1 and Table 2 for classification the accuracy of the retrieved images. Precision is to measure how likely a user to find images belonging to the query category within a certain number of top matches. And recall is a fraction of all good images retrieved. These measures are defined as follows:

baecision=Pr (1)

dccall =Re (2)

DEFINITION FOR PARAMETER USED IN (1) AND (2)

VALUE MEANING a Number of positive retrievals

b Number of total retrievals c Correctly classified positive

samples d Total positive samples

We have randomly selected one of the 1000 images

as query, and other 999 remaining images as two datasets of training samples. The dataset A contains only the most attraction place in Malaysia and the second dataset with B label contains only the images of places in Malaysian state. The retrieval process has automatically been executed to provide user’s interactions. Firstly, a user needs to randomly select a concept that the query image can be recognized, and with regard to this concept the user is asked on the actual images that they are seeking. If the membership elements of the database match with the image of the desire concept, then the image will be marked with positive value. Otherwise, it is marked with a negative value. By repeating such retrievals processes for 50 times the user will select different image as query on each time of the query retrieval process, we could obtain the average precision and recall of the query results as shown in Fig. 4.This image retrieval results are evaluated based on prototype implementation of the Tourism Content Based Image Retrieval (TCBIR) system and result are evaluated in a form of precision and recall.

In this experiment, we have used a large dataset with more images have been assigned to the tourism image database which is the enhancements from the previous research. [11].This dataset has different level of low features for each class. Then we have plotted the graph of precision and recall in order to show the performance of the TCBIR system using multiple classes of images in each cluster as shown in Figs. 4, respectively.

457

DATASET A WITH NUMBER OF POSITIVE CLUSTER (RELEVANT) AND NEGATIVE CLUSTER (NOT RELEVANT)

Major Concept

Total Images

Relevant Not Relevant

Batu Caves 50 20 30 KLCC 50 30 20 AFamosa 50 20 30 KL Tower 50 31 19 Eye Of Malaysia 50 33 17

Taman Negara 50 22 28

KLS Temple 50 23 27 Penang Bridge 50 25 25

Masjid 50 10 40 KTM KL 50 12 38 Tugu Negara 50 31 19 Komtar 50 33 17 Air Terjun 50 12 38 Total 650 302 348

DATASET B WITH NUMBER OF POSITIVE CLUSTER (RELEVANT) AND NEGATIVE CLUSTER (NOT RELEVANT)

Major Concept

Total Images

Relevant Not Relevant

Johor 50 10 30 Kedah 50 12 38 Negeri Sembilan 50 13 37

Pahang 50 14 36 Perak 50 21 29 Perlis 50 12 38 Selangor 50 13 37 Total 350 95 255

Figure 4. Retrieval precision vs. recall for different experience

From the graph above that based on experiences, the average recall becomes lower and precision became higher. When experience becomes 25%, both of graphs were intercepting each other. When the experience is higher than 50%, the recall step down to lower value and precision value are increasing to the higher level. When the retrieval experiences become more than 60% it can achieve 70% of total positive

images in the positive cluster. This graph shows that better result depend on the user own experiences and this retrieval iteration play a major role to measure retrieval performance.

Figure 5. Graphical interface for Tourism Content Based

Image Retrieval (TCBIR) system

VIII. CONCLUSION From the experiments, we have found that the

images that have different meaning with different of shape and structure, will not exactly affect the precision for the evaluation of the retrieval process. But when a user search images related to “Perak” or “Taman Negara”, the precision will depend on user own perception. This issue is happened when certain images may share the same meaning. But when the TCBIR system learned more about user feedback process, the system will became more adaptable and could narrow down what the user wants exactly form the images. In other words, the system will sense the users perceptions based on the relevant feedback (RF) given by them and it will learn to satisfy what the users want

There are several improvement and future works that can be done to make the s TCBIR prototype system more adaptable with user requests. In future, we plan to show results on larger dataset and express more on accuracy of image moment in our system based on the accuracy of the retrieval results.

ACKNOWLEDGMENT This work is supported by the Ministry of Science

& Technology and Innovation (MOSTI), Malaysia and Research Management Center, Universiti Teknologi Malaysia (UTM) under the Vot 78227.

458

REFERENCES [1] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B.

Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, P. Yanker, Query by Image and Video Content: The QBIC System, IEEE Computer, 28(9): 23-32., 1999.

[2] Marie-Jeanne Lesot and Rudolf Kruse, Gustafson-Kessel-like Clustering Algorithm Based on Typicality Degrees, Int. Conf. on Information Processing and Management of Uncertainty, Paris, France,2006.

[3] Sean D. MacArthur, Carla E. Brodley, Avinash C. Kak and Lynn S. Broderick. (2002). Interactive Content-Based Image Retrieval Using Relevance Feedback. Computer Vision and Image Understanding 88:55–75.

[4] B. Bhanu and A. Dong, Concepts Learning With Fuzzy Clustering And Relevance Feedback, In Proceedings Workshop on Machine Learning and Data Mining in Pattern Recognition, Berlin .102–116, 2001.

[5] F. Jing, M. Li, L. Zhang, H.J. Zhang, B. Zhang, Learning in Region-Based Image Retrieval. Image and Video Retrieval: Second International Conference, CIVR 2003, Urbana-Champaign, IL, USA Proceedings. Springer 2003.2728: 206-215, 2003.

[6] Hu, M.K., Visual Pattern Recognition by Moment Invariants, IRE Trans. Info. Theory, vol.IT-8, pp.179-187, 1962.

[7] R.C. Gonzalez and R.E. Woods, Digital Image Processing (2nd Edition), Prentice Hall, pp.234, 2002.

[8] Agma J. M. Traina, Joselene Marques, Caetano Traina Jr,Fighting the Semantic Gap on CBIR Systems through New Relevance Feedback Techniques,Proceedings of the 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06) 0-7695-2517-,2006.

[9] P. Suman Karthik and C. V. Jawahar, Analysis of Relevance Feedback in Content Based Image Retrieval, 9th International Conference on Control, Automation, Robotics and Vision (ICARCV), 2006.

[10] A. Selamat and J.T. Teh, Malaysia Travel Destination Images Search Engine Using Geometric Moment Invariants, The International Journal of Artificial Intelligence and Machine Learning (AIML), Vol. 6, Issue 2, pp. 13-18, 2006

[11] A. Selamat and Nurul Zainuddin,Special Images Information Retrieval Using Gustafson-Kessel Relevance Feedback, The 2nd. Regional Conference on Artificial Life and Robotics, Songkhla, Thailand, pp. 86-9,2006

[12] S. P. Shouche, R. Rastogi, S. G. Bhagwat and Jayashree Krishna Sainis, Shape Analysis of Grains of Indian Wheat Varieties, Computers and Electronics in Agriculture,Volume 33, Issue 1, Pages 55-76,2001

459