SEMANTICS ORIENTED APPROACH FOR IMAGE RETRIEVAL...

SEMANTICS ORIENTED APPROACH FOR IMAGE RETRIEVAL

IN LOW COMPLEX SCENES

WANG HUI HUI

A thesis submitted in fulfilment of the

requirements for the award of the degree of

Doctor of Philosophy (Computer Science)

Faculty of Computer Science and Information Systems

Universiti Teknologi Malaysia

DECEMBER 2012

iii

“Dedicated to my beloved family and friends, without their understanding,

support, and most of all love, the completion of this work would not have been

possible.”

iv

ACKNOWLEDGEMENT

I would like to take this opportunity to thank my supervisors, Prof. Dzulkifli

Mohamad and Assoc. Prof. Dr. Nor Azman Ismail for all the invaluable guidance

and encouragement provided throughout this research. Thank you for the patience

and support you have given to me.

I would like to acknowledge the scholarship provided by the Malaysian

Ministry of Higher Education and study leave granted by Universiti Malaysia

Sarawak.

I would like to thank all of my friends for their kindness and useful

discussions, especially during difficult times.

Finally, I am deeply grateful to my beloved family for their unconditional

support, love and encouragement.

v

ABSTRACT

The explosive growth of image data leads to the need of research and

development of image retrieval. Image retrieval researches are moving from keyword,

to visual features and to semantic features. Drive towards semantic features is due to

the problem of the keywords which can be very subjective and time consuming as

low level features cannot always describe high level concepts in the users’ mind. The

main problem encountered in the image retrieval research is the semantic gap that

exists between the low-level features and high-level semantics in the images due to

the unavailability of low level image features in describing high level concepts in the

users’ mind. The aim of this research is to design and validate the semantics oriented

approach for image retrieval in low complex scenes. In order to achieve the aim and

objectives of the research, the object extraction method for identifying and extracting

the objects in a complex scene based on the colour features has been proposed. The

semantic extraction and representation method with the semantic image similarity

has also been proposed to bridge the semantic gap in image retrieval. In addition, the

semantic visual user query, namely Semantic Visual Query Builder (SeVQer), which

enables users to express their need and intent at semantic level that reduces the

semantic gap in content based image retrieval has been introduced and evaluated.

The prototype has been developed to validate the proposed approach in image

retrieval. The result of the evaluation shows that the proposed system can achieve the

retrieval accuracy of 95.8% and 89.5% for the experiments of semantic object

extraction and semantic object and their spatial relationship. The usability evaluation

indicated that the proposed semantic visual query achieved higher efficiency and user

satisfaction compared to image search by example, keyword and sketch.

vi

ABSTRAK

Pertumbuhan mendadak data imej membawa kepada keperluan penyelidikan dan pembangunan dapatan kembali imej. Penyelidikan dapatan kembali imej bergerak daripada kata kunci kepada ciri-ciri visual imej dan seterusnya kepada ciri-ciri semantik. Kecenderungan ke arah ciri-ciri semantik disebabkan oleh masalah kata kunci yang mengambil masa yang panjang dan juga sangat subjektif, manakala ciri-ciri visual imej tidak boleh sentiasa menggambarkan konsep tahap yang tinggi dalam minda pengguna. Masalah utama yang dihadapi dalam penyelidikan dapatan kembali imej ialah jurang semantik yang wujud antara ciri visual dan ciri semantik peringkat tinggi dalam imej yang disebabkan oleh ketiadaan ciri-ciri visual yang boleh menerangkan konsep tahap yang tinggi dalam minda pengguna. Tujuan kajian ini adalah untuk mereka bentuk dan mengesahkan pendekatan berorientasikan semantik untuk mendapatkan kembali imej dalam adegan kompleks rendah. Dalam usaha untuk mencapai matlamat dan objektif penyelidikan, kaedah pengekstrakan dan pengecaman objek dalam mengenal pasti dan mengekstrak objek dalam adegan kompleks rendah berdasarkan ciri-ciri warna telah dicadangkan. Kaedah pengekstrakan dan perwakilan objek semantik bagi mengekstrak objek semantik serta hubungannya dalam kalangan objek yang wujud dalam imej secara automatik dengan persamaan semantik objek juga telah dicadangkan untuk merapatkan jurang semantik dan seterusnya meningkatkan prestasi gelintaran imej pengguna. Sebagai tambahan, satu pertanyaan pengguna visual semantik iaitu Pembina Pertanyaan Visual Semantik (SeVQer) yang membolehkan pengguna mengungkapkan keperluan dan niat pertanyaan pada peringkat semantik yang merapatkan jurang semantik dalam bidang dapatan kembali imej dicadangkan dan dinilai. Prototaip telah dibangunkan untuk mengesahkan pendekatan yang dicadangkan. Hasil penilaian menunjukkan bahawa sistem yang dibangunkan boleh mencapai ketepatan dapatan kembali 95.8% dan 89.5% bagi eksperimen pengekstrakan dan perwakilan semantik. Penilaian daya guna menunjukkan bahawa Pembina Pertanyaan Visual Semantik yang dicadangkan mencapai kecekapan dan kepuasan pengguna yang lebih tinggi berbanding dengan gelintaran imej dengan contoh, kata kunci dan lakaran.

vii

TABLE OF CONTENTS

CHAPTER TITLE PAGE

DECLARATION ii

DEDICATION iii

ACKNOWLEDGMENT iv

ABSTRACT v

ABSTRAK vi

TABLE OF CONTENTS vii

LIST OF TABLES xiv

LIST OF FIGURES xv

1 INTRODUCTION 1

1.1 Motivation 1

1.2 Problem statement 3

1.3 Research questions 4

1.4 Research aim 5

1.5 Objectives 5

1.6 Scope and limitation 6

1.7 Research operation framework 8

1.8 Significance of the research 9

1.9 Organization of thesis 9

2 LITERATURE REVIEW 11

2.1 Evolution of Image Retrieval 12

2.1.1 Keyword based image retrieval 12

2.1.2 Content based image retrieval (CBIR) 12

2.1.3 Semantic based image retrieval 15

viii

2.2 Gaps in content based image retrieval 16

2.2.1 Semantic gap 16

2.2.2 Intention gap 17

2.3 Image features 18

2.3.1 Text features 18

2.3.2 Visual features 18

2.4 Object detection 19

2.4.1 Colour object extraction 20

2.4.2

Colour object integrated spatial relation extraction

21

2.5 Semantic extraction 22

2.5.1 Manual image semantic extraction 22

2.5.2 Automatic image semantic extraction 26

2.5.2.1 Image extraction 27

2.5.2.2

Region/Object semantic extraction

30

2.5.2.3

Object spatial relationship semantic extraction

32

2.6 Semantic representation 32

2.7 User query 34

2.7.1 Image browsing 35

2.7.2 Image search 35

2.7.2.1 Query by example 35

2.7.2.2 Query by keywords 37

2.7.3 User’s needs 39

2.8 Discussion 40

2.9 Summary 52

3 METHODOLOGY 53

3.1 Research methodology 53

3.2 Object detection 55

3.2.1 Visual image extraction 55

3.2.1.1 The image interpreter 56

3.2.1.2 The feature extraction 56

3.2.1.2.1

Extract color features

57

3.2.1.2.2 Image sub division 57

3.2.1.2.3 CCD layer mapping 58

3.2.1.2.4 CCL smoothing 58

3.2.1.2.5 Thresholding 60

ix

3.2.1.2.6

Remapping to CCD image

60

3.2.1.2.7

CCD image smoothing

61

3.2.2 Object identification 61

3.2.2.1 Road and building extraction 62

3.2.2.1.1 Candidate point determination

63

3.2.2.1.2 Line creation 63

3.2.2.1.3 Best line selection 64

3.2.2.1.4 Best line drawing 65

3.2.2.2 Object location 66

3.2.2.2.1

Colour cluster group identification

66

3.2.2.2.2 Horizontal scanning 67

3.2.2.2.3 Vertical scanning 68

3.2.2.2.4

Object cluster identification

68

3.2.2.2.5

Object cluster verification

69

3.2.2.2.6

Car object identification

70

3.3 Semantic extraction and representation 71

3.3.1 Semantic object extraction 72

3.3.2

Semantic object spatial relationship semantic extraction

73

3.3.2.1

Road slope as Z-axis determination

73

3.3.2.2

Calculate the slope of each possible pair of objects in image

74

3.3.2.3

Determine the relationship/ directional relationship

75

3.3.2.4

Distance of spatial relationship semantic

79

3.3.2.5

Spatial relationship semantic representation

79

3.4 Semantic Visual Query Builder (SeVQer) 79

3.4.1 Semantic colour perception 81

3.4.2 User query builder 82

3.4.2.1

Drag and drop object interested and their spatial relationship arrangement

82 3.4.2.2 Visual query extraction 83

x

3.4.3

Image spatial semantics similarity measurement

84

3.4.3.1 Query matching 85

3.4.3.1.1

Semantic colour query matching

85

3.4.3.1.2

Object semantic relationship query matching

85 3.4.3.2 Image similarity 86

3.4.3.2.1

Object semantic colour similarity

87

3.4.3.2.2

Semantic object spatial relationship similarity

87

3.4.3.3

Range of image spatial semantic similarity

88

3.5 Dataset used 88

3.6 Parameters selection 89

3.6.1 Number of colour clusters 89

3.6.1.1 Experimental setup 90

3.6.1.2

Determination of the number of colour clusters

91

3.6.2 Size of sub area of the images 92


3.6.2.2


92

3.6.3 Colour cluster distance 94


3.6.3.2


94

3.6.4 Minimum size of object clusters 96


3.6.4.2


96

3.7 Evaluation methods 98

3.7.1 Experiments on object detection 99


3.7.2

Experiments on semantic extraction and representation

100


3.7.3 System retrieval performance 101

3.7.3.1 Experiments setup 102

3.7.3.2 User queries 102

3.7.3.3 Measurements 103

xi

3.7.4

User retrieval performance and Subjective Satisfaction

104


3.7.4.2 Scenario and Tasks 105

3.7.4.3 Participants 106

3.7.4.4 Procedures 107

3.7.4.5 Measurement 108

3.8 Implementation tools 108

3.9 Summary 109

4 PROTOTYPE DEVELOPMENT 110

4.1 Algorithms 110

4.1.1 Image interpretation 111

4.1.2 Extract colour features 112

4.1.3 CCL smoothing 114

4.1.4 CCD image smoothing 115

4.1.5 Candidate points determination 117

4.1.6 Line creation 118

4.1.7 Best slope selection 118

4.1.8 Colour cluster group identification 119

4.1.9 Horizontal scanning 120

4.1.10 Vertical scanning 121

4.1.11 Object cluster identification 122

4.1.12 Object cluster verification 123

4.1.13 Semantic object extraction 124

4.1.14 Object spatial semantic relationship 125

4.1.15 Distance of spatial relationship semantic 127

4.1.16 Semantic colour query matching 127

4.1.17

Object semantic relationship query matching

128

4.1.18 Object semantic colour similarity 129

4.1.19

Semantic object spatial relationship similarity

130

4.2 Data structure 131

4.3 Semantic image retrieval and the components 134

4.3.1 New image group 135

4.3.2 Image group selection 136

4.3.3 Inserting images into database 137

4.3.4 Semantic colour name 137

4.3.5 Image browsing 138

xii

4.3.6 Image feature extraction handler 139

4.3.7 Visual query handler 142

4.3.8 Result display 144

4.4 Summary 145

5 RESULTS AND DISCUSSION 146

5.1 Experiments of colour object detection 146 5.2

Experiments of the object spatial relationships semantic extraction and representation

150

5.3 Experiments of system retrieval performance 154

5.4

Experiments of user retrieval performance and subjective satisfaction

163

5.4.1 Efficiency 164

5.4.2 User satisfaction 166 5.5 Summary 167

6 CONCLUSIONS 169

6.1 Contributions 170

6.2 Limitation 172

6.3 Future works 172

REFERENCES 174

Appendices A-G 189-210

xiii

LIST OF TABLES

TABLE NO TITLE PAGE

2.1

Research works on Content Based Image Retrieval based

on Visual Contents

14

2.2

The summary of the various semantic extraction

approaches reviewed

49

3.1 Experiments design on object detection. 100

3.2

Experiments design on semantic extraction and

representation

101

3.3 Characteristics of the experiments and user queries used 103

3.4 Implementation tools 109

5.1

The experiment results of the proposed object detection

method

147

5.3

The experiment results of the proposed semantic

extraction and representation method.

150

5.4

Retrieval Accuracy Experimented Results of Category I

Experiment

155

5.5

Retrieval Accuracy Experimented Results of Category II

Experiment

155

xiv

LIST OF FIGURES

FIGURE NO TITLE PAGE

1.1 Edge detection of traffic images using Iplab’s approach 7

1.2 The overview of the research operational framework 8

2.1 General framework of keyword based image retrieval 12

2.2 General framework of content based image retrieval 13

2.3 Wikimedia image annotation interface 23

2.4 Photoblog image annotation interface 24

2.5 Fotopages image annotation interface 24

2.6 Flickr image annotation interface 25

2.7 Inotes image annotation interface 25

2.8 Facebook image annotation interface 26

2.9 Google search engine with searching of “an apple next to

a banana”

38

2.10 The evolution of image retrieval 41

2.11

Semantic and intention gaps: semantic extraction and

representation of images

42

2.12

Semantically similar images with different low level

features

43

2.13 Semantically different images but with similar low level

features

43

2.14 False objects spatial relationships semantic extraction 46

2.15 False objects spatial relationship created by using centre

of MRB as object representation 47

2.16 False object spatial relationship using height of objects 48

xv

3.1 The research operational framework 54

3.2 The image extraction process data flow 55

3.3 The visual image extraction process data flow 55

3.4 The image interpreter process data flow 56

3.5 Feature extraction process data flow 57

3.6 The 3x3 window operation extraction for CCL images 59

3.7

Window operation for CCL image to produce CCL

smoothed image

59

3.8 The data flow of object identification process 62

3.9 The design of road and building extraction 62

3.10 The changed point of image determination 63

3.11 The possible lines created from candidate points 64

3.12 The line verification 65

3.13 Building and road region separation 65

3.14 The design of object detection 66

3.15 The colour cluster group (indicated by red dotted box) 67

3.16 The colour clusters (indicated by red dotted box) 67

3.17

The individual colour cluster (indicated by red rotted

box)

68

3.18 Objects cluster identification 69

3.19 Object clusters verification 70

3.20 Object clusters (indicated by yellow boxes) 71

3.21 Object clusters and their representative colour 71

3.22 Semantic extraction 72

3.23 Object spatial relationship semantic extraction 73

3.24 Traffic images 74

3.25 Front/Back relationship determination 76

3.26

(a) Absolute Front/Back relationship and

(b) Absolute Right/Left relationship

77

3.27 Composite relation 78

3.28 The image search interface-SeVQer 80

xvi

3.29 Colour representation used 81

3.30 The user query data flow 82

3.31 Colour cluster generated using different HSV settings 90

3.32

Colour features extracted object cluster images using

different colour cluster setting

91

3.33


different sub area size of images

93

3.34


different colour cluster distance.

95

3.35


different minimum object cluster sizes.

97

4.1 The algorithms for the image interpretation 112

4.2 Algorithm for extract colour feature 113

4.3 Algorithm for the GetMinCC function 113

4.4 Algorithm for the CCL smoothing function 115

4.5 Algorithm for CCD image smoothing function 116

4.6

Algorithm for the candidate points determination

function

117

4.7 Algorithm for the line creation function 118

4.8 Algorithm for the best line drawing function 119

4.9

Algorithm for the colour cluster group identification

function

120

4.10 Algorithm for the horizontal scanning function 121

4.11 Algorithm for the vertical scanning function 122

4.12 Algorithm for the object cluster identification function 123

4.13 Algorithm for the object cluster verification function 124

4.14 Algorithm for the semantic object extraction function 125

4.15 Algorithm for the semantic relationship function 126

4.16

Algorithm for the distance of spatial relationship

semantic function

127

4.17

Algorithm for the semantic colour query matching

function

128

xvii

4.18

Algorithm for the object semantic relationship query

matching function

129

4.19

Algorithm for the object semantic colour similarity

function

130

4.20 Algorithm for the image similarity function 131

4.21 Data structure of image group 132

4.22 Data structure of colour cluster 132

4.23 Data structure of image feature 133

4.24 Data structure of object feature 133

4.25 Data structure of colour features 133

4.26 Data structure of relation features 134

4.27 The semantic based image retrieval and the components 134

4.28 Creating a new image group 136

4.29 Image group selection 136

4.30 Inserting images into database 137

4.31 Semantic colour name 138

4.32 Image browsing 139

4.33 Image feature extraction handler 140

4.34 The image feature extraction display 141

4.35 The visual query handler 143

4.36 The generated semantic features 144

4.37 Query image results 144

5.1(a)

Some results of the colour object detection experiments-

detected vehicles (indicated by yellow MBR)

148

5.1(b)


missed car(s) (pointed by green arrow)

149

5.1(c)


False Detection (pointed by Green Arrow)

149

5.2 Sample dataset of Belkhatir(2009)’s work. 151

5.3(a)

Experiment results for semantic extraction and

representation-successful semantic feature extraction

152

xviii

5.3(b)

Experiment results for semantic extraction and

representation - false semantic features extraction

153

5.4 Retrieved results of Experiment A 157

5.5 Retrieved results of Experiment D 158

5.6 Retrieved results of Experiment E 159

5.7 Retrieved results of Experiment F 160

5.8 Missed image in retrieval for Experiment F 161

5.9 Retrieved results of Experiment H 162

5.10 Missed image in retrieval for Experiment H 163

5.11 The mean total task times 164

5.12 The mean task time, per task 165

5.13

Average subjective satisfaction for Question 5 and

Question 6

166

CHAPTER 1

INTRODUCTION

1.1 Motivation

The recently digital visual contents production has become more common

and affordable. The recent technology advancement has resulted in an enormous

increase in multimedia data. Researches (Datta, 2009; Lyman and Varian, 2003)

reported that the annual productions of images are approximately 80 billion, and

home video is approximately 1.4 billion. Online photo sharing sites are increasing

such as Flickr, Photoblog and Fotopages. By December 2010, Flickr had more than

21.3 million unique registered users and there are some five billion photos and short

videos stored on the site (Mellor, 2011). Thus, image retrieval research has been

introduced and has been a very active research area since the 1970s (Rosenfeld, 1969;

Tamura and Mori, 1977) for better image management.

Image retrieval is the field of study concerned with searching and browsing

digital images from a collection of images. Due to rapid generation of images in

digital form, image retrieval attracts interest among researchers in the fields of image

processing, multimedia, digital libraries, remote sensing, astronomy, database

applications and other related area. Effective and fast retrieval of digital images has

not always been always easy, especially when the collections grow into thousands.

2

An effective image retrieval system is needed to operate on the collection of images

to retrieve the relevant images based on human perception.

However, most common methods of image retrieval are based on visual

features or utilize some index methods. The metadata of images are indexed by

keyword, image caption or image descriptions so that retrieval can be performed

based on sample query image with indexed annotation words. As getting more visual

information is available in digital archives, the need for effective image retrieval has

become greater (Liu et al., 2007; Datta, 2007). In image retrieval research,

researchers are moving from keyword based to content based, and then towards

semantic based image retrieval.

The main problem encountered in the image retrieval research is the semantic

gap that exists between the low-level features and high-level semantics in the images

due to the unavailability of low level image features in describing high level concepts

related to the users’ need. A machine is only able to perform automatic extraction by

extracting the low level features which are represented by the colour, texture, shape

and spatial from images with a good level of efficiency. Describing images in

semantic terms is an important and challenging task that needs to be carried out to

fulfill human satisfaction and defining a semantic meaning and representation of the

input query in describing user’s needs remain as major challenges (Russ, 1999)

Therefore, research efforts are required to bridge the semantic gap in Content

based Image Retrieval (CBIR) to have more intelligent image retrieval system in

order to retrieve images that are comfort to human perception.

3

1.2 Problem Statement

“A picture is worth a thousand words"; this familiar proverb emphasizes that

visual information is inherently ambiguous and semantically rich. Human beings are

able to interpret images at different levels, both in low level features (colour, shape,

texture and object detection) and high level semantics (abstract objects and event).

However, a machine is only able to interpret images based on low level image

features. Thus, it introduces an interpretation inconsistency between image

descriptors and high-level semantics called semantic gap (Liu et al., 2007; Smeulders

et al., 2000). In addition, users prefer to articulate high-level queries (Kherfi et al.,

2004; Smeulders et al., 2000) but CBIR systems index images using low-level

features. In general, there is no automated direct link between high-level semantic

concepts and visual image features. There is a persuasive need for CBIR systems to

provide maximum support towards bridging the semantic gap between the visual

features and high level concepts to fully support the query by semantic concept (Liu

et al., 2007; Guan et al., 2009). Researchers (Chang et al., 2009; Idrissi, 2009) found

that bridging the semantic gap for image retrieval is a very challenging problem yet

to be solved.

The semantic content representation has been identified as an important issue

to bridge the semantic gap in visual information access (Wang et al., 2006). The

semantic features especially the semantic object and their semantic spatial

relationship features in the images are not fully captured and extracted (Belkhatir,

2009; Muda et al., 2009; Hollink et al., 2004). It often leads to unsatisfactory search

results (Zha et al., 2010). Representation of spatial relations semantics among objects

are important as it can convey important information about the image and to further

increase the confidence in image understanding contribute to richer querying and

retrieval facilities. In addition, the computer usually processes semantic similarity

based on low-level feature similarity, however the user queries are supposed to be

based on semantic similarity (Agrawal, 2009). Current semantic based image

retrieval are either based on visual features or the image similarity is measured based

4

on semantic matching instead of semantic similarity (Belkhatir, 2009; Muda et al.,

2009; Pratikakis, 2011; Yong, 2011).

User query has become one of the main challenges of research in the field of

CBIR which is to find exactly what a user is looking for in more effective ways.

Most of the researches in CBIR is aimed at improving retrieval performance.

Comparatively little effort has been directed towards improving the scalability

properties of retrieval methods (Heesch, 2008). So, an intuitive and visual user query

design that allows users to express their need and intent easily is needed.

1.3 Research Questions

Research questions of the study are specified as follows:

1. What is the semantic gap in image retrieval and how to bridge them?

2. How to design and demonstrate the semantics oriented approach for image

retrieval in low complex scenes?

3. How to design and demonstrate an intuitive and visual user query which allows

users to express their need for the image retrieval?

4. Will the use of the proposed object detection techniques, semantic extraction and

representation, and the visual user query be validated by the proposed prototype?

This research attempts to answer the above questions.

The first question was carried out by understanding the meaning of semantic

gap and next reviewing the techniques on how to bridge them in the field of image

retrieval. For the second research questions, the model for the semantics oriented

approach for image retrieval in low complex scenes was designed. Next, the visual

5

query that are allows users to express their need was also designed in research

question three. Lastly, for the fourth research questions, prototype was developed

and evaluated to demonstrate the system retrieval performance, user retrieval

performance and subjective satisfaction of the model and system in the image

retrieval for low complex scenes

1.4 Research Aim

The aim of this research is to design and validate the semantics oriented

approach for image retrieval in complex scenes. This aim can be achieved by

fulfilling the research objectives stated in Section 1.5.

1.5 Objectives

1. To design and develop algorithms for object detection using colour features

for low complex scenes.

2. To design and develop the semantic object spatial relationship extraction and

representation for bridging the semantic gap in image retrieval.

3. To design and develop the semantic visual user query design that allows a

user to express their need in image retrieval.

4. To develop a prototype to validate the proposed approach in image retrieval.

6

1.6 Scope and limitation

In this research domain, low complex scenes images (traffic images with

“right hand” perspective view) are chosen as dataset images to perform empirical

evaluation of the proposed method. The complex scenes are scenes that contain

irregular shapes of objects that is viewed from any direction, with self-occlusion or

partially occluded by other objects and with uniform colour intensity (Garijo et al.,

2002) and complex scenes perception is humans’ remarkable perceptual abilities to

perceive, navigate and interact with natural environments dramatically eclipse those

of current robotic systems (IGERT Complex Scene Perception, n.d.).

The colour features with spatial distribution are selected as it is able to use to

identify the objects of building and road. The car can be identified by integrating the

line detection and intersection. Cars are considered as the objects of interest and

main objects in traffic images from the human point of view. In the images used,

only non-overlapped cars are considered.

Since colour is the significance features of the car, thus colour feature with

spatial distribution are needed to differentiate the cars, the attributes of cars as well

as relations of the cars also can be identified. Whereas shape and texture features

could lead to high possibility for getting the same shape or same texture between cars

and building or background. Example, front window of car objects might have the

same shape representation of building as indicated in Figure 1.1 using red dotted box.

Same problem occurs to texture features.

7

Figure.1.1: Edge detection of traffic images using Iplab’s approach (Google Project

Hosting, 2007).

The colour cues posses highly discriminative power as it is contrast to shape

cues and invariant against scale and view (Ritter et al., 1995). Micheli et al. (1995)

stated that the most important advantage of having colour information in addition to

image luminance is that the “colour” of a surface is more stable under changes in

geometry than the corresponding image intensity value. The surfaces arrangement

and lighting conditions can be greatly changeable as well. Thus, colour plays an

important role in the images.

There are none of perfect visual features that can be selected to describe the

object accurately as discussed above. Each feature has its own advantages and

limitations. The more low level features are used, the more accurate result will be

obtained. However it will be very expensive and complex. Hence, colour features

have been selected as the low level features and could be used in extracting

interested object in traffic images.

8

1.7 Research operational framework

The overview of the research operational framework is illustrated as shown in

Figure 1.2.

Figure 1.2: The overview of the research operational framework.

A proposed image retrieval system model was constructed based on a

literature review that involves background reading, surveying and reviewing related

research in the area of bridging the semantic gap in image retrieval. The studies of

visual features, object detection using colour features, semantic extraction and

representation, and image search were carried out. There are three main modules

design of the system model, which are object detection, semantic extraction and

representation and Semantic Visual Query Builder (SeVQer). In the prototype

development, the Visual Basic programming, Microsoft Access database, ADO data

access library were used as the enabling technologies to achieve the aim of this

research. Evaluations are conducted to test the efficiency and effectiveness of the

Background reading and

literature review

Proposed System Model

Object extraction Semantic extraction

and representation

Semantic visual query

builder

Evaluation

Prototype development

Traffic images collection

9

proposed solutions. Usability test is also conducted to evaluate the user retrieval

performance and subjective satisfaction of the system model.

1.8 Significance of the research

Introducing semantic feature extraction and visual query into image retrieval

is necessary to cope with a variety of users having different needs. This study

attempts to fill the gaps that exist in image retrieval. By incorporating the proposed

semantic extraction and representation with visual query builder, it aims to create a

more effective, efficient and accurate prototype for retrieving images that conforms

to human perception.

1.9 Organization of Thesis

The thesis is organized as follows:

Chapter 2 reviews the literature of previous research relevant to this work. These

include (a) the evolution of image retrieval; (b) gaps in content based image retrieval;

(c) images features; (d) object detection techniques; (e) semantic extraction; (f)

semantic representation and (g) user query for image retrieval. A comparison

summary is also discussed.

Chapter 3 describes the research methodology, which includes object detection,

semantic features extraction and representation as well as the semantic visual query

builder used to achieve the research aim and objectives. The dataset used, parameters

selection, related evaluation methods and experimental setup are also described and

discussed.

10

Chapter 4 discusses the implementation and development of the proposed prototype.

The proposed prototype model presents the implementation tools, main algorithms

employed, data structure as well as the system model and their components. This

chapter also illustrates and explains the capabilities of the proposed prototype.

Chapter 5 presents the results and analysis of the various experiments. The

experiments that were carried out include: object detection, semantic extraction and

representation, the system retrieval performance, the user retrieval performance and

subjective Satisfaction.

Chapter 6 summarizes the research work, the contributions and presents the

limitations and future work.

174

REFERENCES

Agrawal, R. (2009). Narrowing Down the Semantic Gap between Content and

Context using Multimodal Keywords. PhD thesis. Wayne State University.2009

Bederson, B. B. (2001) Quantum Treemaps and Bubblemaps for a Zoomable Image

Browser. HCIL Tech Report 2001-10, University of Maryland, College Park,

Unites State.

Belkhatir, M. (2009). An operational model based on knowledge representation for

querying the image content with concepts and relations. Multimedia Tools

Application, 43(1), 1–23.

Belkin, N. J. (2000). Helping people find what they don’t know. Communications

of the ACM, 43(8), 58–61.

Belkin, N. J., Cool, C., Kelly, D., Lee, H.J., Muresan, G., Tang, M.C.(2003). Query

length in interactive information retrieval. Proceedings of the 26th annual ACM

international conference on research and development in information retrieval

(SIGIR’03), 28 July – 1 August, Toronto, CA. 205–212

Berlin, B., and Kay, P. (1991): Basic Color Terms: Their universality and

Evolution. UC Press.

Bovic, A.C., Clark, M and Geisler, W. S.(1990) Multichannel Texture Ana-lysis

Using Localized Spatial Filters. IEEE Transitions Pattern Analysis and Machine

Intelligence, 12(1), 55-73.

Bradshaw B. (2000). Semantic based image retrieval: A probabilistic approach.

Proceedings of the Eight ACM International Conference on Multimedia, 30

October-3 November. USA, 167-176

175

Brickley, D., and Guha, R. V. (2000). Resource Description Framework (RDF)

Schema Specification 1.0, W3C. Candidate Recommendation 2000-03-27,

February 2000.

Carson, C., Belongie, S., Greenspan, H., and Malik, J. (1997). Region-based image

querying. Proceeding of IEEE Workshop on Content-based Access of Image and

Video Libraries, 20 June.Virgin Islands, USA. 42-49.

Carson， C., and Ogle, V. E. (1996). Storage and retrieval of feature data for a very

large online image collection. Computer Society Bulletin of the Technical

Committee on Data Engineering. 19(4), 19–27

Cavazza, M., Green, R. J., and Palmer, I. J.(1998). Multimedia semantic features

and image content description. Proceeding multimedia modeling, 12-15 October

1998, Lausanne, 39–46

Chan Y., Lei, Z. B., Lopresti, D., Kung S.Y. (1997). A feature-based approach for

image retrieval by sketch. Multimedia Storage and Archiving Systems II. 3229,

220-231.

Chang, C.Y., Wang, H.Y., Li, C.F. (2009). Semantic analysis of real world images

using support vector machine. International Journal Expert Systems with

application An international journal. 36(7), 10560-10569.

Chang, S. F., Chen, W., Sundaram, H.(1998). Semantic visual templates: linking

visual features to semantics. International Conference on Image Processing

(ICIP), Workshop on Content Based Video Search and Retrieval. 3, 4-7 October.

Chicago, Illinois. 531–534.

Chang, S. K. and Hsu, A. (1992) Image information systems: where do we go from

here?. IEEE Transition On Knowledge and Data Engineering. 5(5), 431-442.

Chen, C., Gagaudakis, G., Rosin, P. (2000). Similarity-based image browsing.

Proceeding international conference of intelligent information processing, 22

August 2000,Beijing, 206–213

176

Chen,. J, Bouman, C., and Dalton J. (1998). Similarity pyramids for browsing and

organization of large image databases. Proceeding SPIE conference human

vision and electronic imaging III, 17 July. Bellingham, Washington. 563–575.

Chen, J. Y., Bouman, C.A., and Dalton, J.C. (2000). Heretical Browsing and Search

of Large Image Database. IEEE Transitions on Image Processing, 9(3), 442-455.

Chen, Z. J. (2010). Semantic Research on Content-Based Image Retrieval.

Multimedia Technology, International Conference on Multimedia Technology,

29-31 October, Ningbo, China. 1-4

Choi, Y. (2010). Effects of contextual factors on image searching on the Web.

Journal of the American Society for Information Science and Technology,

61(10), 2011-2018.

Choi, Y., Tasmussen, E. M., (2003）. Searching for Images: The Analysis of Users’

Queries for Image Retrieval in American History. Journal of the American

Society for Information Science and Technology, 54(6), 498-511.

Chris Mellor (n.d.). The register. Retrieved 10 December 2010. Website:

http://www.theregister.co.uk/2011/02/02/flickr_nix_4000_pix.

Colombo, C., Bimbo, D. A., and Pala, P. Semantics in visual information retrieval.

IEEE Multimedia, 6(3), 38–53.

Cox, I. J., Minka, T. P., Papathomas, T. V. and Yianilos, P. N. (2000). The bayesian

image retrieval system, pichunter: Theory, implementation, and psychophysical

experiments. IEEE Transaction on Image Processing, 9(1), 20-37.

Cox K. (1992). Information retrieval by browsing. Proceeding international

conference new information technology, 30 November–2 December, Hong

Kong, 69-80.

Croft, W. B., and Thompson, R. H. (1987). I3R: A new approach to the design of

document retrieval systems. Journal of the American Society for Information

Science, 38(6), 389–404.

Datta, R. (2009) Semantics and Aesthetics inference for Image Search: Statistical

Learning Approaches. PhD THesis, The Pennsylvania State University.

177

Datta, R., Joshi, D., Li, J., Wang, J. Z. (2008). Image Retrieval: Ideas, Influences,

and Trends of the New Age. ACM Transactions on Computing Surveys, 40(2),

5(1-60).

Daoudi, M., and Matusiak, S. (2000). Visual Image Retrieval by Multiscale

Description of User Sketches. Journal of Visual Languages and Computing

11(3), 287-301.

Descampe, A,. Vleeschouwer, C., Iregui, M., Macq, N., Marqués, F. (2007).

Prefetching and caching strategies for remote and interactive browsing of

JPEG2000 images. IEEE Transition Image Process 16(5):1339–1354.

Drucker, H., Shahraray, B., and Gibbon, D. C.(2001). Relevance feedback using

support vector machine. Proceedings of the 18th

International Conference on

Machine Learning. 28 June -1 July. MA, USA. 122-129.

Duygulu, P., Barnard, K., Fretias, N., and Forsyth, D. (2002). Object recognition as

machine translation: Learning a lexicon for a fixed image vocabulary.

Proceedings of the European Conference on Computer Vision, 27 May- 2 June.

Copenhagen, 97–112.

Eakins, J. P., Briggs, P., and Burford, B. (2004). Image Retrieval Interfaces: A User

Perspective. Image and Video Retrieval. 3115, 628-637.

Egenhofer, M.(1997). Query processing in spatial query by sketch. Journal of

Visual Language and Computing, 8(4), 403-424.

Facebook (2004), Retrieved 3 June 2010. Website: http://www.facebook.com.

Fensel, D. (2000). The semantic web and its languages. Intelligence Systems and

their applications. 15(6), 67-73.

Flicker (2004.), Retrieved 3 June 2010.

Website: http://www.flickr.com/photos/upload/done/

Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani,

M., Hafner, J., Lee, Dennis, Petkovic, D., Steele, David ., Yanker, P. (1995).

Query by Image and Video Content: The QBIC system. IEEE Computer Special

Issue on Content-based Retrieval. 28 (9), 23-32.

178

Frankel, C., Swain, M.J., and Athitsos, V.(1996). WebSeer : An image search engine

for the world wide web. Technical Report. University of Chicago.

Garijo, F. J., Riguelmen, J. C., Bonilla, M. T. (2002). Advances in Artificial

Intelligence. Springer; 1 edition.

Gerrig, R. J., and Zimbardo, P. G. (2001). Psychology and Life. 16 Edition. Allyn

and Bacon.

Google Goggles. (2010), Retrieved 30 May 2011.

Website: www.google.com/mobile/goggles/

Google: Image Search (1996).Retrieved10 December 2010, website :

http://support.google.com/images/bin/answer.py?hl=en&answer=1325808.

Google project hosting (2007), Image Processing Lab (iplab). Retrieved 3 April

2011. Website : http://code.google.com/p/iplab/

Guan, H.Y., Antani, S., Long, L. R., and Thomas, G. R. (2009). Bridging the

semantic gap using ranking SVM for image retrieval. Proceedings of the Sixth

IEEE international conference on Symposium on Biomedical Imaging, 28 June-1

July. Boston, MA. 354-357.

Gudivada, N. V., and Raghavan, V. V (1995). Content-based image retrieval

systems. Journal Computer, 28(9),18–22.

Hafner, J., Harpreet S. H., Equitz, W., Flickner, M. and Niblack, W. (1995).

Efficient color histogram indexing for quadratic form distance functions. IEEE

Transactions on Pattern Analysis and Ma-chine Intelligence, 17(7), 729-736

Hearn, D., and Baker, M. P. (1994). Computer Graphic. National Center for

Supercomputing Applications, University of Illinois. Prentice Hall.

Heesch, D. (2008) A survey of browsing models for content based image retrieval.

Multimedia Tools and Application, 40(2), 261-284.

Hermes, T., Klauck, C., Kreyb, J., and Zhang, J. (1995). Image retrieval for

information systems. Proceeding storage and retrieval for image and video

databases III, 5-10 February, CA, USA. 2420,394–405

179

Hollink, L., Nguyen, G., Schreiber, G., Wielemaker, J., and Wielinga, B. (2004).

Adding spatial semantics to image annotations. The 4th International Workshop

on Knowledge Markup and Semantic Annotation, 7-11 November, Hiroshima,

Japan, 31-40

Hong, V., Paulus, D (2009). Parameter Study and Optimization of a Color-Based

Object Classification System. International conference of Soft Computing and

Pattern Recognition, 4-7 December. Melacca, Malaysia, 31-36.

Hou, T. Y.; Hsu, A., Liu, P. Y.; Chiu, M. Y. (1992). A content-based indexing

technique using relative geometry features. Image Storage and Retrieval

Systems, Proceeding SPIE, 1662, 59-68.

Hsu, W., Chua, T. S., and Pung, H. K. (1995). An integrated color-spatial approach

to content-based image retrieval. Proceedings of the third ACM international

conference on Multimedia, 5-9 November, San Francisco, USA. 305-313.

Hu, J. H., Ma, P. C., Chau, Y. K. (1999). Evaluation of user interface designs for

information retrieval systems: a computer-based experiment. Decision Support

Systems. 27(1-2), 125–143.

Hu, J., Wang, G.., Lochovsky, F., Sun, J. T., Chen, Z. (2009). Understanding User’s

Query Intent with Wikipedia early years. Proceeding of the 18th

international

conference on world wide wide. 20-24 April. Raleigh, North Carolina. 471-480.

Huang, J. (1998). Color-Spatial Image Indexing and Applications. PhD thesis,

Cornell University.

HyvÄonen, E., Harjula, P., and Viljanen, K.(2002). Representing metadata about

web resources. Helsinki Institute for Information Technology (HIIT) Publishing.

Idrissi, N.(2009). Bridging the Semantic Gap for Texture-based Image Retrieval

and Navigation. Journal Of Multimedia. 4(5), 277-283.

IGERT Complex Scene Perception? (n.d). Retrieved on 1 October 2012. Website:

http://igert.perception.upenn.edu/index.php?n=Main.HomePage

Inote: Image Annotation in Java (1998), Retrieved 3 June 2010.

Website :http://www.iath.virginia.edu/inote/.

180

Ion, A., Stanescu, L., Burdescu, D., Udristoiu, S. (2008). Mapping image low level

descriptors to semantic concept. The 3rd international multi conference on

computing in the global information Technology. 27 July-1 August. Athens.

Greece, 154-159.

ISO 9241-11: Guidance on usability. (n.d.). Retrieved 3 June 2011, Website:

http://www.userfocus.co.uk/resources/iso9241/part11.html.

ISO/IEC 9126-4: Quality in use metrics. Retrieved on June 2011. Website:

http://www.usabilitynet.org/tools/r_international.htm#9126-4.

Jansen, B. J., Spink, A., and Saracevic, T. (2000). Real life, real users, and real

needs: a study and analysis of user queries on the web. Information Processing

& Management. 36, 207–227.

Jeon,J., Laverenko,V., and Manmatha, R. (2003), Automatic image annotation and

retrieval using cross-media relevance models. Proceedings of international ACM

Conference on Research and Development in Information Retrieval. 28 July- 1

August. Toronto, Canada,119-126

Juang, C.F., Sun, W. K., Chen, G..C. (2009). Object detection by color histogram

based fuzzy classifier with support vector learning. Journal of Neurocomputing,

72(10-12), 2464–2476

Kanehara, F., Satoh, S. and Hamada, T. (1995). A flexible image retrieval using

explicit visual instruction. Proceedings of the Third International Conference on

Document Analysis Recognition, 14-16 August. Montreal, Canada.175–178.

Kankanhalli, M. S. and Mehtre, B. M. (1995) Color and Spatial Feature For

Contents-Based Image Retrieval. Technical Report. R95-184-0. Institute OF

System Science, National University Singapore.

Kato, T., Kurita, T., Otsu, N., and Hirata., K. (1992). A sketch retrieval method for

full color image database query by visual example. Proceeding of International

Conference on Pattern Recognition, 30 August–3 September. Hague, Holland.

530–533.

181

Kelly, D., Fu, X. (2007). Eliciting better information need descriptions from users

of information search systems. Information Processing and Management 43,

30–46.

Kelly, P., Cannon, T., and Hush, D. (1995). Query by image example: The

comparison algorithm for navigating digital image databases (CANDID)

approach. In Storage and Retrieval for Image and Video Databases (SPIE).

238–248..

Khan, W., Kumar, S., Gupta, N., Khan, N. (2001). Signature Based Approach For

Image Retrieval Using Color Histogram And Wavelet Transform. International

Journal of Soft Computing and Engineering. 1(1), 43-46.

Kherfi, M.L., Ziou,D. and Benard, A.(2004). Image retrieval from the World Wide

Web: Issues, techniques and systems. ACM Computing Surveys, 36(1), 35-67.

Laaksonen, J., Koskela, M., and Oja, E. (1999). Picsom: Self-organizing maps for

content-based image retrieval. Proceeding of International Joint Conference on

Neural Network, 10 July-16 July. Washington, DC, 2470–2473

Lai, T. S.(2000). CHROMA: A Photographic Image Retrieval System. PhD Thesis,

University of Sunderland.

Lassila, O., and Swick, R. R. (1999). Resource description framework (RDF): Model

and syntax specification. Technical report, W3C, February 1999. W3C

Recommendation

Lim, J., Arbeláez, P., Gu, C., and Malik, J.(2009). Context by Region Ancestry.

Proceedings of the IEEE International Conference on Computer Vision, 27

September- 4 October, 1978-1985

Lavrenko,V., Manmatha,R., and Jeon, J. (2003). A model for learning the semantic

of pictures. Proceeding of Advances in Neural Information Processing Systems,

8-13 December. British Columbia, Canada, 80-89.

Lew, M. S., Sebe, N., Djeraba, C., Jain, R. (2006). Content-based multimedia

information retrieval: State of the art and challenges. ACM Transaction

Multimedia Computation Communication Application. 2(1), 1-19.

182

Liu, Y. (2007). Region-Based Image Retrieval with High-Level Semantics. PhD

Thesis, Monash University.

Liu, Y., Zhang, D. S., Lu, G.. J., Ma, W. Y. (2007). A survey of content-based image

retrieval with high-level semantics. Pattern Recognition, 40(1), 262 – 282

Lyman, P. and Varian, H. R. (2003). How much information, Retrieved 10

December 2010.Website:

http://www2.sims.berkeley.edu/research/projects/how-much-info-2003ChrisMel

on.

MacArthur, S.D., Brodley, C. E., and Shyu, C. R. (2000). Relevance feedback

decision trees in content- based image retrieval. Proceeding of IEEE Workshop on

Content-based Access of Image and Video Libraries. Hilton Head Island, SC, 68-72.

Manjunath, B. S and Ma, W. T. (1996). Texture features for browsing and retrieval

of large image data. IEEE Transactions on Pattern Analysis and Machine

Intelligence, 18(8), 837-842

Meghini, C., Sebastiani, F., and Straccia, U.(1997). The terminological image

retrieval model. Proceeding of 9th international conference on image analysis

and processing (ICIAP’97).17-19 September. Florence, Italy, 156–163.

Mehtre, B. M., and Kankanhalli, M.S. (1995). Contents-Based Image Retrieval

Using Composite Color-shape Approach. TR95-190-0. Institute OF System

Science, National University Singapore. 1995

Ménard, E. (2011). Search Behaviours of Image Users: A Pilot Study on Museum

Objects. Partnership: the Canadian Journal of Library and Information

Practice and Research. 6, 1(1-18).

Micheli, D. E., Prevete, R., Piccioli, G.., Campani, M. (1995), Colour cues for

traffic scene analysis. Proceeding of the Intelligent vehicles symposium.25-26

September. USA, 466-471

Michel, S., Karoubi, B., Bigun, J. and Corsini, S. (1996). Orientation radiograms

for indexing and identification in image databases. Proceedings Eusipco-96,

European conference on signal processing, pages. 1693–1696.

183

Min, Y., Rong, A., and Zhen, Y. (2008). Research on Methodology of Image

Semantic Understanding. Proceeding of SPIE. 6623(1-6).

Minka, T.(1996). An image database browser that learns from user interaction. MIT

Media Laboratory Technical Report #365, Massachusetts Institute of Technology

Mittal, A., Cheong, L. F.(2003). Framework for synthesizing semantic-level

indexes. Multimedia Tools and Applications 20(2),135–158.

Mokhtarian, F., Abbasi, S. and Kittler, J. (1996). Efficient and robust retrieval by

shape content through curvature scale space. Series on Software Engineering

and Knowledge Engineering. 8, 35-42.

Mori, Y., Takahashi, H., and Oka, R. (1999). Image-to-word transformation based

on dividing and vector quantizing images with words. Proceedings of the First

International Workshop on Multimedia Intelligent Storage and Retrieval

Management.

Muda, Z., Lewis, P. L., Payne, T. R., Weal, M. J. (2009). Enhanced Image

Annotations Based on Spatial Information Extraction and Ontologies.

Proceedings of the IEEE International Conference on Signal and Image

Processing Application, 18-19 November, Kuala Lumpur, Malaysia,173-178.

Mueen, A. (2009) Multi-level Automatic Medical Image Annotation. PhD Thesis.

University of Malaya.

Muller, H., Muller,W., Squire,D. M., Maillet, S. M., Pun, T. (2001). Performance

evaluation in content-based image retrieval: overview and proposals.

Image/Video Indexing and Retrieval. 22(5), 598-601,

Mukherjea, S., Hirata, K. and Hara, Y. (1997). Towards a multimedia world wide

web information retrieval engine. Journal computer Networks. 29, 8-13.

Muselet, D. and Macaire, L. (2007). Combining color and spatial information for

object recognition across illumination changes. Journal Pattern Recognition

Letters 28 (10), 1176–1185

184

Nakazato, M., Manola, L. and Huang, S. (2002). ImageGrouper: Search, Annotate

and Organize Images by Groups. Proceedings of the 5th International

Conference on Recent Advances in Visual Information Systems. 11-13 March,

Hsin Chu, Taiwan 129-142

Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E., Petkovic, D.,

Yanker, P., Faloutsos, C. and Taubin, G. (1993). The QBIC project: querying

images by content using color, texture, and shape. Storage and Retrieval for

Image and Video Databases, 1908, 173-181

Ortega, M., Rui, Y., Chakrabarti, K., Mehrotra, S. and Huang, T.S. (1997).

Supporting similarity queries in MARS. Proceedings of the 5th ACM

International Multimedia Conference. 8-14 November. Seattle. Washington,

403–413.

Pass, G. and Zabih, R. (1996). Histogram Refinement for Content Based Image

Retrieval. Proceeding 3rd IEEE Workshop on Applications of Computer Vision,

2-4 December. Florida, USA, 96- 102.

Pecenovic, Z., Do, M. N., Vetterli, M. and Pu., P. (2000). Integrated Browsing and

Searching of Large Image Collections. Advances in Visual Information Systems,

1929, 179-189.

Pentland, A. R., Picard, W., Sclaroff, S. (1995). Photobook: tools for content-based

manipulation of image databases. International Journal of Computer Vision

18(3), 233-254.

Photoblog-Your Life in Photos (2000), Retrieved 3 June 2010. Website :

http://www.photoblog.com/create/

Photo Blogging made Easy-fotopages (2000), Retrieved 3 June 2010. Website:

http://www.fotopages.com/cgi-bin/account.pl?page=3

Pratikakis, I., Bolovinou, A., Gatos, B., and Perantonis, S. (2011).Semantics

Extraction from Images. Multimedia Information Extraction, 50–88.

185

Rahman, M. M. (2008). Semantical representation and retrieval of natural

photographs and medical images using concept and context based feature

spaces. PhD thesis. Concordia University

Ritter, W., Stein, F. and Janssen, R.(1995). Traffic sign recognition using colour

information. Mathematical and Computer Modelling. 22(4), 149-157

Rocchio, J.(1971). Relevance feedback in information retrieval. Prentice-Hall.

Roden, K. (1999). How do people organise their photographs? Proceedings of the

BCS IRSG 21st Annual Colloquium on Information Retrieval Research, 19-20

April., Glasgow, UK

Rosenfeld, A. (1969). Picture processing by computer. ACM Computing Surveys,

1(3), 147-176.

Rui, Y., Huang, T. S., and Mehrotra, S. (2007). Content-based image retrieval with

relevance feedback in mars, Proceeding of international conference image

processing. 26-29 October. Texas, USA, 815-818.

Russ, J. C. (1999). The Image processing handbook. Third edition. A CRC

Handbook published in cooperation with IEEE press.

Santini, S. and Jain, R.(2000) Integrated Browsing and Querying for Image

Database. Journal IEEE Multimedia, 7(3), 26-39.

Shen H T., Ooi B C., and Tan K L. Giving meanings to www Images. Proceeding

of ACM international conference on Multimedia. 30 October–3 November. CA,

USA, 39-48.

Shih, T. K., Huang, J. Y., Wang, C. S and Kao, C. H. (2001). An intelligent

content-based image retrieval system based on color, shape and spatial

relations. Proceedings of National Science Council, 25(4), 232-243.

Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., and Jain, R. (2000).

Content-based image retrieval at the end of the early years. IEEE Transactions

on Pattern Analysis and Machine Intelligence, 22 (2), 1349-1380.

186

Smith, J. R. and Chang, S.F. (1994). Transform features for texture classification

and discrimination in large image databases. IEEE International Conference on

Image Processing Proceedings. 1-3 November. 3, 407-411.

Smith, J., and Chang, S.F. (1996). Tools and Techniques for Color Image Retrieval.

Storage and Retrieval for Still image and video database, 2670, 1630-1639.

Srihari, R K., Zhang, Z., and Rao, A. (2000). Intelligent indexing and semantic

retrieval of multimodal documents. Information Retrieval, 2 (2), 245-275.

Stricker, M., and Dimai, A. (1996). Color indexing with weak spatial constraints.

Storage and Retrieval for Still image and video database, 2670, 29-40.

Stricker, M., and Orengo, M. (1995). Similarity of color images. Storage and

Retrieval for Image and Video Databases III. 5-10 February, San Diego, USA,

2420, 381-392.

Su, Y., Allan, M., Jurie, F. (2010) Improving object classification using semantic

attributes. Proceedings of the British Machine Vision Conference, 31 august - 3

September. Aberystwyth., UK, 26(1-10).

Swain, M. and Ballard, D. (1991). Color indexing. International Journal of

Computer Vision, 7(1), 11–32.

Szummer, M. and Picard, R.W. (1998). Indoor-Outdoor Image Classification.

Proceedings of IEEE International workshop on Content-based Access of Image

and Video Database, 3 January. Bombay, India.45-51.

Tamura, H., and Mori, S. (1977). A data management system for manipulating large

images. Proceedings of Workshop on Picture Data Description and

Management, 21-22 April, Los Alamitos, Calif. 45-54.

Tamura, H., and Yokoya, N. (1984). Image database systems: A survey. Pattern

Recognition. 17(1). 29-43.

Tieu, K., and P. Viola. (2004). Boosting image retrieval. Journal of computer vision

–special issues in content based image retrieval. 56(1-2), 17-36.

187

Tong.S., and Chang, E. (2001). Support vector machine active learning for image

retrieval. Proceeding of ACM Multimedia, 30 September–5 October. Ottawa,

Canada, 101-118

Tzagkarakis, G and Tsakalides, P. (2004). A Statistical approach to texture image

retrieval via alpha-stable modeling of wavelet decompositions. Proceeding 5th

International Workshop on Image Analysis for Multimedia Interactive Services,

April 21-23, Lisbon, Portugal. 21-23.

Uijlings, J., Smeulders, A.W.M. and Scha, R. J. H. (2009). What is the spatial

extent of an object? Proceedings of the IEEE Conference on Computer Vision

and Pattern Recognition, 20-25 June 2009. Florida, USA. 770-777.

Usability Guideline.(n.d.). Retrieved on 3 June 2010. Webiste:

http://www.testingstandards.co.uk/usability_guidelines.htm.

Urban, J., Jose, J. M. (2005) An Explorative Study of Interface Support for Image

Searching. Adaptive Multimedia Retrieval, 207-221

Vailaya A, Figueiredo, A. T., Jain, A. K., Zhang H. J. (2001). Image classification

for content-based indexing. IEEE Transactions on Image Processing, 10(1),

117-130.

Vailaya, A., Jain, A.K. and Zhang, H. (1998). On Image Classification: City Images

vs. Landscapes, Pattern Recognition, 31(12), 1921-1935.

Vasconcelos, N., and Lippman, A. (1999). Learning from user feedback in image

retrieval systems. Proceeding of the 12th Conference on Advances in Neural

Information Processing Systems, 29 November-4 December. Denver, Colorado,

76-89.

Wang, H. H. (2003). Spiral bit map signature for content based image retrieval.

Master thesis. Universiti Malaysia Sarawak.

Wang, L., Khan, L. (2006). Automatic image annotation and retrieval using

Weighted Feature Selection. Journal Multimedia Tools an Application, 29(1),

55-71.

188

Wang, Y.C. (2000). The integration of colour cluster and distribution for image

retrieval. Master thesis. Universiti Malaysia Sarawak

Wang, Z.Y., Feng, D., Chi, Z.R. and Xia, T.(2006). Annotating Image Regions

Using Spatial Context. Proceedings of International Symposium Multimedia,

December. San Diego, CA. 55-61

Westman, S. (2009). Image Users’ Needs and Searching Behaviour in

Information Retrieval Searching in the 21st Century. Willey publisher.

Wikipedia (2001), Retrieved 3 June 2010. Website:

http://commons.wikimedia.org/wiki/Special:Upload

Yoon, J. W. (2011). Searching images in daily life. Library and Information Science

Research 33, 269–275.

Yu, X. H., Xu, J. H. (2008). The Related Techniques of Content-based Image

Retrieval. Proceedings of International symposium on computer science and

computational technology. 1, 154-158.

Zha, Z. J, Yang, L. J, Mei, T., Wang, M, Wang, Z.F, Chua, T. S., Hua, X.S. (2010).

Visual Query Suggestion: Towards Capturing User Intent in Internet Image

Search. ACM Transactions on Multimedia Computing, Communications, and

Applications. 6(3),13(1-19)

Zhang, D. Y. (2011). A Survey of User Interfaces in Content-based Image Search

Engines on the Web. Journal of Communication and Computer. 8(11),

129-132.

Zhao, R., and W. I. Grosky. (2001). Bridging the Semantic Gap in Image Retrieval.

Distributed Multimedia Databases: Techniques and Applications, Idea Group

Publishing, Hershey, Pennsylvania, 13-36.

Zhuang, Y., Mehrotra, S., and Huang, T. S. (1999). A multimedia information

retrieval model based on semantic and visual content. Proceeding of 5th

international conference for young computer scientists (ICYCS), Nanjing, China

1999

http://www.mendeley.com/research/image-users-needs-searching-behaviour/

SEMANTICS ORIENTED APPROACH FOR IMAGE RETRIEVAL...

Documents

Transcript of SEMANTICS ORIENTED APPROACH FOR IMAGE RETRIEVAL...