Semantic Strategic Satellite Image Retrieval

Wassim Messaoudi, Imed Riadh Farah,

Karim saheb ettabâa

Laboratoire de Recherche en Informatique Arabisée et

Documentique Intégrée (R.I.AD.I)

Ecole Nationale des Sciences de l'Informatique.

Campus Universitaire de Manouba, 2010 Manouba, Tunisie

e-mail: wassim_messaoudi@yahoo.fr

e-mail: riadh.farah@ensi.rnu.tn

e-mail: karim.sahebettabaa@riadi.rnu.tn

Basel Solaiman

Laboratoire Image et traitement de l’information (ITI)

Ecole nationale supérieure de télécommunication de

Bretagne

Technopôle Brest Iroise CS 83818,

29238 Brest Cedex France

e-mail: Basel.Solaiman@enst-bretagne.fr

Abstract—Large amounts of spatial data are becoming available

today due to the rapid development of remote sensing techniques.

Several retrieval systems are proposed to retrieve necessary,

interested and effective information such as key-word based

image retrieval and content based image retrieval. However, the

results of these approaches are generally unsatisfactory,

unpredictable and don’t match human perception due to the well

gap between visual features and semantic concepts [1].

In this paper, we propose a new approach allowing semantic

satellite image retrieval, describing the semantic image content

and managing uncertain information. It’s based on ontology

model which represents spatial knowledge in order to provide

semantic understanding of image content.

Our retrieval system is based on two modules: ontological model

merging and semantic strategic image retrieval. The first module

allows developing ontological models which represent spatial

knowledge of the satellite image, and managing uncertain

information. The second module allows retrieving satellite images

basing on their ontological model. In order to improve the quality

of retrieval system and to facilitate the retrieval process, we

propose two retrieval strategies which are the opportunist

strategy and the hypothetic strategy.

Our approach attempts to improve the quality of image retrieval,

to reduce the semantic gap between visual features and semantic

concepts and to provide an automatic solution for efficient

satellite image retrieval.

Keywords-Semantic image retrieval; scene interpretation;

knowledge representation; ontology merging; semantic similarity.

I. INTRODUCTION

With the rapid development of remote sensing platform and sensor technique, the amounts of satellite images have increased frequently. Many approaches have been proposed such as the text- based retrieval and the content based image retrieval (CBIR). Text-based approach consists to attach keywords or labels to each item and then to perform searches based in on these labels [2]. CBIR approach extracts low-level features to index image such as color, texture and shape [2]. However, these approaches are inefficient due to the well gap between visual features and semantic concepts. Several systems are proposed to improve the retrieval quality and to

provide semantic in retrieval process such as relevant feedback [3], semantic template [4], machine learning [5] and ontology [6], etc.

In this paper, we propose a new approach allowing semantic satellite image retrieval, describing the semantic image content and managing uncertain information. It’s based on ontology approach which represents spatial knowledge in order to provide semantic understanding of image content. Ontology is recently-introduced tools for structuring, processing and managing knowledge. It is a formal explicit specification of a shared conceptualization [7]. It provides a shared and common understanding of a domain that can be communicated across people and application systems. The conceptualization is the couching of knowledge about the world in terms of entities (things, the relationships they hold and the constraints between them). The specification is the concrete representation of this conceptualization.

Our retrieval system is based on two modules: ontological model merging and semantic strategic retrieval. The first module allows developing ontological models which represent spatial knowledge of the satellite image. In addition, it manages uncertain information and resolves conflicts situations using a fusion algorithm for merging ontological models which represent satellite image knowledge. The second module allows retrieving satellite images basing on their ontological model. In order to improve the quality of retrieval system and to facilitate the retrieval process, we propose two retrieval strategies which are the opportunist strategy and the hypothetic strategy. These strategies allow reducing the ontological model base and facilitate the retrieval process.

This paper is organized as follows: section 2 describes image retrieval systems and the semantic gap; section 3 presents the ontology approach and techniques for measuring similarity between ontological models; section 4 presents the proposed approach and discusses their modules; section 5 presents our conclusions.

II. IMAGE RETRIEVAL SYSTEMS

Several image retrieval systems are developed such as text-based image retrieval and content-based image retrieval. In the text-based approach, the images are manually annotated by text

descriptors (keyword) which are then used in the retrieval process. But, this approach requires a considerable effort for manual annotation, and the keyword does not necessarily describe image content. In addition, the user may know little about the domain, and thus can’t specify the most appropriate keywords for image retrieval [8].

To overcome the above disadvantages in text-based retrieval system, content-based image retrieval (CBIR) was introduced in the early 1980s. In this approach, images are indexed by their visual content, such as color, texture, shapes, etc [9]. To retrieve images, users provide image query or sketched figures; then, the system changes these examples into its internal representation of feature vectors. The similarities between the feature vectors of the query example and those of the images in the database are then calculated and retrieval is performed with the aid of an indexing scheme. The indexing scheme provides an efficient way to search for the image database [2]. However, the similarity measures between visual features do not match human perception [8]; for example, two images can be very similar in color, size, and shape, despite containing different objects. Thus, the retrieval results of low-level features based retrieval approach are generally unsatisfactory and often unpredictable.

A. The semantic gap

The discrepancy between the limited descriptive power of low-level image features and the richness of user semantics is referred to as the ‘semantic gap’ [10]. In order to reduce the semantic gap between low-level features and high-level features, and to provide semantic in retrieval process several approaches are proposed such as relevant feedback [3], semantic template [4], machine learning [5] and ontology [6], etc.

Relevant feedback approach was used in text-based information retrieval and was introduced to CBIR to bring user in the retrieval process for reducing the semantic gap between what queries represent (low-level features) and what the user thinks [11]. In this approach, the system provides initial retrieval results. Then, the user judges the above results by selecting the accepted results. Then, a machine learning algorithm is applied to learn the user feedback [12].

In order to derive high-level semantic features, machine learning techniques have been introduced in CBIR such as neural network for concept learning [13], Bayesian network for indoor/outdoor image classification [14] and SVM for image annotation [15].

Ontology-based approach is proposed to define high-level concepts. In [16], authors are proposed an ontology-object which describes each region of an image by its average color in lab color space, its position in vertical and horizontal axis, its size and shape. In [17] authors are proposed and ontology-approach for retrieving images basing on its semantic content.

III. ONTOLOGY

Ontologies now play an important role for many knowledge-intensive applications for which they provide a source of precisely defined terms [17]. It is a formal explicit

specification of a shared conceptualization [7]. It defines a set of terms called concepts. It can be constructed in two ways which are domain-dependent and generic. Generic ontologies are definitions of concepts in general; such as WordNet, which defines the meaning and interrelationships of English words. A domain-dependent ontology generally provides concepts in a specific domain, which focuses on the knowledge in the limited area, while generic ontologies provide concepts more comprehensively.

In ontology, concepts are the fundamental units for specification, and provide a foundation for information description. In general, each concept has three basic components: terms, attributes and relations. Terms are the names used to refer to a specific concept, and can include a set of synonyms that specify the same concepts. Attributes are features of a concept that describe the concept in more detail. Finally relations are used to represent relationships among different concepts and to provide a general structure to the ontology [17].

Three kinds of inter-relationships are generally represented in an ontology: “IS-A”, “Instance-Of”, and “Part-Of”. These relations correspond to key abstraction primitives in object-based and semantic data models. “Instance-Of” relation shows membership between concepts, while “Part-Of” shows composition relationships. “IS-A” relation shows concept inclusion, it used in similarity comparison in ontology-based image retrieval. When concept has an “IS-A” relation to another concept, this means that the second concept is more general than the first concept. If concept A has relation “IS-A” to concept B, we call concept A a sub-concept and call concept B a super-concept. One characteristic of “IS-A” relation is that all the attributes of a super-concept can be inherited by its sub-concepts. Sub-concepts normally have more attributes than super-concepts and as a result, correspondingly sub-concepts are more specific [17].

A core purpose for the use of ontologies is the exchange of data not only at a common syntactic, but also at a shared semantic level.

A. Measuring similarity between ontological models

The measures for similarity computation can be divided into two general groups [18]; namely, lexical measures comparing entity labels, and structural measure comparing taxonomic hierarchy. In [18], seven criteria are extracted for deciding that two entities are similar:

• Their direct super-entities (or all of their super-entities) are already similar.

• Their sibling-entities (or all of their sibling-entities) are already similar.

• Their direct sub-entities (or all of their sub-entities) are already similar.

• All (or most) of their descendant-entities (entities in the sub-tree rooted at the entity in question) are already similar.

• All (or most) of their leaf-entities (entities, which have no sub-entity, in the sub-tree rooted at the entity in question) are already similar.

• All (or most) of entities in the paths from the root to the entities in question are already similar.

• All (or most) of relative entities to the entities in question using properties are similar.

Several approaches are proposed to measure similarity between ontologies; some of them are developed specifically for ontology alignment while some others have been developed for other domains. Maedche and al. [19] computes the dissimilarity between two taxonomies by comparing for each class the labels of their super classes and subclasses. Anchor-Prompt [20] tries to find relationships between entities using a bounded path comparison algorithm with the originality that anchor points can be provided by the users as a partial alignment. The similarity distance [21] computes the relationship among entities for a single hierarchy. The Similarity Flooding (SF) [22] compares graphs representing the schemas, looking for similarities in the graph structure. SF uses a hybrid matching algorithm based on the ideas of similarity propagation. The basic concept behind the algorithm is the similarity spreading from similar nodes to the adjacent neighbors through propagation coefficients. OLA (OWL Lite Aligner) [23] is designed with the idea of balancing the contribution of each component that compose an ontology. OLA converts definitions of distances based on all the input structures into a set of equations. The algorithm then looks for the matching between the ontologies that minimizes the overall distance between them.

IV. THE PROPOSED APPROACH

We propose an ontology approach for satellite image retrieval as shown in figure 1. The proposed system contains two main modules: ontological model merging and semantic strategic retrieval.

A. Module 1: Ontological model merging

1) Ontological modeling of the scene Scene modeling consists of creating an ontological model

describing features and spatial relations between them [24]. This process contains two steps as shown in figure 2: the treatment step and the modeling step.

In the first step, we classify the satellite image, and extract objects from these classes. In the second step, we develop an ontological model representing extracted objects and spatial relation between them: The sensor ontological model, the scene ontological model and the spatial relation ontological model.

The sensor ontological model allows representing the types of sensor according to their functioning mode. The scene ontological model allows representing objects in the satellite scene and their semantic hierarchy. The spatial relation ontological model allows representing spatial relation, position, direction and distance between objects.

Figure 1. The proposed architecture

2) Merging ontological models The developed ontological models given by the first

module can not represent all objects of the real scene. Each model gives a particular aspect of the scene. For example, a feature can be detected only by a radar sensor, or a feature

Real scene

Optic, radar,

infrared sensor

Satellites images

representing the

same scene

MODULE 1: ONTOLOGICAL MODEL

MERGING

MODULE 2: SEMANTIC

STRATEGIC IMAGE

Similar

scenes

Similar

models

Ontological

model base

Merged

ontological

model

acquired by an optic sensor is clearer than a radar sensor. In addition, there are many conflicts situations: a scene feature can have many significations. For example, a line object in a scene can be interpreted as a road in a satellite image, and a river in another image.

In order to resolve these problems and to develop a complete and reliable model, we propose a fusion algorithm allows merging these incompletes models.

Figure 2. Ontological modeling process

Figure 3. Ontological models merging

3) Probabilistic method for merging ontological models An approach for merging uncertain information is

presented in [25]. It allows merging structured reports in XML document by using fusion approach such as probabilistic method, evidence method and possibility method. Our

ontological model is formalized in OWL language. This language is based on XML/RDF. Therefore, we propose to adapt this approach in our fusion module by adding a specific attribute representing and managing uncertain information. Especially, we add the probability attribute to the object of the ontological model. For example, Scene_feature is a cultivated parcel with probability value of 80%, and forest with probability value of 20%.

<Cultivated parcel rdf:ID=" Scene_feature ">

<Probability rdf:datatype=

"&xsd;string">0.8</Probabilité>

</Cultivated parcel>

<Forest rdf:ID=" Scene_feature ">

<Probability

rdf:datatype="&xsd;string">0.2</Probability>

</Forest>

Figure 4. OWL probabilistic model

We chose to apply the probabilistic fusion method to obtain new and reliable information. This method allows representing and managing uncertain information.

4) Algorithm for merging ontological models In order to merge ontological models, we propose a fusion

algorithm: We consider two ontological models O1 and O2, and the merged model M:

For each instance in O1 and O2

If (Instance exists in O1 and not in O2) Or

(Instance exists in O2 and not in O1) Then

Add Instance to M

Else //(Instance not exists in tow models)

If (Instance has the same probability value in the two

models O1 and O2) Then

Add Instance to M

Else //(Instance has different probability value)

Apply the probabilistic method

Add the accepted Instance.

End If

End

Figure 5. Fusion algorithm

This algorithm starts by comparing the instances of the models. If an instance exists in a model but not in another, we add it in the new model. If the instances exist in two models with the same probability value, we add it in the new model. But, if the instances have different probabilistic value; we calculate the new value by applying the probability method which manages uncertain information, next, we add the accepted instance in the new model.

We consider two ontological models O1 and O2. Forest instance exists only in the model O1 and cultivated parcel exists only in O2, thus, we add these instances in the new model. Road and River instances exist in the two models with different probabilistic value. We calculate the new value using average of probabilistic values; we obtain a new ontological model that contains new probabilistic values: the value of road and river becomes 0.7 and 0.3 respectively.

Low-level

Satellite image

Intermediate-level

Image segmentation

High-level

Ontological modeling

Incompletes

ontological model

Merged model

MEDELS MERGING

<Road >

<Name>R</Name>

<Probability >0.8</Probability>

</Road>

<River >

<Name>R</Name>

<Probability >0.2</Probability>

</River>

< Forest>

<Name>F</Name>

</Forest>

<Cultivated parcel>

<Name>CP1</Name>

<Superficies> 500 </Superficies>

</ Cultivated parcel >

< Urban zone>

<Name >UZ1</Name>

<Superficies> 10 </Superficies>

</Urban zone >

Figure 6. Model O1

<Road >

<Name>R</Name>

<Probability >0.6</Probability>

</Road>

<River>

<Name>R</Name>

<Probability >0.4</Probability>

</River>

<Cultivated Parcel>

<Name>F</Name>

</Cultivated Parcel>

<Lake>

<Nom LK1</Nom>

<Superficies> 300 </Superficies>

</Lake>

< Urban zone>

<Name >UZ1</Name>

<Superficies> 10 </Superficies>

</ Urban zone >

Figure 7. Model O2

<Road >

<Name>R</Name>

<Probability >0.7</Probability> ((0.8 +0.6)/2)

</Road>

<River>

<Name>R</Name>

<Probability >0.3</Probability> ((0.2 +0.4)/2)

</River>

< Forest>

<Name>F</Name>

</Forest>

<Cultivated parcel>

<Name>CP1</Name>

<Superficies> 500 </Superficies>

</Cultivated parcel >

<Lake>

<Nom LK1</Nom>

<Superficies> 300 </Superficies>

</Lake>

< Urban zone>

<Name >UZ1</Name>

<Superficies> 10 </Superficies>

</Urban zone >

Figure 8. Model OM

B. Module 2: Semantic strategic retrieval

This module allows retrieving satellite images basing on their ontological model. The retrieval system contains a scene base composed of satellite images, and an ontological model base composed of ontological models which correspond to these satellites images.

The idea is to retrieve satellite images basing on their ontological models; two images are similar if their ontological models are similar. In order to improve the quality of retrieval system and to facilitate the retrieval process, we propose two retrieval strategies which are the opportunist strategy and the hypothetic strategy.

1) The opportunist strategy This strategy is based on atypical objects which are the

eminent and distinguished objects in the satellite image. It composed by two steps which are ontological model analysis and ontological model retrieval. The first identifies atypical objects from the merged ontological model. The second retrieves ontological models which contain atypical objects from the ontological model base. The result of this step is a set of accepted ontological models and another of rejected model.

This strategy allows reducing the ontological model base and facilitates the retrieval process.

We consider an ontological model which contains a cultivated parcel and a lake as atypical objects and adjacency as spatial relation between these objects. The result of this strategy is the set of ontological models which contains these atypical objects.

2) The hypothetic strategy

This strategy is based on the result of the opportunist

strategic and the merged ontological model. It establishes the

similarity degree for each model in the ontological model base

according to it’s conformity with the query model.

The result of this strategy is a set of ontological models

ordered by their similarity degree. The similarity between

ontological models is based on several levels such as the

lexical similarity and the structural similarity.

V. CONCLUSION

In this paper, we presented an ontology approach for semantic retrieving satellite image taking into account the representation of spatial knowledge and based on two main modules: ontological models merging and semantic strategic retrieval.

Our approach permits to overcome the obstacles of keyword-based approach. It has the potential to fully describe the semantic content of an image. Besides, it manages uncertain information and resolves conflicts situation by fusion algorithm that allows developing an ideal and reliable ontological model describing satellite scene.

This work contributes the methodologies for the semantic representation, and the retrieval processes using ontology approach.

REFERENCES

[1] N. Ruan, N. Huang, W. Hong, “Semantic-Based Image Retrieval in Remote Sensing Archive: An Ontology Approach”, Geoscience and Remote Sensing Symposium, 2006. IGARSS 2006, pages 2903-2906.

[2] E. Hyvönen, A. Styrman, and S. Saarela. “Ontology-based Image Retrieval”, HIIT Publications Number 2002-03, pp.15-27.

[3] Y. Rui, T.S. Huang, “Optimizing learning in image retrieval”, Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, June 2000, pp.1236-1243.

[4] Y. Rui, T.S. Huang, S.-F. Chang, Image retrieval: current techniques, promising directions, and open issues, J. Visual Commun. Image Representation, 1999, pp.39–62.

[5] A. Vailaya, M.A.T. Figueiredo, A.K. Jain, H.J. Zhang, Image classification for content-based indexing, IEEE Trans. Image Process, 2001, pp.117–130.

[6] N. Ruan, N. Huang, W. Hong, “Semantic-Based Image Retrieval in Remote Sensing Archive: An Ontology Approach”, Geoscience and Remote Sensing Symposium, 2006. IGARSS 2006, pp. 2903-2906.

[7] T. R. Gruber, Toward Principles for the Design of Ontologies Used for Knowledge Sharing, International Journal Human-Computer Studies 43, pages 907-928. 1993.

[8] W. Zheng, Y. Ouyang, J. Ford, Fillia S. Makedon “Ontology-based Image Retrieval”, WSEAS MMACTEE-WAMUS-NOLASC 2003, Vouliagmeni, Athens, Greece, December 29-31, 2003

[9] Y. Liua, D. Zhanga, G. Lua,W.Y. Mab, “A survey of content-based image retrieval with high-level semantics”, The journal of the recognition society, pp. 262-282, 2007

[10] Y. Chen, J.Z.Wang, R.Krovetz, “An unsupervised learning approach to content-based image retrieval”, IEEE Proceedings of the International Symposium on Signal Processing and its Applications, July 2003, pp. 197–200.

[11] Y. Rui, T.S. Huang, M. Ortega, S. Mehrotra, “Relevance feedback: a power tool for interactive content-based image retrieval”, IEEE Trans. Circuits Video Technol. 8 (5) (1998) 644–655.

[12] Y. Rui, T.S. Huang, “Optimizing learning in image retrieval”, Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, June 2000, pp.1236–1243.

[13] C.P. Town, D. Sinclair, “Content-based image retrieval using semantic visual categories”, Society for Manufacturing Engineers, Technical Report MV01-211, 2001.

[14] N. Vasconcelos, A. Lippman, “Library-based coding: a representation for efficient video compression and retrieval”, Proceedings of the Data Compression Conference (DCC97), March 1997, pp. 121–130.

[15] R. Shi, H. Feng, T.-S. Chua, C.-H. Lee, “An adaptive image content representation and segmentation approach to automatic image annotation, International Conference on Image and Video Retrieval” (CIVR), 2004, pp. 545–554.

[16] V. Mezaris, I. Kompatsiaris, M.G. Strintzis, “An ontology approach to object-based image retrieval”, Proceedings of the ICIP, vol. II, 2003, pp. 511–514.

[17] W. Zheng, Y. Ouyang, J. Ford, Fillia S. Makedon “Ontology-based Image Retrieval”, WSEAS MMACTEE-WAMUS-NOLASC 2003, Vouliagmeni, Athens, Greece, December 29-31, 2003

[18] B. B. Hariri, H. Abolhassani, A. Khodaei, "A New Structural Similarity Measure for Ontology Alignment", in the Proceedings of 2006 International Conference on Semantic Web and Web Services (SWWS'2006), Las Vegas, USA, June 2006.

[19] A. Maedche, S. Staab, "Measuring Similarity between Ontologies", in the Proceedings of the European Conference on Knowledge Acquisition and Management EKAW-2002, Madrid, Spain, October 1-4, pp. 251-263, 2002

[20] Natalya Noy and Mark Musen. “Anchor-PROMPT: Using non-local context for semantic matching”. In Proc. IJCAI 2001 workshop on ontology and information sharing, Seattle (WA US), pp.63–70, 2001.

[21] J. Zhong, H. Zhu, Y. Li, and Y. Yu, “Conceptual graph matching for semantic search,” in Proceedings of Conceptual Structures: Integration and Interfaces (ICCS-2002), 2002, pp. 92–106.

[22] S. Melnik, H. Garcia-Molina, and E. Rahm, “A versatile graph matching algorithm,” in Proceedings of ICDE, 2002.

[23] J. Euzenat and P. Valtchev, “An integrative proximity measure for ontology alignment,” in Proceedings of Semantic Integration workshop at ISWC, 2003.

[24] H. Sun, S. Li, W. Li, Z. Ming, and S. Cai, "Semantic-Based Retrieval of Remote Sensing Images in a Grid Environment", IEEE geoscience and remote sensing letters, vol. 2, no. 4, pages 440- 444, 2005

[25] A. Hunter and R. Summerton. “Fusion rules for context-dependent aggregation of structured news reports”. Journal of Applied Non-Classical Logics, pp. 329-366, 2004.