A Non-parametric Unsupervised Approach for Content Based Image

Retrieval and Clustering

Technical University

of Crete

Authors:Konstantinos Makantasis (Technical University of Crete) Anastasios Doulamis (Technical University of Crete)Nikolaos Doulamis (Cyprus University of Technology)

4th Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams (ARTEMIS 2013)

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

OUTLINE

System Overview

Web Query

Image Retrieval

Outliers Removal

Image Clustering

Results

SYSTEM OVERVIEW

Web Query Image Retrieval

Image Clustering

Fully Automatic CBIR4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

WEB QUERY – IMAGE RETRIEVAL

Query Flickr Image Database

Flickr API in Python

Query keywords are associated with images’ title

Retrieval Retrieved Set: hundreds to

thousands of photos

Retrieved Content: cultural heritage monuments 4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS

(ARTEMIS 2013)

OUTLIERS REMOVAL (1)

Retrieved dataset contains many outliers (visually dissimilar images) Inconsistent human generated

tags

Uninformative machine generated tags

Absence of camera generated meta-data

(Retrieved dataset for “Porta Nigra”)4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS

(ARTEMIS 2013)

OUTLIERS REMOVAL (2)

STEP 1: Visual information encoding Global descriptors SIFT keypoints Each image is

represented by a matrix

STEP 2: Two-way image matching Matches from image to image

Matches from image to image

Final matches between images and

STEP 3: Similarity metric definition For images and

STEP 4: Outliers removal through DBSCAN The goal is to assign to one class

visually similar images and denote visually dissimilar as outliers

However, DBSCAN requires the definition of minimum number of samples per class () and “area” of classes ()

DBSCAN tuning mechanism4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

OUTLIERS REMOVAL (3)

DBCAN Tuning Mechanism

For a given define function that maps the minimum distance that required for images to have at least neighbors

Find best trade-off point of

Repeat for where and represent the 10% and 90% of retrieved set’s size

Choose and associated with the trade-off point that maximizes “distance from line” 4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS

(ARTEMIS 2013)

OUTLIERS REMOVAL (4)

Visually Similar Images

Outliers

Initial Retrieved

Set

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

SPECTRAL CLUSTERING

Similarity Matrix is already computed

Graph where is the set of images and stands for the similarity between them Goal: partition to sub-graphs that contain visually similar images

Random walks on

From compute diagonal degree matrix , with

Compute Laplacian matrix as and normalized Laplacian as

Multiplicity of zero eigenvalues of shown the number of connected components in

Eigengap criterion: set number of clusters for spectral clustering equal to (we used clusters to further eliminate outliers by removing the smallest cluster)

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

EXPERIMENTAL RESULTS (1)

Queries for 3 cultural heritage monuments Porta Nigra

Parthenon, Athens

Descobrimentos

Initial Set Outliers Final Set Number of Clusters

Porta Nigra 500 227 (45.4%) 273 (54.6%) 2

Parthenon 500 321 (64.2%) 179 (35.8%) 1

Descobrimentos

500 104 (20.8%) 396 (79.2%) 2

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

EXPERIMENTAL RESULTS (2)

Porta Nigra

Initial retrieve

d Set

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

EXPERIMENTAL RESULTS (3)

Parthenon

Initial retrieve

d Set

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

EXPERIMENTAL RESULTS (4)

Descobrimentos

Initial retrieve

d Set

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

CONTRIBUTION

Fully Automatic and Non-parametric algorithm

Handle digital and analog “born” images Handle historic images

Based only on visual information

No a priori knowledge of the dataset

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)

QUESTIONS

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS 2013)