Post on 29-Mar-2015
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
IMAGE RETRIEVAL WITH RELEVANCE FEEDBACK
Hayati CAMOzge CAVUS
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Outline
Question: What is Content Based Image Retrieval?
Recent Work on CBIR
Our Approach
Evaluation
Summary
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
CBIR
Large quantities of multimedia data is used in archives
Traditional way: Using keywords in IR(Image Retrieval)
Problems: Annotation is very difficult Keywords may be insufficient to represent the contents
of the images Keywords are user dependent
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
CBIR
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Recent Work
Extracting global low-level features (texture or color) from images
Problem: limited in capability of deriving higher semantic meanings of the images
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Recent Work
Partitioning images into nonoverlapping grid cell Problem: Grids are not meaningful regions
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Our Approach
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Our Approach
Image Segmentation
Codebook Construction
Image Representation by using Posterior Class Probability Values
Content Based Image Retrieval with Relevance Feedback
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Dataset
TRECVID 2005 dataset
29832 video shots
Contain approximately 20 different classes exp: mountain, seaside, urban, sports …
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Segmentation
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Segmentation
Cluster the RGB color values of the pixels by k-means
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Segmentation
Smooth the regions by combined classifier approach
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Codebook Construction
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Representation
Calculate region k=1000 bins histograms for each image
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Representation
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Image Representation
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Relevance Feedback
At the first iteration images are ranked by distances to the query image
After each iteration user labels the images as relevant and irrelevant
The new result are retrieved according to the user feedback
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Content Based Image Retrieval
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Relevance Feedback
Assign a weight value w to each class probability value
The weights are assigned uniformly in the first iteration.
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Relevance Feedback
Given two images: Distances between the corresponding
probability terms are computed di = distance between the ith probability values of
two images where i=1, …, c
These distances are combined as d = ∑ wi di
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Relevance Feedback
Given the positive and negative examples, for a probability term being significant for a particular query:
Distances for the corresponding probability values for relevant images must usually be similar (hence, a small variance),
Distances between the probability values for relevant images and irrelevant images must usually be different (hence, a large variance).
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Relevance Feedback
Weights are computed as:
std(distances of ith probability term between relevant and irrelevant images)
Wi =
std(distances of ith probability term between relevant images)
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Evaluation
Yao’s formula for cluster validation ntr > nt
Why do we need this? Better Clustering -> Better Probability Values ->
Better Retrieval
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Evaluation
Precision-Recall
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
Summary
Steps of Our Approach Image Segmentation Codebook Construction Image Representation by probabilities CBIR with Relevance Feedback
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus
THANK YOU