by P. Muneesawang and L. Guan IEEE Transactions...

45
by P. Muneesawang and L. Guan IEEE Transactions on Multimedia, 2004 Joseph Lee

Transcript of by P. Muneesawang and L. Guan IEEE Transactions...

Page 1: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

by P. Muneesawang and L. GuanIEEE Transactions on Multimedia, 2004

Joseph Lee

Page 2: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

1. Backgroundg

2. General Framework

d l3. Previous Models

4. Proposed RBF Model

5. Learning Strategy

E perimental Res lts6. Experimental Results

7. Conclusion

Page 3: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Content-based Image Retrieval (CBIR)◦ use colors, shapes, textures → rely on image itself◦ without image content → rely on metadata

Page 4: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Content-based Image Retrieval (CBIR)◦ use colors, shapes, textures → rely on image itself◦ without image content → rely on metadata

Relevance Feedback◦ User refines the result by marking images as

relevant or non-relevant and repeats search.

Page 5: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Image Similarity

Page 6: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Image Similarity

Page 7: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Examples of Image Similarity Problems

MARS-1 RBF

Page 8: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case

Page 9: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case – User Query

Page 10: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Ideal Case – Similarity Measure

Page 11: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case

Page 12: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case – Inaccurate Query

Page 13: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Real Case – Inaccurate Query

Page 14: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

We should refine Query

QueryQuery

Page 15: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

We should refine Query & Metric

Metric

QueryQuery

Page 16: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Query Reformulation Model

◦ Relevance feedback → learn query representation

Adaptive Metric Model

◦ Relevance feedback → learn similarity function

Page 17: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 18: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 19: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 20: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 21: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning user perception

◦ MARS-1◦ MARS 1

◦ MARS-2

◦ OPT-RF

Assumption◦ same distance gives same degree of similarity

Page 22: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

CBIR◦ online learning◦ two-class problem

Page 23: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

CBIR◦ online learning◦ two-class problem

Page 24: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Example Vectors

Page 25: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Voronoi Vectors

Page 26: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Voronoi Cells

Page 27: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 28: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 29: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 30: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Learning Vector Quantization

Page 31: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 32: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 33: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 34: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 1)

Page 35: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Modified LVQ (Model 2)

Page 36: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 37: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 38: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 39: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Effects of Positive and Negative Learning

◦ Relevant samples → common interest

◦ Non-relevant samples → specific interest

Page 40: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

Page 41: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

Page 42: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Selection of RBF Width

◦◦

Page 43: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Average Precision Rate (%)

Page 44: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

Non-linear model for similarity evaluation

Learning from positive and negative samples

Page 45: by P. Muneesawang and L. Guan IEEE Transactions ...courses.cs.tamu.edu/rgutier/cpsc636_s10/student...`Content-based Image Retrieval (CBIR) use colors, shapes, textures → rely on

http://videolectures.net/minh_hoai_nguyen/http://www mqasem net/vectorquantization/vq htmlhttp://www.mqasem.net/vectorquantization/vq.html