Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON...
-
Upload
edwina-joseph -
Category
Documents
-
view
220 -
download
1
Transcript of Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON...
![Page 1: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/1.jpg)
1
Region-Based Saliency Detection and Its Application in Object Recognition
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24
NO. 5, MAY 2014
Zhixiang Ren, Shenghua Gao, liang-Tien Chia, and Ivor Wai-Hung Tsang
![Page 2: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/2.jpg)
2
Overview
• Introduction
• Related Work
• Proposed Method of Saliency
• Experiments For Saliency Detection
• Conclusion
![Page 3: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/3.jpg)
3
Introduction
![Page 4: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/4.jpg)
4
Introduction
• Visual Saliency
Measure to what extent a region attracts human attention.
• Potential Application
Adaptive compression, image retargeting, object detection
![Page 5: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/5.jpg)
5
Introduction (cont.)
• Many saliency detection algorithms (pixel-grid) have been proposed
[36]-[38], [57], [68]
• Drawbacks
• Perform poorly in the images with large salient regions
• Suffer from the messy background, e.g. natural scenes.
![Page 6: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/6.jpg)
6
Introduction (cont.)
• [17], [18] suggest that early feature like color, contrast, and orientation indirectly affect human attention
Human is attracted by objects not by individual pixels
• It is natural to work with those perceptually meaningful image regions in saliency detection
Concept of superpixel [54]
![Page 7: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/7.jpg)
7
Introduction (cont.)
• Proposed work applies two existing techniques to improve saliency detection
Superpixel representation – Used to represent the input image
PageRank algorithm – Applied to propagate saliency among similar clusters and refine saliency map
![Page 8: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/8.jpg)
8
Related Work
![Page 9: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/9.jpg)
9
Related Work
• Saliency detection methods can be divided into two categories
Top-down method : Task-dependent and based on prior knowledge about the object and their interrelations
Bottom-up method : Hypothesis for saliency is that salient stimulus is distinct from its surrounding stimuli. (contrast)
• For bottom-up method, research usually focus on identifying those regions with high contrast.
![Page 10: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/10.jpg)
10
Related Work (cont.)
• [38] proposed to determine the contrast by DoG
• [57] measured the likeness of a pixel to its surroundings by the local regression kernels
• [1] measured the saliency of each pixel by the difference between the feature of each pixel and mean of the whole image.
• [68] measured the global contrast with all the other pixels.
• [30] model both local and global contrast by taking the positional distance into account
• Most of approach represent the input image in pixel-grid manner, and these method may failed to detect the homogeneous and quite large salient objects
![Page 11: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/11.jpg)
11
Related Work (cont.)
![Page 12: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/12.jpg)
12
Proposed Method
![Page 13: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/13.jpg)
13
Proposed Method
• Superpixel Extraction and Clustering
• Salient Region Detection
• Saliency Refinement With Propagation
![Page 14: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/14.jpg)
14
Superpixel Extraction
• Given an input image, mean shift algorithm[13] will be performed in color space to extract superpixels.
• In mean shift algorithm, , , and are needed.
Spatial Radius
Range Radius
Minimum Point Density
Maximum range radius
Average color variance
Color variance
![Page 15: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/15.jpg)
15
Superpixel Extraction (cont.)
• Mean shift
Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function.
Scale parameter
![Page 16: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/16.jpg)
16
Superpixel Extraction (cont.)
m(x)
![Page 17: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/17.jpg)
17
Superpixel Clustering
• After mean shift, every superpixel will obtain a unique RGB color
• GMM is introduced to cluster superpixels in RGB color space
• The RGB value of this superpixel will be set as a 3-D vector to represent the superpixel during GMMR 139
G 160
B 127
![Page 18: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/18.jpg)
18
Superpixel Clustering (cont.)
• K-means is used to initialize the GMM
• Expectation maximization algorithm is used to train GMM parameter [5]
• The probability of the th superpixel belong to the th cluster
![Page 19: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/19.jpg)
19
Superpixel Clustering (cont.)
![Page 20: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/20.jpg)
20
Superpixel Clustering (cont.)
• Mixture density
Assume the set of N training samples is drawn from a mixture of models
![Page 21: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/21.jpg)
21
Superpixel Clustering (cont.)
• How to estimate and
EM algorithm
![Page 22: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/22.jpg)
22
Superpixel Clustering (cont.)
![Page 23: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/23.jpg)
23
Superpixel Clustering (cont.)
![Page 24: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/24.jpg)
24
Superpixel Clustering (cont.)
![Page 25: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/25.jpg)
25
Superpixel Clustering (cont.)
![Page 26: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/26.jpg)
26
Salient Region Detection
• Idea : background has larger spread in spatial domain
• i.e., the more compact the clusters are spread, the more salient they will be
• [32] proposed compactness metric to evaluate the spread of cluster
• Inter-cluster distances defined as
Cluster Spatial Center
![Page 27: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/27.jpg)
27
Refinement With Propagation
• In some situation, the perceptually meaningful regions are less than the cluster number.
• That is, some regions, which should belong to one cluster, will be grouped into several clusters.
R 139
G 160
B 127
![Page 28: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/28.jpg)
28
Refinement With Propagation (cont.)
• If one cluster is over-segmented into several subclusters, the compactness may be highly distorted.
• Thus PageRank algorithm is proposed to propagate saliency between similar clusters.
• Original PageRank algorithm
• Question : How the original PageRank come from?
![Page 29: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/29.jpg)
29
Refinement With Propagation (cont.)
• Idea : A page linked by many pages with high PageRank receives high rank as well.
• Modified algorithm
![Page 30: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/30.jpg)
30
Refinement With Propagation (cont.)
![Page 31: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/31.jpg)
31
Experiment Result
![Page 32: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/32.jpg)
32
Experimental DataSet and Compared Method
• Dataset
EPFL dataset [1], CMU dataset [4], MSRA dataset [46] ,Itti’s method (ITTI) [38]
• Method
Spectral residual method (SR) [37]
Graph-based saliency method (GB) [36]
Frequency-tuned method (FT) [37]
Method based on color and orientation distributions (COD) [32]
Region contrast method (RC) [12] (Region-based)
Context-based method (CB) [39] (Region-based)
![Page 33: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/33.jpg)
33
Experiment Result
![Page 34: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/34.jpg)
34
Experiment Result (cont.)
• Linear Correlation Coefficient
![Page 35: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/35.jpg)
35
Experiment Result (cont.)
![Page 36: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/36.jpg)
36
Conclusion
![Page 37: Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,](https://reader030.fdocuments.net/reader030/viewer/2022032722/56649f4e5503460f94c6f6b8/html5/thumbnails/37.jpg)
37
Conclusion
• The paper proposes a promising saliency detection approach, which can generate accurate saliency maps with well-defined object boundary.
• Mean shift, GMM are used to extract meaningful superpixel.
• Saliency value is refined as well with a modified PageRank algorithm.