Salient Point Detection

OutlineIntroduction

Salient Point DetectionChallenges

Results

Salient Point Detection

Tyler Karrels

Department of Electrical and Computer EngineeringUniversity of Wisconsin - Madison

April 22, 2009

Tyler Karrels Salient Point Detection

OutlineIntroduction


Results

1 IntroductionDefining SaliencyThe Saliency ExperienceHuman Visual System (HVS)Psychology of PerceptionPrevious Work

2 Salient Point DetectionMathematical FrameworkFeaturesClusteringSaliency

3 Challenges

4 Results


OutlineIntroduction


Results

Defining SaliencyThe Saliency ExperienceHuman Visual System (HVS)Psychology of PerceptionPrevious Work

What is saliency?

Definitions1 SALIENT: “strikingly conspicuous; prominent; noticeable”

American Heritage Dictionary

2 VISUAL SALIENCY: “. . . the distinct subjective perceptualquality which makes some items in the world stand out fromtheir neighbors and immediately grab our attention.”Laurent Itti, Scholarpedia


OutlineIntroduction


Results



OutlineIntroduction


Results


Popout Effect 1


OutlineIntroduction


Results


Popout Effect 2


OutlineIntroduction


Results


What is salient?


OutlineIntroduction


Results


Conjunction Test 1


OutlineIntroduction


Results


Conjunction Test 2


OutlineIntroduction


Results


FORGET(o.0)


OutlineIntroduction


Results


Conjunction Test 3


OutlineIntroduction


Results


What is salient?


OutlineIntroduction


Results


Phase Transition


OutlineIntroduction


Results


What is salient?


OutlineIntroduction


Results


The Eye: Physiology

Peripheral Vision, Wikipedia

Foveal Vision: attended location; line of sight

Peripheral Vision: surrounding locations

1-1 photoreceptor to ganglion in Foveal Vision

many-1 for Peripheral Vision (low res. compression)

50% Fovea + 50% Peripheral = 100% Data Sent to Brain!Tyler Karrels Salient Point Detection

OutlineIntroduction


Results


The Eye: Feature Detector

Bottom-Up Processing

Detects low-level features in parallel, e.g. color, orientation,contrast, . . .

Occurs before brain perceives data

Feature detectors compete to direct attention to salientlocations

How do they compete, communicate, and cooperate?

Top-Down Processing

The brain’s expectations guide attention


OutlineIntroduction


Results


Helmholtz Principle

“. . . whenever some large deviation from randomness occurs, astructure is perceived.”Desolneux, From Gestalt Theory to Image Analysis: A Probabilistic Approach


OutlineIntroduction


Results


Gestalt Laws

Perceptual Grouping Principles

Closure

Similarity

Proximity

Symmetry

Continuity

Common Fate


OutlineIntroduction


Results


Sha’asua [4]

Continuity & Closure

Detect edges

Form contours: Connectedges

Maximize lengthMinimize total curvature

Longer contours, moresalient

Disregards other gestalt laws& image features


OutlineIntroduction


Results


Itti [2]

Itti’s BiologicalSaliency Map Model

Proximity & Similarity

Multiple scales encodeproximity

Center-surround

Measures local contrastFine scale ‘center’ minuscourse scale ‘surround’

Normalization encodessimilarity

Feature map combinationdetermines success


OutlineIntroduction


Results

Mathematical FrameworkFeaturesClusteringSaliency

Salient Point Detection

Not constrained to be biologically plausible

Not image-processing; clustering/outlier detection in Rd

Pixels are salient, not objects or regions

Challenges

When is something salient? When not?

Can we quantify saliency?

Can we relate computer/human performance?

Can we improve on previous methods? Will we?


OutlineIntroduction


Results


Framework

Data

Pixels {Xi}ni=1

Xi = (yi , xi , ri , gi , bi )

Video? Include time coordinate: X = (y , x , r , g , b, t)

Feature Space

Feature Maps {Fj}mj=1

Vi = (yi , xi , ri , gi , bi ,Fi1, . . . ,Fim)

Feature Vectors {Vi}ni=1

Feature Space Vi ∈ [0, 1]d


OutlineIntroduction


Results


The Process

Easy As 1,2,3?

1 Create feature maps

2 Cluster points in Rd

3 Choose the salient cluster


OutlineIntroduction


Results


Proposed Features

Salient Scenarios1 Intensity

2 Color

3 Orientation

4 Size

5 Location

Feature Maps {Fj}mj=1

1 Intensity

2 Colors [Red, Green, Blue]

3 Edge orientations[0 ◦, 45 ◦, 90 ◦, 135 ◦]

4 Scale Description

5 Pixel Location

How does our data representation affect performance?


OutlineIntroduction


Results


2-D Example

Do we really need 2 dimensions? Is 1 sufficient?


OutlineIntroduction


Results


1-D Example

Background pixels: noorientation?

Horizontal pixels: 0 ◦ or180 ◦?


OutlineIntroduction


Results


Feature Subset Selection

Choosing Salient Dimensions

Interpret variance

Projections onto feature subspaces

Projections

Pr[Vi = (v1, . . . , vd)] empirical distribution

I = {i1, . . . , il} index set

Project onto subset I , induce Pr[Vi = (vi1 , . . . , vil )]

Minimize the KL Divergence between Empirical and Subsetdistributions


OutlineIntroduction


Results


Vertical, Horizontal, Intensity, Red Example

Notice Pr[Red ,VerticalBar ] ≈ Pr[RedBar ]


OutlineIntroduction


Results


Clustering in Feature Space

Gaussian Mixture Clustering

Figueiredo’s algorithm determines best number of clusters [1]

Fits distribution in feature space to a mixture of Gaussians

Uses EM algorithm, results vary depending on initialization

Subspace Clustering

Ma’s algorithm provides distortion parameter ε [3]

Based on rate distortion theory

Deterministic, same results every time

Tight cluster requires low rate

Additional clusters increase rate


OutlineIntroduction


Results


Subspace Clustering

Large ε→ few clusters, Small ε→ many clusters

As ε varies large → small, salient clusters emerge


OutlineIntroduction


Results


Salient Clusters

How to determine a cluster’s saliency?

Compare clusters for relative notion of saliency

Relative cluster size and variance

Small relative size indicates uniquenessSmall relative variance indicates similarity

‘Distant’ clusters have less in common.

Centroid DistanceMahanalobis Distance

Outlier detection methods


OutlineIntroduction


Results

Existence of Salient Points


OutlineIntroduction


Results

Quantification of Salient Points


OutlineIntroduction


Results

Performance


OutlineIntroduction


Results

Orientation Test Results


OutlineIntroduction


Results

Google Tyler Karrels - Click Homepage - Click Papers


OutlineIntroduction


Results

References

M. A. T. Figueiredo and A. K. Jain.

Unsupervised learning of finite mixture models.IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 381–396, 2002.

L. Itti and C. Koch.

Feature combination strategies for saliency-based visual attention systems.Journal of Electronic Imaging, 10:161, 2001.

Y. Ma, H. Derksen, W. Hong, and J. Wright.

Segmentation of multivariate mixed data via lossy data coding and compression.IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1546–1562, 2007.

A. Sha’asua.

Structural saliency: The detection of globally salient structures using a locally connected network, 1988.ID: 1.


Salient Point Detection

Documents

Transcript of Salient Point Detection