SVCL

Automatic detection of object based Region-of-Interest for image

compression

Sunhyoung Han

SVCL

Transmission in erroneous channel

Basic Motivation

Spatially differentSuper resolution

ConstraintsLimited Resources &Channel Errors

SVCL

Basic motivation

By having information about importance of regionsOne can wisely use the limited resources

SVCL

User-adaptive Coder

v

• visual concepts of interest can

be anything

• main idea:

• let users define a universe

of objects of interest

• train saliency detector for

each object

• e.g. regions of “people”,

“the Capitol”, “trees”, etc.

SVCL

User Adaptive Coder

query providedby user

traindetector

current trainingsets

SVCL

User-adaptive coder

• user-adaptive coder:– detector should be generic enough to handle large

numbers of object categories

– training needs to be reasonably fast (including example preparation time)

“face” “lamp” “car”

SVCL

User-adaptive coder• proposed detector

– top-down object detector (object category specified by user)

– focus on weak supervision instead of highly accurate localization

– composed of saliency detection and saliency validation

– discriminant saliency:

saliencyfilters

training

FIND best features

SVCL

Discriminant Saliency

• start from a universe of classes (e.g. “faces”, “trees”,

“cars”, etc.)

• design a dictionary of features: e.g. linear

combinations of DCT coefficients at multiple scales

• salient features: those that best distinguish the object

class of interest from random background scenes.

• salient regions are the regions of the image where

these detectors have strong response

• see [Gao & Vasconcelos, NIPS, 2004].

SVCL

Top-down Discriminant Saliency Model

Scale Selection

W j

WTA

Faces Discriminant Feature

Selection

Salient Features

Background

Saliency Map

Original Feature Set

Malik-Perona pre-attentive perception model

ZXIk kk

;maxarg*

SVCL

• saliency detector

• salient point sali:– magnitude i

– location li– scale si

• saliency map approximated by a Gaussian mixture

Saliency representation

image saliency map salient points

Probability map

SVCL

Saliency validation• saliency detection:

– due to limited feature dictionary and/or limited training set

– coarse detection of object class of interest

• need to eliminate false positives

• saliency validation:– geometric consistency– reject salient points whose

spatial configuration is inconsistent with training examples

original Image

saliency map for ‘street sign’

example of saliency map

SVCL

Saliency validation• learning a geometric model of salient point con-

figuration• two components:

- image alignment

• model:- classify pointsinto

• true positives

- configuration model

• false positives- model eachas Gaussian

SVCL

Saliency validation

• model: two classes of points Y={0,1}– Y=1 true positive– Y=0 false positive

• saliency map: mixture of true and false positive saliency distributions

• each distribution approximated by aGaussian

SVCL

• this is a two class clustering problem– can be solved by expectation-maximization

• graphical model

• non-standard issues– we start from distributions, not points– alignment does not depend on false

negatives

Saliency validation

E-stepM-step

Y X

L~uniform

Y~Bernoulli (1)

C|Y=i~multinomial (i)

X|Y=i,L=l,S=s,~G(x, l-, )

L,S

C

SVCL

Saliency ValidationFor K training examples (# of saliency point is Nk for kth example) Missing data Y= j,

j {1,0}∈ Parameters j (probability for class j)

∑j (Covariance for class )

k (displacement for kth example)

For robust update

DERIVATION DETAILS

SVCL

Saliency Validation• visualization of EM algorithm

Saliency detection result

Init saliency points overlapped over 40 samples

Visualized variance ∑1Overlapped points classified as ‘’object’’

Overlapped points classified as ‘’noise’’

Visualized variance ∑0

SVCL

Saliency Validation

• examples of classified Points

• in summary, during training we learn– discriminant features– The “right” configuration of salient points

Examples of classified saliency points White if hij1>hij

0 Black otherwise

SVCL

Region of interest detection

• find image window that best matches the learned configuration

• mathematically: - find location p where the posterior probability of the object class is the largest

SVCL

Region of interest detection• by Bayes rule

– Posterior Likelihood x Prior

– likelihood is given by matching saliencies within the window

& the model

- prior measuresthe saliency massinside window ?

?

likelihoodPrior

SVCL

Region of Interest Detection

• given the model– the likelihood, under it, of

a set of points drawn from the observed saliency distribution is

– and the optimal location is given by

Prior for location PWith saliency detector

DERIVATION DETAILS

Measure configuration matching

SVCL

2. Determine scale(shape) of ROI mask Observation(∑*) from data and

prior(∑1) from training data are used

3. Thresholds PY|X,P(1|x,p*) to get binary ROI mask

Region of Interest Detection

** Once the center point is known the assignment of each point is given by

The observed configuration for Y=1 isx

∑1∑*

SVCL

Region of Interest Detection

Saliency detection (for statue of liberty)Probability map (saliency only)

Probability map (with configuration info.) ROI mask

• Example of ROI Detection

SVCL

Evaluation

• Using CalTech “Face” database &UIUC “Car side” database

• Evaluate robustness of learning– Dedicated Training set vs. Web Training set

• Evaluation Metric– ROC area curve

– PSNR gain for ROI coding vs. normal coding

Number of positive example: 550

Number of positive example: 100

SVCL

Evaluation

• ROC area curve

False Positive

Tru

e P

ositi

ve

False PositiveT

rue

Pos

itive

“Car” “Face”

SVCL

Evaluation

• PSNR performance comparison

“Car” “Face”Bit Per Pixel

PS

NR

Bit Per PixelP

SN

R

14.3% bits can be saved even with web train uniform casefor the same image quality

SVCL

Result Examples

SVCL

ResultComparison of needed bits to get the same PSNR (30 dB) for ROI

Maximally, ¼ bits are enough to get the same quality for ROI area

SVCL

Result Examples

Normal coding ROI coding

SVCL

SVCL

EM derivation• Want to fit lower level observation

• For a virtual sample X = {Xik|i=1, …, Nk and k=1, …, K} with the size of Mik=ik*N, likelihood becomes

• For complete set the log likelihood becomes

SVCL

EM derivation

Maximization in the m-step is carried out by maximizing the Lagrangian

SVCL

ROI Detection

For one sample point x1

For samples having distribution of

SVCL

ROI Detection

Therefore,