MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

MultisensoryPerception

Alex Teichman

August 26, 2009

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

The Object Recognition Problem

How much does this look like a hand?

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

The Object Recognition Problem

Where is the coke can?

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

The Object Recognition Problem

What is the object in the box?

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Some Difficulties Among Many

Labels are painful to acquire.

Generalization to data not in your training set can bearbitrarily hard.

Descriptors are generally suited to a particular task.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Some Difficulties Among Many

Labels are painful to acquire.

Generalization to data not in your training set can bearbitrarily hard.

Descriptors are generally suited to a particular task.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Some Difficulties Among Many

Labels are painful to acquire.

Generalization to data not in your training set can bearbitrarily hard.

Descriptors are generally suited to a particular task.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Intuition

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Intuition

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Math

Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.

minH

∑m

exp(−ymH(~xm)) (1)

H(~x) =∑

t

ht(~x) (2)

ht(~x) =

{at if dist(~x , ~xt) < θt

0 otherwise(3)

Stagewise optimization:minht+1

∑m exp (−ym (H(~xm) + ht+1(~xm)))

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Math

Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.

minH

∑m

exp(−ymH(~xm)) (1)

H(~x) =∑

t

ht(~x) (2)

ht(~x) =

{at if dist(~x , ~xt) < θt

0 otherwise(3)

Stagewise optimization:minht+1

∑m exp (−ym (H(~xm) + ht+1(~xm)))

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Math

Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.

minH

∑m

exp(−ymH(~xm)) (1)

H(~x) =∑

t

ht(~x) (2)

ht(~x) =

{at if dist(~x , ~xt) < θt

0 otherwise(3)

Stagewise optimization:minht+1

∑m exp (−ym (H(~xm) + ht+1(~xm)))

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Boosting - Math

Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.

minH

∑m

exp(−ymH(~xm)) (1)

H(~x) =∑

t

ht(~x) (2)

ht(~x) =

{at if dist(~x , ~xt) < θt

0 otherwise(3)

Stagewise optimization:minht+1

∑m exp (−ym (H(~xm) + ht+1(~xm)))

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Image Descriptors Library

Goal: Plug-and-play image descriptors.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Image Descriptors Library

See descriptors 2d package

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Hand Detection

Histogram of Oriented Gradient descriptors (3 differentparameter settings)

Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)

1 hour to train.

Collaboration with Romain and Alex S.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Hand Detection

Histogram of Oriented Gradient descriptors (3 differentparameter settings)

Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)

1 hour to train.

Collaboration with Romain and Alex S.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Hand Detection

Histogram of Oriented Gradient descriptors (3 differentparameter settings)

Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)

1 hour to train.

Collaboration with Romain and Alex S.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Hand Detection

Histogram of Oriented Gradient descriptors (3 differentparameter settings)

Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)

1 hour to train.

Collaboration with Romain and Alex S.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Video Label Propagation

Label a video given just a few frame labels.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Mechanical Turk Feedback Loop

Implementing the feedback loop of the hands training methodologyon mechanical turk with a bottles dataset.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Online Learning

mina

∑m

exp(−ymH(~xm))

Can solve this for all weak classifier responsessimultaneously.Constant space for any number of training examples.

Maintain the gradient and Hessian as a stream of labels comesinto the classifier:

g =1

M

∑m

−ymIm

H =1

M

∑m

ImITm

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Moving Forward

Labels are painful to acquire.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Generalization to data not in your training set can bearbitrarily hard.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Descriptors are generally suited to a particular task.

Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Moving Forward

Labels are painful to acquire.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Generalization to data not in your training set can bearbitrarily hard.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Descriptors are generally suited to a particular task.

Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Moving Forward

Labels are painful to acquire.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Generalization to data not in your training set can bearbitrarily hard.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Descriptors are generally suited to a particular task.

Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Moving Forward

Labels are painful to acquire.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Generalization to data not in your training set can bearbitrarily hard.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Descriptors are generally suited to a particular task.

Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Moving Forward

Labels are painful to acquire.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Generalization to data not in your training set can bearbitrarily hard.

Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.

Descriptors are generally suited to a particular task.

Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.

MultisensoryPerception

AlexTeichman

The ObjectRecognitionProblem

Boosting

ImageDescriptorsLibrary

Applications

MovingForward

Thanks!