12th International Fall WorkshopVISION, MODELING, AND VISUALIZATION 2007November 7-9, 2007Saarbrücken, Germany

Estimating Natural Activity by Fitting 3D Models via Learned Objective Functions

1Faculty of Science and Engineering, Waseda University, Tokyo 169-8555, Japan2Institut für Informatik, Technische Universität München, 85748 Garching, Germany

3Kognitive Neuroinformatik, Universität Bremen, 28359 Bremen, Germany

Matthias Wimmer1, Christoph Mayer2,

Freek Stulp3 and Bernd Radig2

Natural Activity

tactile channel

visual channel

auditory channel

olfactory channel

auditory channel

visual channel

tactile channel

olfactory channel

Model-based Image Interpretation

• Model Describes the image content with the help of a parameter vector p.

• Objective Function Calculates how well a parameterized model p fits to an image I.

• Fitting Algorithm Optimizes the objective function and therefore estimates the model that fits the image best.

Objective Functions

f(I,p) 0.6 0.3 0.0Splitting the Objective Function to Local Objective Functions• Evaluate one objective function per model point.• Approximate the model parameters.

Evaluation of the Objective Function• Along characteristic direction.• Often perpendicular to the model.

Traditional Approach

Shortcomings: Requires domain knowledge. Based on designer’s intuition. Time-consuming.

Manually design the

objective function

Manually evaluate on test images

designed objective function

good

not good

Ideal Objective Functions

P1: Correctness Property:The global minimum corresponds to the best model fit.

P2: Uni-modality Property:The objective function has no local extrema.

¬ P1 P1

¬P2

P2

Learning the Objective Function (1)

Learning theObjective Function (2)

Generate further Annotations• Annotations in three-dimensional space along

characteristic direction

• One characteristic direction is not sufficient.

Learning theObjective Function (3)

Learning theObjective Function (4)

• Three characteristic directions in three-dimensional space.• Model point is moved along the most important

characteristic direction.• Characteristic direction with largest angle to the image

normal is considered most important.

Learning theObjective Function (5)

Learning theObjective Function (6)

Advantages The loop is removed. The objective function approximates the ideal objective

function. No domain-dependent knowledge is needed.

Evaluation

General approach

• Uniformly distributed error is applied to models and fitting is performed afterwards.

• Distances are measured in centimeters.• Fraction of models located at a certain distance or better

is evaluated. • Two objective functions fA and fB with learning radii

∆A = 3 × ∆

B.

Evaluation (2)

• fB handles small displacements better.

• fA handles large displacements better.

• Subsequent execution shows both advantages.

Evaluation (3)

• Results are improved with every iteration.

• Lower bound of quality is reached after several iterations.

Thank you !