Variational Modeling

Statistical Geometry Processing Winter Semester 2011/2012

Variational Modeling Basic Techniques

3

Bayesian Approach

P(S |D) = P(D| S ) P(S )

P(D)

prior assumptions measurement model (“likelihood”)

optimize (best S)

Candidate reconstruction S –

Measured data D –

4

Calculus of Variation

Basic Idea:

• We look at a set of functions f: S D

• We define an “energy functional” E: (S D)

A functional assigns real numbers to functions

Each function gets a “score”

“Energy” means: the smaller the better

• We set up additional requirements (“constraints”) on f.

Soft constraints violation increases energy.

Hard constraints violation not allowed.

• We then compute the function(s) f that minimize E.

5

Calculus of Variation

Very general framework:

• A lot of problems can be directly formulated as variational problems.

• Example 1:

We are looking for a curve.

It should be as smooth as possible (energy = non-smoothness).

It should go through a number of points (hard constraints).

E large E small

constraints

6

Calculus of Variation

Another example:

• Problem: We want to go to the moon.

• Given:

Orbits of moons, planets and star(s).

Flight conditions (athmosphere, gravitation of stellar bodies)

• Unknowns:

Throttle (magnitude, direction) from rocket motors (vector function)

• Energy function:

Usage of rocket fuel (the fewer the better)

Perhaps: Overall travel time (maybe not longer than a week)

7

Calculus of Variation

To the moon:

• Constraints:

We want to start in Cape Canaveral (upright trajectory) and end up on the moon.

We do not want to hit moons or planets on our way.

We want to approach the moon at no more than 20 km/h relative speed upon touchdown.

The rocket motor has a limited range of forces it can create (not more than a certain thrust, no backward thrust)

So flying to the moon is just minimizing a functional. (ok, this is slightly simplified)

8

A Simple Example

Simple example: variational splines

• Energy:

We want smooth curves

Smooth translates to minimum curvature

Quadratic penalty:

curve

2|)(|)( dttcurvaturefE f

9

A Simple Example

Simple example: variational splines

• Energy:

Problem: curvature is non-linear

Easier to minimize: second derivatives

curve

2

2

2

)()( dttdt

dfE f

10

A Simple Example

Simple example: variational splines

• Soft constraints

Parameter values t1,...,tn at which we should approximate points p1, ..., pn:

controls (lack of) smoothness.

n

iii

t

tt

tdttdt

dfE

n

1

2

2

2

2

)()()(1

pff

11

A Simple Example

Simple example: variational splines

• Soft constraints

Specify the accuracy by error quadrics Q1, ..., Qn:

n

iiiiii

t

tt

ttdttdt

dfE

n

1

T

2

2

2

)()()()(1

pfQpff

12

Rank-Deficient Quadrics

The rank deficient error quadric trick:

• A rank-1 matrix constraints the curve in one direction only

• E.g.: Point-to-surface constraints

n TnnQ i

ti

13

Numerical Treatment

Numerical computation:

• No closed form solution

Discretize (finite dimensional function space)

Solve for coefficients (coordinate vector in this function space)

14

Finite Differences

FD solution:

• Represent curve as array of k values:

• Unknowns are the curve points y1, ..., yk

t 0 0.1 0.2 ... 7.4 7.5

y y0 y1 y2 ... Y74 y75

y1

y2

yk

15

Discretized Energy Function

Discretized Energy Function:

• Energy is a squared linear expression

Quadratic discrete objective function

• Constraints are quadratic by construction

• Solution by linear system

n

iitindexiitindex

k

i

iiidiscr

n

iiiiii

t

tt

ii

n

hfE

ttdttdt

dfE

1)(

T

)(

2

12

11)(

1

T

2

2

2

2)(

)()()()(1

pyQpyyyy

pfQpff

(neglected here: handling boundary values)

16

Summary

Summary:

• Variational approaches look like this:

• Connection to statistics:

Bayesian maximum a posteriori estimation

E(data) is the data likelihood (log space)

E(regularizer) is a prior distribution (log space)

s}constraint hard satifies |{

,)()()(

),(minarg compute

)()(

ffFf

fEfEfE

fE

rregularizedata

Ff

Variational Toolbox Data Fitting, Regularizer Functionals,

Discretizations

18

Toolbox

In the following:

• We will discuss...

...useful standard functionals.

...how to model soft constraints.

...how to model hard constraints.

...how to discretize the model.

• Then snap & click your favorite custom variational modeling scheme.

• (Click & snap means: add together to a composite energy)

Functionals

20

Functionals

Standard Functional #1: Function norm

• Given a function f: m n

• Minimize:

• Function values should not become too large

• Often useful to avoid numerical problems

Adding E(zero) to quadratic energy: smallest eigenvalue bounded by ( condition number)

System always solvable

xxf dfE zero 2)( )()(

21

Functionals

Standard Functional #2: Harmonic energy

• Given a function f: m n

• Minimize:

• Differences to neighboring points as small as possible

• Appears all the time in physics & engineering

xxf dfE harmonic 2)( )()(

22

Harmonic Energy

Example: Heat equation

• Metal plate

• Hard constraints:

Heat source

Heat sink

• Final heat distribution?

Heat flow tends to equalize temperature.

– Stronger heat flow for larger temperature gradients.

Gradients become as small as possible.

heat sink heat source

23

Harmonic Energy

Example: Harmonic energy

• Curves that minimize the harmonic energy:

Shortest path, a.k.a. polygons

• Two-dimensional parametric surface:

24

Surface Example

Surface fitting with Laplacian Regularizer:

initialization result

Data attraction: point-to-plane, Gaussian window Regularizer: minimize triangle edge length

25

Functionals

Standard Functional #3: Thin plate spline energy

• Given a function f: m n

• Minimize:

• Objective: minimize integral second derivatives (approx. curvature)

• “Be smooth”:

Yields smooth curves & surfaces

A true curvature based energy is rarely used (non-quadratic)

xxff d

xxE

m

i

m

j ji

TSS

1 1

22

)( )()(

26

Energies for Vector Fields

Vector fields:

• Now consider volume deformations: n n

• Think of an object moving (over time).

f(x) describes its deformation.

f(x,t) describes its motion over time.

n f() n

f: n n

27

Functionals

Standard Functional #4: Green’s deformation tensor • Given a function f: n n

• Minimize:

• Objective: minimize metric distortion First fundamental form

• Physically-based deformation modeling: Invariant under rigid transformations.

Bending, scaling, shearing is penalized.

Energy is non-quadratic (4-th order).

99 Matrix M encodes material properties (often M = I).

xIffMf dEF

deform2

T)( )(

28

Functionals

Standard Functional #5: Volume preservation • Given a function f: n n

• Minimize:

• Objective: minimize local volume changes

• This energy tries to preserve the volume at any point.

Incompressible materials (for example fluids)

Invariant under rigid transformations

Non-quadratic (6th-order in 3D)

xff dE volume 2)( 1)det()(

29

Functionals

Standard Functional #6: Infinitesimal volume preservation • Velocity v: n n

• Minimize:

• Objective: minimize local volume changes in a velocity field

• Difference to the previous case: The vectors are instantaneous motions (v(x) = d/dt f(x,t))

A divergence free (time dependent) vector field will not introduce volume changes

Linear, but works only for small time steps

Large (rotational) displacements are not covered

xxvxvxxvv d

xxdE

n

volume

2

1

2)( )(...)()(div)(

30

Functionals

Standard Functionals #7 & #8: Velocity & acceleration • Given a function v: (n ) n

• Minimize:

• Objective: minimize velocity / acceleration

• Models air resistance, inertia.

dtdt

dt

dEdtdt

dt

dE accvelocity xxffxxff

2

2

2)(

2

)( ),()(,),()(

Soft Constraints

32

Soft Constraints

Penalty functions • Uniform • General quadrics • Differential constraints

Types of soft constraints • Point-wise constraints • Line / area constraints

Constraint functions • Least-squares • M-estimators

33

Uniform Soft Constraints

Uniform, point-wise soft constraints: • Given a function f: n n

• Minimize:

n

iiii

constr qE1

2)( )()( yxff

constraint weights (certainty)

prescribed values (x,y)i

34

Uniform Soft Constraints

General quadratic, point-wise soft constraints: • Given a function f: n n

• Minimize:

n

iiiiii

constrE1

T)( )()()( yxfQyxff

constraint weights (general quadratic form, non-negative)

prescribed values (x,y)i

35

Uniform Soft Constraints

Differential constraints: • Given a function f: n n

• Minimize:

n

iiiiii

constr DDDDE1

T)( )()()( yxfQyxff

constraint weights (general quadratic form, non-negative)

prescribed values (x,Dy)i

Differential operator:

mmkm

k

ii

ii

xx

xx

D

,,1

1,11,1

...

...

This is still a quadratic constraints ( linear system).

36

Examples

Examples of differential constraints:

• Prescribe normal orientation of a surface

• Prescribe rotation of a deformation field

• Prescribe velocity or acceleration of a particle trajectory

2

)(

1

)(,:

nfff v

u

constr qE

2)( )(,:F

constr qE Rfff

2)( ),(),(),()(,),(,: tttxqEt constr xaxffposxff

37

Line / Area Soft Constraints

Line and area constraints: • Given a function f: n n

• Minimize:

• A.k.a: “transfinite constraints”

A

constrE )()()()()()(T)( xyxfxQxyxff

quadric error weights (may be position dependent)

prescribed values y(x) (function of position x)

area A on which the constraint is placed (line, area, volume...)

38

Constraint Functions

Constraint Functions: • Typical: quadratic constraints E(x) = f (x)2

Easy to optimize (linear system)

Well-defined critical point (gradient vanishes)

However: sensitive to outliers

• Alternatives for bad data: L1-norm constraints (E(x) = |f (x)|)

– more robust

– still convex, i.e. can be optimized

Truncated constraints

– even more robust

– non-convex, difficult to optimize

Discretization

40

Finite Element Discretization

Finite-element discretization:

• Finite dimensional space spanned by basis functions

• Finite differences (FD)

Special case

Grid of piecewise constant basis

• General approach:

k

iii

f

xbxf

fEfE

1

)()(~

)~

(minarg)(minarg

41

Finite Element Discretization

Derive a discrete equation:

• Just plug in the discrete f.

• Then minimize the it over the .

• Compute the critical point(s):

• Quadratic functionals: linear system.

• Non-linear, smooth functionals: Newton, Gauss-Newton, LBFGS, or the similar

~

0)(~

:...1

min)(~

xfEki

xfE

i

42

Example

(Abstract) example:

• Minimize square integral of a differential operator

• Quadratic differential constraints

• Yields quadratic optimization problem in the coefficients

43

Example

(Abstract) example (cont):

n

ii

k

iii

k

i

k

jjiji

n

ii

k

iii

k

iii

n

ii

k

iii

k

iii

k

iii

n

iii

yxbDdxxbDxbD

yxbDdxxbD

yxbDdxxbDfE

xbxf

yxfDdxxfDfE

1

2

1

)2(

1 1

)1()1(

1

2

1

)2(

2

1

)1(

1

2

1

)2(

2

1

)1(

1

1

2)2(2)1(

)()()(

)()(

)()()~

(

)()(~

)()()(

Numerical Aspects

45

How to solve the problems?

Solving the discretized variational problem:

• Quadratic energy and quadratic constraints:

The discretization is a quadratic function as well.

The gradient is a linear expression.

The matrix in this expression is symmetric.

If the problem is well-defined, the matrix is semi-positive definite.

It is usually very sparse (coefficients of basis functions only interact with their neighbors, as far as their support overlaps).

We can use iterative sparse system solvers:

– Most frequently used: conjugate gradients (needs SPD matrix). CG is available in GeoX.

46

How to solve the problems?

Solving the discretized variational problem:

• Non linear energy functions: If the function is convex, we can get to a critical point that is the global

minimum.

In general, we can only find a local optimum (or critical point).

– Need a good initialization

Newton optimization:

– 2nd order Taylor expansions (Hessian matrix, gradient)

– Iteratively solve linear problems.

– Typically, Hessian matrices are sparse. Use conjugate gradients to solve for critical points.

Non-linear conjugate gradients: with line search (faster than simple gradient decent).

LBFGS: Black box-solver, only needs gradient.

Hard Constraints

48

Hard Constraints

Hard Constraints:

• Properties of the solution to be met exactly

• Three options to implement hard constraints:

Strong soft constraints (easy, but not exact)

Variable elimination (exact, but limited)

Lagrange multipliers (most complex and general method)

49

Hard Soft Constraints

Simplest Implementation:

• Soft constraints with large weight

• A few serious problems:

Technique is not exact

– Might be not acceptable for some applications

The stronger the constraints, the larger the weight. This means:

– The condition number of the quadric matrix (condition of the Hessian in the non-linear case) becomes worse.

– At some point, no solution is possible anymore.

– Iterative solvers are slowed down (e.g. conjugate gradients)

)10 (say large very with),( )()( 6)()( fEfEfE sconstraintrregularize

50

Variable Elimination

Idea: Variable elimination

• We just replace variables by fixed numbers.

• Then solve the remaining system.

Example:

4.0

2.5

4.5

y1 y2

y3 y4 y5 y6

y7 y8

y9

f ’(x0) = h-1(y1 – 4.0)

f ’(x3) = h-1(y4 – y3)

51

Variable Elimination

Advantages: • Exact constraints

• Conceptually simple

Problems: • Only works for simple constraints (variable = value)

• Need to augment system Not easy to implement generically

• Does not work for FE methods (general basis functions) Values are sum of scaled basis functions

52

Lagrange Multipliers

Most general technique: Lagrange multipliers

• Works for complex, composite constraints

• General basis functions

• Exact solutions (no approximation)

53

Lagrange Multipliers

Here is the idea:

• Assume we want to optimize E(x1, ..., xn) subject to an implicitly formulated constraint g(x1, ..., xn) = 0.

• This looks like this:

E g 0)(, xggE

E

E g

E

g

54

Lagrange Multipliers

Formally:

• Optimize E(x1, ..., xn) subject to g(x1, ..., xn) = 0.

• Formally, we want:

• We get a local optimum for:

• A critical point of this equation satisfies both and .

gE

0)( and )()( xxx ggE

0)(,,...,:i.e.

0)(

)()()(

1

,

x

x

xxx

x

LG

LG

gELG

nxx

0)( xg)()( xx gE

E

g

55

Example

Example: Optimizing a quadric subject to a linear equality constraint

• We want to optimize:

• Subject to:

We obtain:

•

• Linear system:

bxAxxx T)(E

0)( ng mxx

nxELG mxbxAxxgxx T)()()(

nLG

LG

mxx

mbAxxx

)(

2)(

n

b

x

m

mA

0

2T

56

Multiple Constraints

Multiple Constraints:

• Similar idea

• Introduce multiple “Lagrange multipliers” .

0)(,...,,,...,:i.e.

0)(

)()()(

0)(:...1 :to subject

min)(

11

,

1

x

x

xxx

λx

LG

LG

gELG

xgki

xE

knxx

k

iii

i

Lagrangian objective function:

57

Multiple Constraints

Example: Linear subspace constraints

• subject to

•

• Linear system:

• Remark: M must have full rank for this to work.

bxAxxx T)(E 0nMxx )(g

n

iiii

n

iii nxELG

1

T

1

)()()( xmbxAxxgxx

n

b

λ

x

M

MA

0

2 T

58

What can we do with this?

Multiple linear equality constraints:

• Constraint multiple function values, differential properties, integral values

• Area constraints: Sample at each basis function of the discretization and prescribe a value

• Need to take care:

We need to make sure that the constraints are linearly independent at any time

Inequality constraints:

• There are efficient quadratic programming algorithms. (Idea: turn on and off the constraints intelligently.)

Manifold Constraints

60

Optimization on Unit Sphere

Solution: Local Parameterization

• Current normal estimate

• Tangent parameterization

• New variables u, v

• Renormalize

• Non-linear optimization

• No degeneracies

tangentu

tangentv

n0

n(u,v)

v

u

tangentv

tangentunvun

0),(

[Hoffer et al. 04]

61

Optimization on SO(3)

Orthonormal matrices

• Local, 1st order, non-degenerate parametrization:

• Optimize parameters, , , then recompute A0

• Compute initial estimate using [Horn 87 ]

0

0

0)(

t

iC

)(

)exp(

)(

0

0

t

i

ii

I

CA

CAA

c.f: normal optimization

The Euler Lagrange Equation (some more math)

63

The Euler-Lagrange Equation

Theoretical Result:

• An integral energy minimization problem can be reduced to a differential equation.

• We look at energy functions of a specific form:

f is the unknown function

F is the energy at each point x to be integrated

F depends (at most) on the position x, the function value f (x) and the first derivative f'(x).

],[: baf

b

a

dxxfxfxFfE ))('),(,()(

64

The Euler-Lagrange Equation

Now we look for a minimum:

• Necessary condition:

• (critical point)

• In order to compute this:

Approximate f by a polygon (finite difference approximation)

f = ((x1, y1), ..., (xn, yn))

Equally spaced: xi – xi-1 = h

0)(""

fEdf

d

^

y6

y1 y2

y3 y4 y5 y7

y8 y9

y10

(Can be formalized more precisely using functional derivatives)

65

The Euler-Lagrange Equation

Minimum condition:

y6

y1 y2

y3 y4 y5 y7

y8 y9

y10

n

i

iiii

b

a

h

yyyxFEfE

dxxfxfxFfE

2

1,,)(~

)(

))('),(,()(

y

n

i

iiii

iiii

n

i

iiii

yy

h

yyyxF

hh

yyyxF

h

yyyxF

EEn

2

0

1

1

0

13

0

1

0

0

12

2

1

,,1

,,

,,

~,...,

~1

y

y

66

The Euler-Lagrange Equation

Minimum condition:

h

yyyxF

h

yyyxF

hh

yyyxFE

i

h

yyyxF

hh

yyyxFE

iiii

iiii

iiiiy

n

i

iiii

iiii

i

13

13

12

2

0

1

1

0

13

0

1

0

0

12

,,,,1

,,~

:entry th

,,1

,,~

y

0))('),(,())('),(,( 32 xfxfxFdx

dxfxfxF

Letting h 0, we obtain the continuous Euler-Lagrange differential equation:

67

The Euler-Lagrange Equation

0))('),(,())('),(,( 32 xfxfxFdx

dxfxfxF

f '(x)

f

(at every point x)

f (x)

x

68

Example

Example: Harmonic Energy

b

a

dxxfdx

dfE

2

)()(

2)('))('),(,( xfxfxfxF

0)(

0)(20

0))('(0

0))('),(,())('),(,(

2

2

2)('

32

xfdx

d

xfdx

d

dx

d

xfdx

d

xfxfxFdx

dxfxfxF

xf

69

Generalizations

Multi-dimensional version:

df :

dxxd dxdxffxfxxFfEd

...)(),...,(),(,,...,)( 11 1xx

Necessary condition for extremum:

0)( 1

d

i xi if

E

dx

d

f

E

x

)(: xfx

fi

xi

This is a partial differential equation (PDE).

70

Example

Example: General Harmonic energy

xxf dfE harmonic 2)( )()(

0)(...)()(2

2

2

1

2

xxxf f

xf

x d

Euler Lagrange equation:

71

Summary

Euler Lagrange Equation: • Converts integral minimization problem into ODE or PDE.

• Gives a necessary, but not sufficient condition for extremum (critical “point”, read: function f )

• Application: From a numerical point of view, this does not buy us much.

– We can usually directly optimize the integral expression.

– Similarly complex to compute (boundary value problem for a PDE vs. variational problem).

Analytical tool

– Helps understanding the minimizer functions.

Animation Reconstruction

73

Variational Animation Modeling

f (x, t) – deformation field

t = 0 t = 1 t = 2

x – point on urshape S

S

f f

74

Variational Framework

)(

2

),(),()()(SV

F

Trigidrigid dxttxE Ixfxff xx

)(

21),()()(

SV

volvolume dxtxE xff x

S

accaccel dxtt

xE

2

2

2

),()()( xff

S

velocityvelocity dxtt

xE

2

),()()( xff

T

t

n

iimatch

t

SfddistE1 1

2))(,()(f

))(()()(

ndeformatiosconstraint

fff velocityaccelvolumerigidmatch EEEEEE

[B. Adams, M. Ovsjanikov, M. Wand, L. Guibas, H.-P. Seidel, SCA 2008]

[B. Adams, M. Ovsjanikov, M. Wand, L. Guibas, H.-P. Seidel, SCA 2008]

[M. Wand, B. Adams, M. Ovsjanikov, M. Bokeloh, A. Berner, P. Jenke, L. Guibas, H.-P. Seidel, A. Schilling, 2008] (data set courtesy of P. Phong, Stanford. U.)