Boundary Value Problems and Least Squares Minimization 1Daniel Baur / Numerical Methods for Chemical...

1Daniel Baur / Numerical Methods for Chemical Engineers / BVP and PDE

Boundary Value Problems andLeast Squares Minimization

2

22

d ( , ) d ( , )0 , , , min ,

d d

y t z y t zf y z f

z z

X

Daniel Baur

ETH Zurich, Institut für Chemie- und Bioingenieurwissenschaften

ETH Hönggerberg / HCI F128 – Zürich

E-Mail: [email protected]

http://www.morbidelli-group.ethz.ch/education/index


Boundary Value Problems (BVP) for ODEs

Problem definition: Find a solution for a system of ODEs

Subject to the boundary conditions (BCs):

The total number of BCs has to be equal to the number of equations!

d ( )( , )

d

y ty f t y

t

0 0( ( ), ) 0

( ( ), ) 0f f

g y t t

h y t t


Shooting method

A first approach is to transform the BVP into an initial value problem (IVP), by guessing the missing initial conditions and using the BC to refine the guess, until convergence is reached

This way, the same algorithms as for IVPs can be used, but the convergence can be very problematic

Target

Too high: reduce initial velocity!

Too low: increase initial velocity!


Relaxation methods

Relaxation methods try to find a solution that «relaxes» from the disturbance that are the initial conditions

Example: Dimensionless tubular reactor

The solution «relaxes» into the steady state solution

2

2

1y y yr y

Pe x x

2

1 1 12ii i i i i i

y Ny y y N y y r y

Pe

2

1 1 10 2i i i i i i

Ny y y N y y r y

Pe


Visualization

Take a look at the discretized internal profile

0 L

yin

y

x

y1

y2y3 y4

y5 yN

iy

0.2y0

yN+1

L/Δx

in 10

/

1 /

y y N Pey

N Pe

1N Ny y


Relaxation methods (continued)

Therefore, we can iterate until convergence:

For a first order reaction, we have a linear system

With the «boundary conditions»

2

11 1 12k k k k k k k k

i i i i i i i i

Ny y y y y N y y r y

Pe

2

1 1 10 2i i i i i i

Ny y y N y y Da y

Pe

10

1

/ 1

1 / 1 /

N N

y N Pey

N Pe N Pey y


Matrix formulation of the linear system

If we cast the linear system into matrix form, we get y b A

2 2 2

2 2 2

2 /1 0 0

1 2 /

2 / 2 2 / 2 2 / 0 0 0

0 2 / 2 2 / 2 2 / 0 0 0

0 0 0 1 1

N Pe

N Pe

N Pe N N Pe N Da N Pe

N Pe N N Pe N Da N Pe

A

1

1 2 /

0

0

N Pe

b

1

2

N

y

yy

y


Iterative methods

Instead of simply iterating, we can use the properties of the matrix to speed up the solution

The main idea is to reformulate the problem in the following way

The goal is to choose S in a way that is easy to solve (e.g. diagonal, tridiagonal, triangular), but in some sense similar to A to keep ρ[S-1*(S-A)] small

The iteration formula therefore reads

( )

( )

y y b

y b y

A S A S

S S A

1 1k ky b y S S A

9Daniel Baur / Numerical Methods for Chemical Engineers / Iterative Methods

Jacobi’s method

The Jacobi method chooses S to be a diagonal matrix

The procedure is therefore given by

This method is guaranteed to converge if A is diagonally dominant, i.e.

( 1) ( )

( ( ))

( )k k

diag diag

y b y

D A

D D A

1 1k ky b y D D A

, ,i i i jj i

a a

1,1 1,2 1,3 1,

2,1 2,2 2,3 2,

1,1 1, 2 1, 1 1,

,1 , 2 , 1 ,

N

N

N N N N N N N

N N N N N N N

a a a a

a a a a

a a a a

a a a a

A

D


Collocation method

A more sophisticated approach is the collocation method;it is based on approximating the unknown function with a sum of polynomials multiplied with unknown coefficients

The coefficients are determined by forcing the approximated solution to satisfy the ODE at a number of points equal to the number of coefficients

Matlab has a built-in function bvp4c which implements such a method

1

1

( ) ( )N

i ii

y t u t a P t


Collocation method (continued)

We require that at every point d,

di

it c

uf c u

t


Transformation into a first order ODE

bvp4c solves first order ODEs, so if we remember the «trick» and transform our ODE, we get

And the boundary conditions

1

12

d d

d d

y y

y yy

x x

12

22

2 12

d

d

d d d

d d dn n

yy

x

y y yPe Da y Pe y Da y

x z z

1 2

2

1( 0) 1 ( 0)

( 1) 0

y z y zPe

y z


Least squares minimization

In data fitting, we generally try to solve a problem of the form

where f is some model with parameters β, X are the experimental conditions and y are the results

2 2

min , min ,i i ii

f x y f y

X

x

y


Some nomenclature

In prospect of the statistics lecture, please note the differences in nomenclature!

1 2 31 1 1 1

1 2 32 2 2 2

1 2 3

1

1

1

1

m

m

mn n n n

x x x x

x x x x

x x x x

X

1

2

n

y

yy

y

m variables (plus constant term)

n measurements /

data points

0

1

1p

p parameters (in linear

regression p = m+1)


Linear least squares minimization

If the model f(β,X) is linear (this means that the parameters β only appear in linear terms!), we saw that the unique minimizer to the problem reads

However, since XTX is much harder to solve (worse conditioned) than X, we can try to find another decomposition

where Q is an orthogonal matrix (Q-1 = QT) and R is an upper triangular matrix

T T y X X X

X QR


Linear least squares minimization (continued)

Substituing our originial problem

This yields the same minimizer, but is easier to solve

T T

T

y

y

y

y

X

QR

Q QR Q

R Q


Assignment 1

1. Solve the steady state of the dimensionless tubular reactor using Jacobi’s method. Use Pe = 10; N = 150; Da = 1; ystart = zeros(N,1); n = 1;

Define the core of the matrix A; This is easiest done using diag():

After this, change the first and the last row of A to reflect the boundary conditions and define b (see slide 6).

Use A = sparse(A); and b = sparse(b); to tell Matlab that the matrices contain mostly zeros. Check with spy(A) if this is true. This will vastly speed up all calculations.

Use Jacobi’s iteration formula and iterate until the change in y is smaller than 1e-8 or 1e5 iterations are exceeded.

Plot the results using plot(linspace(0,1,N), y)


Assignment 1 (continued)

2. Solve the same steady state of the dimensionless tubular reactor using bvp4c bvp4c uses a call of the formsol = bvp4c(ode_fun, bc_fun, solinit);

ode_fun is a function taking as inputs a scalar t and a vector y, returning as an output dy / dt

bc_fun is a function taking as inputs vectors where the boundary conditions are evaluated, returning as output the residual at the boundary

solinit is an initial guess for the solution:solinit = bvpinit(range, @initfun)

sol is a struct containing the solution and other parameters Plot the solution using plot(sol.x, sol.y(1,:)) and

compare with the solution from the first part. More on the usage of bvp4c on the next slide


Usage of bvp4c

In our case use the following

1 2

2

1( ) 1 ( ) 0

( ) 0

y a y aPe

y b

12

22 1

d

dd

dn

yy

zy

Pe y Da yz

1 2

2

1( 0) 1 ( 0)

( 1) 0

y z y zPe

y z


Exercise 2

Find online the example files from the lecture for the linear least squares problem.

We fit a linear model of the form

20 1 2,f x x x

X

0 1

1 3

2 4

3 6

4 10

x y


Assignment 2

1. Run example01.m to see the results of the fit.

2. Read through the code and ask questions if you have any.

3. Is the model really linear, even though it contains x2?

4. In the definition of the data matrix X on line 7, there is a vector of ones. Why is that? Check with the definition of a linear model in matrix form

Note: no programming will be required for this assignment.

20 1 2,f x x x

X


Exercise 3

Find online the example files from the lecture for the non-linear least squares problem.

We fit a model of the form

1 2, expf x x


Assignment 3

1. Run example02.m to see the results of the fitting.

2. Read through the code and ask questions if you have any.

3. The functions provide a Hessian and a Jacobian specific to the problem. What could be done to make a solver more general and easier to use (remember back when we implemented the Newton method!)?

4. In what cases can the Newton method fail?

5. In what cases can the Gauss-Newton method fail?

Note: no programming will be required for this assignment.

Boundary Value Problems and Least Squares Minimization 1Daniel Baur / Numerical Methods for Chemical...

Documents

Transcript of Boundary Value Problems and Least Squares Minimization 1Daniel Baur / Numerical Methods for Chemical...