1

9. Approximation Theory (Spectral Methods)

§ Curve fitting

GIVEN: , , 1,2, ,i ix y i N

Find: a function which best approxomates

the unknown function

(not necessary passing the data points)

y f x

Approximation Theory :

STEP2: optimize the fitting in some way, i.e., minimize the error term

STEP1: choose a proper funtion from ; , , ,

with some adjustable parameters , , ,

y f x a b c

a b c

2

1

; , , , , , ,N

i ii

E y f x a b c E a b c

i.e. looking for a set of a,b,c, such that

0

0

0

EaEbEc

2

Example: Given , and find a linear curve fitting.

Assume .i ix y

y ax b

To minimize the error , , we look for , such that E a b a b

Define the derivation as

2

1

,N

i ii

E y ax b E a b

1

1

0 2

0 2 1

N

i i ii

N

i ii

Ey ax b x

a

Ey ax b

b

2

1 1 1

0N N N

i i i ii i i

x y a x b x

1 1 1

0 1N N N

i ii i i

y a x b

1 1 12

2

1 1

N N N

i i i ii i i

N N

i ii i

N x y x ya

N x x

2

1 1 1 12

2

1 1

N N N N

i i i i ii i i i

N N

i ii i

x y x y xb

N x x

3

§ Approximation Theory

GIVEN: some function and some domain a,bf x

STEP1: select a proper set of linearly independent

funtions i x

which is expected to be convergent

to the exact function as f x n

0

STEP2: approximate the funtion from of the form

n

i ii

y f x

f x S x c x

STEP3: adjust the values of parameters

so that the resulting approximation is the best.ic

S x

Definition (linearly Independence)

0 1 2 n

0 0 1 1 n

0 1

The set of funtions , , , , is said

to be linearly independent on , if whenever

0 for and all , ,

then 0.n i

n

x x x x

a b

c x c x c x c s R x a b

c c c

4

Definition (Orthogonal)

if function weight therespect to with interval on the

functionsofset orthogonalan betosaidis,,,, 210

xωa,bn

ijii

b

a ji ji

jidxxxx

if0

if0

. of lsubintervaany on 0but

allfor 0 where

a,bxω

a,bxxω

Definition (Ortho-Normal):: orthogonal with i=1

n

iii xcxSxf

0

• Best approximation: define the -weighted error term as follows

b

adxxSxfxE 2

b

a

n

iii dxxcxfx

2

0

The best approximation has a set of {ci} which minimizes the error E.

njcE

j

,,2,1,0 allfor 0

~ (n+1) equations solve (n+1) unknowns {ci} ~

5

Solution:

0

2nb

i i jai

x f x c x x dx

b

a

n

i

b

a jiij dxxxxcdxxxfx0

2

b

a j

n

i

b

a jii dxxxfxdxxxxc 0

~ (n+1) equations (j=0,1,2,…,n) for (n+1) unknowns ~

(solutions exists as long as {i} is L.I.)

2

0

0nb

i iaij j

Ex f x c x dx

c c

With orthogonality (not necessary), we have

b

a j

n

iijii dxxxfxc

0

b

j j jac x f x x dx

0 if

0 if

b

i j i ijai

i jx x x dx

i j

b

j j jac x f x x dx

b

a jj

b

a j dxxxxdxxxfx

6

§ Commonly used functions {i}

nkkxx

nkkxx

x

xwba

nk

k

,,2,1forsin)(

,,2,1forcos)(

1

and1and,,choose

0

dxkxdxkx 22 cossin

nkkxxk ,,2,1,0for,1exp

(i) Fourier series (orthogonal)

2

1

choose , 1,1 and 1 1 and

( ) cos cos for 0,1,2, ,k k

a b w x x

x T x k x k n

0for

1for2/1

1

1

22

k

kdxxxTk

(ii) Chebychev polynomials (orthogonal)

choose , 1,1 and 1 and

( ) for 0,1,2, ,k k

a b w x

x L x k n

1 2

1

2

2 1kL x dxk

(iii) Legendre polynomials (orthogonal)

7

§ Application to ODEs ~ Rayleigh-Ritz method

xyy :ODEorder -2ndlinear aConsider

1 0where 0,1 , , 0,1

and 0 0 and 0 for 0 1

sufficient conditions for a unique solution

p x C q f C

p x q x x

for 0 1 and 0 1 0

d dyp x q x y f x

dx dx

x y y

The unique solution is the function which

1,02C

1

0

22

2 dxxuxfxuxqdxdu

xpxuI

PRINCIPLE OF VARIATION

satisfies BCs

minimizes the integral

8

§ Approximation Theory ~ ONE IDEA

0 select a set of

each of which satisfies the given BCs:

0 1 0

n

k k

k k

LI x

0

0

Define the set of functions spaned by

n

n k k

n

n n k kk

x

y x y x c x

xsoultuion exact theeapproximat to

fromfunction aselect

yn

2The exact solution 0 1 and B.C.sy x C ,

n

n

kkkn xcxyxy

0

011

000

0

0

n

kkkn

n

kkkn

cy

cy

2The approximation 0 1 and B.C.sn ny x C ,

9

nCn as1,02

Which is the best? The one having a minimum value of ny I

nxyyn as:eConvergenc

0

n

n n k kk

I y x I c x

SOLUTION::

1

0

22

2 dxxuxfxuxqdxdu

xpxuI

2 2

1

00 0 0

2n n n

k k k k k kk k k

p x c x q x c x f x c x dx

MINIMIAZATION: 0, for 0,1, 2, ,

j

Ij n

c

10

jc

I

0

1

00 0

2 2 2n n

j k k j k k jk k

p x x c x q x x c x f x x dx

1 1

0 00

2 2n

k j k j k jk

c p x x x q x x x dx f x x dx

0

Thus we obtain for 0,1, 2, ,n

jk k jk

a c , jb n

~ Solving ODE becomes solving matrix (algebraic) equations ~

2 2

1

00 0 0

2n n n

k k k k k kk k kj

p x c x q x c x f x c x dxc

kj jka a jb

txfuLtu

, :Given

§ Application to PDE ~ spectral methods, finite element methods, etc.

2Laplacian e.g. operatior, spartial some is where LL

term.sourceexternalknown a is x,tf

bxaxgu ,x,0 :I.C.

0fortx, :FIND tu

11

STEP1: find a set of LI and

the associated weight function

j x

w x

§ Global approximation theory:

fuLtu

:PDE into formula above theSubstitute :STEP2

txfxcLxdt

dc n

jjj

n

jj

j ,0

?

0

0

P.S. The coefficients is a function of

,

n

,

ow.j

n

n j jj

u x t

c

u x t c t x

t

for all x

§ Global approximation theory:

txfxcLxdt

dc n

jjj

n

jj

j ,0

?

0

0

Let the space spaned by . nk

n j jx

nLHS

?

nRHS

not necessary!

12

• Several ways to obtain an approximation:

0 0

,n n

jj j j

j j

dcx L c x f x t

dt

(i) best approximation: look for a set of which minimizesjc t

at which nodes ofset a choose :methodsn Collocatio (ii) n

0iix

0 0

,n n

jj j j

j ji i i

dcL c f t

dtx x x

(iii) Galerkin approximation

0

0

,

nj

j

b

ia

b b

i

j

n

j j ia aj

x x dx

x x d

dcx

dt

L c x dx tx xfx x

§ Galerkin approximation – linear L

dxtxfxxdxxLxxc

dxxxxdt

dc

b

a i

b

a ji

n

jj

n

j

b

a jij

,0

0

define if orthonorma

,

lb

ij i ja

b

ij

ij

i ja

b

i ia

a x x x dx

b x x L x dx

r t x x f x t dx

ω

trtcbdt

dca i

n

jjij

n

j

jij

00

~ PDE is reduced to a system of coupled ODEs ~

~ go time-marching ~

13

trtcbdt

dci

n

jjij

j 0

~ decoupled on LHS ~

special case: -orthonormal : j ij ijx a δ

§ Initial Conditions, i.e. cj(0)

0

,0 ,0 0n

n j jj

u x g x u x c x

n

jjj xcxg

0

0 minimize :ionapproximatbest

0

Collocation: 0n

i j j ij

g x c x

dxxxxcdxxgxxb

a ji

n

jj

b

a i

0

0 :Galerkin

e.g. Bu(x,t) = 0 for some linear spatial operator B

§ Boundary Conditions:

0 0

, 0n n

n j j j jj j

Bu x t B c t x c t B x

i.e. 0jB x

(a) select with the addional restriction

that each .

j

j

x

x BCs

(b) (Tau method) Discard as may ODEs as the number

of BCs and impose

0

, 0n

n j jj

Bu x t c t B x

14

§ Example: thermal boundary layer

2

2

:equation GoverningxT

DtT

iTtxT 0, :I.C.

sTtxT 0,1 :B.C.s

define ( , ) i

s i

T Tx t

T T

2

2

:equation GoverningxT

DtT

iTtxT 0, :I.C.

sTtxT 0,1 :B.C.s

2

2D

t x

,0 0x

1, 1t

15

1near developtoexpectedislayer bounday A thermal x

1near pointsgriddenser better x

Expect the solution is symmetric.

12

choose Chebychev polynomials of even degrees

cos 2 cosjT x j x

§ collocation + methods

collocation grid points: choose the locations of peaks of Chebychev,

xTx jj 2 choose

20

, ,n

n j jj

x t x t c t T x

ni

nin

xTx ni ,2,1,04

2cos ofpeak of locations 4

16

2

2D

t x

n

j

jj

n

jj

j

dx

xTdcDxT

dt

dc

02

22?

02

)()(

20

, ,n

n j jj

x t x t c t T x

2

22 2

0 0

( )( )

n nj j

j i j ij j

dc d T xT x D c t x

dt dx

discareded1,2,1,0for nini

20 0

B.C.: 1, 1 1n n

j j jj j

t c t T c t

§ collocation + methods

• Initial Conditions

20

,0 0 0

for 0 1 2 1

n

i j j ij

x c T x

i , , , ,n

0

1,0 1 0n

jj

c

20

, ,n

n j jj

x t x t c t T x

~ (n+1) equations for (n+1) unknowns ~

17

§ Galerkin method+{j(x)} B.C.s

11 and 1 Since 20 jTxT

1 1

1, 1 1 1 0 1n n

n j j jj j

t c t c t

1

2 01

, , 1

1

n

n j jj

n

j jj

x t x t c t x

c t T x T x

2 0Let . Thus 1 0j j jx T x T x

B.C.s

n

j

jj

n

jj

j

dx

xdtcDx

dt

dc

12

2?

1

)()(

• Use orthogonal property:

1 1

2

2 2

21 1

1 1

( ( ))( )

1 1

( ) in n

jj jj

j

i

j

d x dxdc xD c

xxdx dx

xt

dt x

n

jjij

n

j

jij nitcbD

dt

dca

11

,1,2,for

• Substitute into governing equation:2

2D

t x

1

, 1n

n j jj

x t c t x

18

1

1

20202 1 dxxxTxTxTxT ji

1

1

220202022 1 dxxTTTTTTT jiji

1 2 22 2 01

1i jT T T x dx

1 if2

3

if

ji

ji

1

21

( ) ( ) , , 1, 2, ,

1

i jij

x xa dx i j n

x

1

1

2022

2

02 1 dxxxTxTdx

dxTxT ji

2

1 2 22 0 21

1ji

d TT T x dx

dx

21 2

21

( )( ) 1j

ij i

d xb x x dx

dx

2 1

22 2 22

0

jj

j k kk

d T xP x T x

dx

1 1 2

2 2 0 210

1j

ij k i k kk

b T T T T x dx

19

Initial Conditions: ,0 0x

1 1

2 21 11

( ) ( )( ,0) 1 ( )0

1 1

ni ji

jj

x xx xdx c dx

x x

1 2

211

1 ( )0 for 1,2, ,

1

ni

ij jj

T xdx a c i n

x

1

, 1n

n j jj

x t c t x

1

,0 1 0n

n j jj

x c x

1

divide the interested domain

into several subdomins K

kk

a,b

♣ Local approximation theory

0

0P.S. is different from one subdomain to anoth

for all ,

.

,

er

nk k kn j j k

j

nkj j

x

u x t u x t c t x x

0

0

i.e. require sets of L.I. functions

and will have sets of corresponding coefficients

nkj j

nkj j

K x

K c t

20

§ Finite Element Methods ~ piecewise approximation

)boundary with (domain :problemTest

2221

( ) ( , ) ( , ) ( , ) ( , ) ( , )2

u uI u p x y q x y r x y u f x y u x y dxdy

x y

ofvaluetheminimizesandBCs satisfieswhich

function theis unique) and exists (ifsolution exact The

::VARIATION OF PRINCIPLE2 C

onyxgyxu

yxfyxuyxryu

yxqyx

uyxp

x

,,

,,,,,

1

STEP1: Divide the interested domain into several parts, i.e.

Each subdomain is called an "element".

K

kk

k

0

STEP2: within each subdomain , choose a proper set of , j

nk

kj

x y

k0

for ,,,

x,yyxcyxuyxun

j

kkk

jjn

0

P.S. , are different from one element to another in general.j

nk

jx y

21

01 1

Put , , ,K K n

k k kn j j

jk k

u x y u x y c x y

1

Thusk

Kkj

k

I u I c

STEP3: Look for the values of 0 1 and 0 1

such that , minizes

kj

kn

c , j , , ,n k , , ,K

u x y I u

2221

( ) ( , ) ( , ) ( , ) ( , ) ( , )2

u uI u p x y q x y r x y u f x y u x y dxdy

x y

2

0

1( , )

2m

mnjm

jmji

p x y c dxdyc x

)(minimized required 0

10alland10 allfor Therefore

micuI

,K,,m,n,,i

K

k

n

j

kjk

jmi

k xcyxp

c 1

2

0

,21

2

0

1,

2m

mnjm

jmji

p x y cc x

0 0

( , )m

mnjm

jj

mnjm

jmji

p x y c dcx

x yc

dx

0

m mnj i

ijj x x

22

0

0 ( , ) ( , ) ( , )k

k kk knj jk k ki i

j j ij

c p x y q x y r x y dxdyx x y y

( , ) ( , )k

kif x y x y dxdy

)(minimized required 0 m

icuI

0 0

0

, ,

, ,

m

m mm mn nj jm mi i

j jj j

nm m m mj j i i

j

p x y c q x y cx x y y

r x y c f x y dxdy

)variable(change :ntrearangemeAfter km

, , ,k

k kk kj jk k ki i

ij i j

kji

a p x y q x y r x y dxdyx x y y

a

, ,k

k ki ib f x y x y dxdy

0

or ~ elemental matrix equation

for 1 2

nk k k k k kij j i

j

a c b A c b

k , , ,K

• Ak is singular! i.e. Each element matrix equation is underminate.

► too many degrees of freedom!

23

then,alsoand,,if:trequiremen Continuity21 kkyx

1 1 1

0

, ,n

k k kn j j

j

u x y u c x y

2 2 2

0

, ,n

k k kn j j

j

u x y u c x y

needst requiremen Thus 0-C

1 1 2 2

0 0

, ,n n

k k k kj j j j

j j

c x y c x y

tsrequiremen the with

redonebeshouldproblemon minimizati The0-C

STEP1: within element , choose 1 points

denoted by , for 0 1 2

k

k ki i

n

x y i , , , ,n

,k k ki j j ijx y

n

ill

kl

ki

kl

ki

kl

klk

i

yyxx

yyxxyx

022

22

,

§ Example: use collocation points

yxki , s,polynomial Lagrangian ldimentiona-say two

function,interpointan chooseelement,each within :STEP2

and approximate the desired function as

k

n

i

ki

ki

kn yxyxcuyxu

,whenever,,,0

24

Notice:

iikj

n

j

kj

kiii

kn yxcuyxu ,,

0

ij

n

j

kj

ki cu

0

kic

0

, ,n

k k kn j j

j

u x y u x y

k

n

i

ki

ki

kn yxyxcuyxu

,whenever,,,0

§ example – triangular elements (linear interpolation)

1 2

3

11u

12u

13u

21u

22u

23u

31u

33u

32u

0 1 1 1 2 2 2 3 3 31 2 3 1 2 3 1 2 3If no constraint: d.o.f.s , , , , , , , ,C u u u u u u u u u

0with constraint: d.o.f.s = , , , ,a b c d eC - u u u u u

a

b c

d

e

11

1 22 1

0 2 32 1

32

1 2 33 3 3

:

a

b

c

d

e

u u

u u u

C u u u

u u

u u u u

25

: d.o.f. 11uua

1 0

,k

K nk kj j

k ja

u x yu

0

auuI

1

1 1

0

,n

j jja

u x yu

1

1 11

01

,n

j jj

u x yu

11

13

113

12

112

11

1110 buauaua

32d.o.f. :du u 3 3 3 3 3 3 3

21 1 22 2 23 3 20 a u a u a u b

1 2

1 1 2 2

0 0

, ,n n

j j j jj jb

u x y u x yu

0

buuI

: d.o.f. 21

12 uuub

1 2

1 1 2 21 2

0 02 1

, ,n n

j j j jj j

u x y u x yu u

21

23

213

22

212

21

211

12

13

123

12

122

11

1210 buauauabuauaua

26

2 3

2 2 3 3

0 0

, ,n n

j j j jj jc

u x y u x yu

0

cuuI

: d.o.f. 31

22 uuuc

2 3

2 2 3 32 3

0 02 1

, ,n n

j j j jj j

u x y u x yu u

31

33

313

32

312

31

311

22

23

223

22

222

21

2210 buauauabuauaua

321

cu

0

e

I u

u

1 2 33 3 3d.o.f. :eu u u u

1 2

3

1 1 2 21 2

0 03 3

3 33

03

, ,

,

n n

j j j jj j

n

j jj

u x y u x yu u

u x yu

1 1 1 1 1 1 1 2 2 2 2 2 2 231 1 32 2 33 3 3 31 1 32 2 33 3 3

3 3 3 3 3 3 331 1 32 2 33 3 3

0 a u a u a u b a u a u a u b

a u a u a u b

27

1 1 111 12 13

1 1 2 2 1 221 22 11 12 23 13

2 2 3 3 2 321 22 11 12 23 13

0 0

0

0

0

a a a

a a a a a a

a a a a a a

3 3 321 22 23

1 1 2 2 3 3 1 2 331 32 31 32 31 32 33 33 33

0

a a a

a a a a a a a a a

11

1 22 1

2 32 1

32

1 2 33 3 3

a

b

c

d

e

u u

u u u

u u u

u u

u u u u

11

1 22 1

2 32 1

32

1 2 33 3 3

b

b b

b b

b

b b b

• Global matrix equation Au=b b c d ea

1 1 111 12 13

1 1 121 22 23

1 1 131 32 33

0 0

0 0

0 0 0 0 0

0 0 0 0 0

0 0

a a a u

a a a

a a a

11

12

13

0

0

u

u

2 2 2 211 12 13 1

2 2 2 221 22 23 2

22 2 2331 32 33

0 0 0 0 0 0

0 0

0 0

00 0 0 0 0

0 0

a a a u

a a a u

ua a a

33

32

31

333

332

331

323

322

321

313

312

311

0

0

0 0

0 0

0 0

0 0 0 0 0

0 0 0 0 0

u

u

u

aaa

aaa

aaa

11

12

13

0

0

b

b

b

21

22

23

0

0

b

b

b

31

32

33

0

0

b

b

b

stiffness summation

28

• local matrix equation

1 1 1 1 1 1 1 111 12 13 1 11 12 13 11 1 1 1 1 1 1 121 22 23 2 21 22 23 21 1 1 1 1 1 1 131 32 32 3 31 32 32 3

1:a

b

e

a a a u a a a u b

k a a a u a a a u b

a a a u a a a u b

2 2 2 2 2 2 2 211 12 13 1 11 12 13 12 2 2 2 2 2 2 221 22 23 2 21 22 23 22 2 2 2 2 2 2 231 32 32 3 31 32 32 3

2 :b

c

e

a a a u a a a u b

k a a a u a a a u b

a a a u a a a u b

33

32

31

332

332

331

323

322

321

313

312

311

33

32

31

332

332

331

323

322

321

313

312

311

:3

b

b

b

u

u

u

aaa

aaa

aaa

u

u

u

aaa

aaa

aaa

k

e

d

c

§ Example – 2D Poisson Solver

tyxfyP

xP

P ,,2

2

2

22

29

• Let

K

k

N

i

N

j

kj

ki

kij yhxhPtyxP

1 0 0

',,

K

k

N

i

N

j

kj

ki

kij yhxhftyxf

1 0 0

',,

N

imm

ki

km

kik

i xx

xxxh

0

N

jmm

kj

km

kjk

j yy

yyyh

0

nodeson Intergrati Quadrature Gauss are and where00

N

j

kj

N

iki yx

Numerical Analysis• 挑戰人類結合數學及物理知識能力

• 激發人類的創造力與邏輯思考

• 數位化儀器的秘密武器

• 提供經濟的設計分析工具

• 數值誤差 – truncation error & rounding error

• 數值現象 – numerical diffusion & dispersion

• 穩定性、準確性、效率

30

31

32

33

Happy Summer Vacation!