LU-factorization and Positive Deﬁnite Matrices · Uniqueness Theorem 5. The LU-factorization of a...

LU-factorization and PositiveDefinite Matrices

Tom Lyche

University of Oslo

Norway

LU-factorization and Positive Definite Matrices – p. 1/49

http://www.ifi.uio.no/

Topics TodayBlock multiplication of matrices

Basics on triangular matrices

LU factorization of matrices

Positive definite matrices

examples

criteria for positive definiteness

LU factorization of positive definite matrices


Partitioned matricesA rectangular matrix A can be partitioned into submatrices by drawinghorizontal lines between selected rows and vertical lines betweenselected columns. For example, A =

[

1 2 34 5 67 8 9

]

can be partitoned as

(i)

A11 A12

A21 A22

=

1 2 3

4 5 6

7 8 9

, (ii)[

a.1, a.2, a.3

]

=

1 2 3

4 5 6

7 8 9

,

(iii)

aT1.

aT2.

aT3.

=

1 2 3

4 5 6

7 8 9

, (iv)[

A11, A12

]

=

1 2 3

4 5 6

7 8 9

.

The submatrices in a partition is often referred to as blocks and apartitioned matrix is sometimes called a block matrix .


Column partitionSuppose A ∈ R

m,p and B ∈ Rp,n.

If B =[

b.1, . . . , b.n

]

is partitioned into columns then the partition of

the product AB into columns is

AB =[

Ab.1, Ab.2, . . . , Ab.n

]

.

In particular, if I is the identity matrix of order p then

A = AI = A[

e1, e2, . . . , ep

]

=[

Ae1, Ae2, . . . , Aep

]

and we see that column j of A can be written Aej for j = 1, . . . , p.


Row partitionIf A is partitioned into rows then

AB =

aT1.

aT2.

...

aTm.

B =

aT1.B

aT2.B

...

aTm.B

and taking A = Ip it follows that row i of B can be written eTi B.

It is often useful to write the matrix-vector product Ax as a linearcombination of the columns of A

Ax = x1a.1 + x2a.2 + · · · + xpa.p.

One way to see that this is correct is to partition A into columns andx into rows.


Rules for 2 × 2 blocks

If B =[

B1, B2

]

, where B1 ∈ Rp,r and B2 ∈ Rp,n−r then

A[

B1, B2

]

=[

AB1, AB2

]

.

If A =

A1

A2

, where A1 ∈ Rk,p and A2 ∈ R

m−k,p then

A1

A2

B =

A1B

A2B

.

If A =[

A1, A2

]

and B =[

B1, B2

]

, where A1 ∈ Rm,s, A2 ∈ R

m,p−s,

B1 ∈ Rs,p and B2 ∈ R

p−s,n then

[

A1, A2

]

B1

B2

=[

A1B1 + A2B2

]

.


The general rule for 2 × 2 blocks

If A =

A11 A12

A21 A22

and B =

B11 B12

B21 B22

then

A11 A12

A21 A22

B11 B12

B21 B22

=

A11B11 + A12B21 A11B12 + A12B22

A21B11 + A22B21 A21B12 + A22B22

,

provided the vertical partition line in A matches the horizontal line inB, i.e. the number of columns in A11 and A21 equals the number ofrows in B11 and B12.


The general caseIf

A =

A11 · · · A1s

......

Ap1 · · · Aps

, B =

B11 · · · B1q

......

Bs1 · · · Bsq

,

and if all the matrix products in

Cij =

s∑

k=1

AikBkj , i = 1, . . . , p, j = 1, . . . , q

are well defined then

AB =

C11 · · · C1q

......

Cp1 · · · Cpq

.


Block-Triangular MatricesLemma 1. Suppose

A =

A11 A12

0 A22

where A, A11 and A22 are square matrices. Then A is nonsingular if and only if both

A11 and A22 are nonsingular. In that case

A−1 =

A−111 −A−1

11 A12A−122

0 A−122

(1)


Proof ⇐If A11 and A12 are nonsingular then

A−111 −A−1

11 A12A−122

0 A−122

A11 A12

0 A22

=

I 0

0 I

= I

and A is nonsingular with the indicated inverse.


Proof ⇒Conversely, let B be the inverse of the nonsingular matrix A. We partitionB conformally with A and have

BA =

B11 B12

B21 B22

A11 A12

0 A22

=

I 0

0 I

= I

Using block-multiplication we find

B11A11 = I, B21A11 = 0, B21A12 + B22A22 = I.

The first equation implies that A11 is invertible, this in turn implies thatB21 = 0 in the second equation, and then the third equation simplifies toB22A22 = I. We conclude that also A22 is invertible.


The inverseConsider now a triangular matrix.Lemma 2. An upper (lower) triangular matrix A = [aij ] ∈ R

n,n isnonsingular if and only if the diagonal entries aii, i = 1, . . . , n arenonzero. In that case the inverse is upper (lower) triangular with diagonal

entries a−1

ii , i = 1, . . . , n.

Proof: We use induction on n. The result holds for n = 1:The 1-by-1 matrix A = (a11) is invertible if and only if a11 6= 0

and in that case A−1 = (a−1

11). Suppose the result holds for

n = k and let A =∈ Rk+1,k+1 be upper triangular.


ProofWe partition A in the form

A =

Ak ak

0 ak+1,k+1

and note that Ak ∈ Rk,k is upper triangular. By Lemma 1.1 A isnonsingular if and only if Ak and (ak+1,k+1) are nonsingular and in thatcase

A−1 =

A−1k −A−1

k aka−1k+1,k+1

0 a−1k+1,k+1

.

By the induction hypothesis Ak is nonsingular if and only if the diagonalentries a11, . . . , akk of Ak are nonzero and in that case A−1

k is uppertriangular with diagonal entries a−1

ii , i = 1, . . . , k. The result for A follows.


Unit Triangular MatricesLemma 3. The product C = AB = (cij) of two upper(lower) triangular matrices

A = (aij) and B = (bij) is upper(lower) triangular with diagonal entries cii = aiibii

for all i.

Proof. Exercise.

A matrix is unit triangular if it is triangular with 1’s on the diagonal.

Lemma 4. For a unit upper(lower) triangular matrix A ∈ Rn,n:

1. A is invertible and the inverse is unit upper(lower) triangular.

2. The product of two unit upper(lower) triangular matrices is unit upper(lower)

triangular.

Proof. 1. follows from Lemma 1.2, while Lemma 1.3 implies 2.


LU-factorizationWe say that A = LU is an LU -factorization of A ∈ R

n,n if L ∈ Rn,n is

lower triangular and U ∈ Rn,n is upper triangular. In addition we will

assume that L is unit triangular.

Example 1. The equation

A =

2 −1

−1 2

=

1 0

−1/2 1

2 −1

0 3/2

gives an LU -factorization of the 2-by-2 matrix A.


ExampleEvery nonsingular matrix has a PLU -factorization, but not necessarily anLU -factorization.

Example 2. An LU -factorization of A =[

0 11 1

]

must satisfy the equation

0 1

1 1

=

1 0

l1 1

u1 u3

0 u2

=

u1 u3

l1u1 l1u3 + u2

for the unknowns l1 in L and u1, u2, u3 in U . Comparing (1, 1)-elements we see that

u1 = 0, which makes it impossible to satisfy the condition 1 = l1u1 for the (2, 1)

element. We conclude that A has no LU -factorization.


UniquenessTheorem 5. The LU -factorization of a nonsingular matrix is unique whenever it exists.

Proof. x

Suppose A = L1U1 = L2U2 are two LU -factorizations of the nonsingular

matrix A.

The equation L1U1 = L2U2 can be written in the form L−12 L1 = U2U

−11 ,

where by lemmas 1.2-1.4 L−12 L1 is unit lower triangular and U−1

2 U1 is upper

triangular.

But then both matrices must be diagonal with ones on the diagonal.

We conclude that L−12 L1 = I = U1U

−12 which means that L1 = L2 and

U1 = U2.


Leading Principal SubmatricesSuppose A ∈ C

n,n. The upper left k × k corners

Ak =

a11 · · · a1k

......

ak1 · · · akk

for k = 1, . . . , n

of A are called the leading principal submatrices of A


A LemmaThe following lemma will be used for existence.

Lemma 6. Suppose A = LU is the LU -factorization of A ∈ Rn,n. For k = 1, . . . , n

let Ak, Lk, Uk be the leading principal submatrices of A, L, U , respectively. Then

Ak = LkUk is the LU -factorization of Ak for k = 1, . . . , n.

Proof: We partition A = LU as follows:

A =

Ak Bk

Ck Dk

=

Lk 0

Mk Nk

Uk Vk

0 Wk

= LU , (2)

where Dk, Nk, Wk ∈ Rn−k,n−k.


ProofUsing block-multiplication we find the equations

Ak = LkUk (3)

Bk = LkVk (4)

Ck = MkUk (5)

Dk = MkVk + NkWk (6)

Since Lk is unit lower triangular and Uk is upper triangular we see that (3)gives the LU -factorization of Ak.


ExistenceTheorem 7. Suppose A ∈ R

n,n is nonsingular. Then A has an LU -factorization if and

only if the leading principal submatrices Ak are nonsingular for k = 1, . . . , n − 1.

Proof : Suppose A is nonsingular with the LU -factorization A = LU . SinceA is nonsingular it follows that L and U are nonsingular. By Lemma 6 wehave Ak = LkUk. Since Lk is unit lower triangular it is nonsingular.Moreover Uk is nonsingular since its diagonal entries are among thenonzero diagonal entries of U . But then Ak is nonsingular.


Proof continuedConversely, suppose A = An is nonsingular and Ak is nonsingular fork = 1, . . . , n − 1. We use induction on n to show that A has aLU -factorization. The result is clearly true for n = 1, since theLU -factorization of a 1-by-1 matrix is (a11) = (1)(a11). Suppose thatA1, . . . , An−1 are nonsingular implies that An−1 has an LU -factorization,and suppose that A1, . . . , An are nonsingular. To show that A = An hasa LU -factorization we consider (3)-(6) with k = n − 1. In this case Ck andMk are row vectors, Bk and Vk are column vectors, and Dk = (ann),Nk = (1), and Wk = (unn) are 1-by-1 matrices, i.e. scalars. TheLU -factorization of An−1 is given by (3), and since An−1 is nonsingularwe see that Ln−1 and Un−1 are nonsingular. But then (4) has a uniquesolution Vn−1, (5) has a unique solution Mn−1, and setting Nn−1 = (1) in(6) we obtain unn = ann − Mn−1Vn−1. Thus we have constructed anLU -factorization of A.


LDLT -factorizationFor a symmetric matrix the LU -factorization can be written in a specialform.

Definition 8. Suppose A ∈ Rn,n. A factorization A = LDLT , where L is unit lower

triangular and D is diagonal is called an LDLT -factorization.

Theorem 9. Suppose A ∈ Rn,n is nonsingular. Then A has an LDLT -factorization if

and only if A = AT and Ak is nonsingular for k = 1, . . . , n − 1.


ProofProof : If A1, . . . , An−1 are nonsingular then Theorem 7 implies that A hasan LU -factorization A = LU . Since A is nonsingular it follows that U isnonsingular and since U is triangular the diagonal matrixD = diag(u11, . . . , unn) is nonsingular (cf. Lemma 2). We can then factorA further as A = LDMT where MT = D−1U . It is easy to see that MT

is unit upper triangular and since AT = A we findAT = (LDMT )T = MDLT = LU = A. Now M (DLT ) and LU are twoLU -factorizations of A and by the uniqueness of the LU -factorization wemust have M = L. Thus A = LDMT = LDLT is an LDLT -factorizationof A. Conversely, if A = LDLT is an LDLT -factorization of A then A issymmetric since LDLT is symmetric and A has an LU -factorization withU = DLT . By Theorem 7 we conclude that A1, . . . , An−1 arenonsingular.


Quadratic FormsSuppose A ∈ R

n,n is a square matrix. The function f : Rn → R given

by

f(x) = xT Ax =n

∑

i=1

n∑

j=1

aijxixj

is called a quadratic form.

We say that A is

(i) positive definite if AT = A and xT Ax > 0 for all nonzero x ∈ Rn.

(ii) positive semidefinite AT = A and xT Ax ≥ 0 for all x ∈ Rn.

(iii) negative (semi-)definite if −A is positive (semi-) definite.


Some ObservationsA matrix is positive definite if it is positive semidefinite and in addition

xT Ax = 0 ⇒ x = 0. (7)

The zero-matrix is positive semidefinite;

a positive definite matrix must be nonsingular.

Indeed, if Ax = 0 for some x ∈ Rn then xT Ax = 0 which by (7)implies that x = 0.


ExampleT

Example 3. The 3-by-3 tridiagonal matrix

T = T 3 =

2 −1 0

−1 2 −1

0 −1 2

= tridiag3(−1, 2,−1) (8)

is positive definite. Clearly T is symmetric. Now it can be shown that

xT Tx = x21 + x2

3 + (x2 − x1)2 + (x3 − x2)

2.

Thus xT Tx ≥ 0 and if xT Tx = 0 then x1 = x3 = 0 and x1 = x2 = x3 which

implies that also x2 = 0. Hence T is positive definite.


ExampleB

Example 4. Let A = BT B, where B ∈ Rm,n and m, n are positive integers. (Note

that B can be a rectangular matrix). Since AT = (BT B)T = BT B we see that A is

symmetric. Moreover, for any x ∈ Rn

xT Ax = xT BT Bx = (Bx)T (Bx) =∥

∥Bx∥

∥

2

2. (9)

Since the Euclidian norm ‖ ‖2 of a vector is nonnegative this shows that A is positive

semidefinite and that A is positive definite if and only if B has linearly independent

columns.

Note that A and B have the same null-space and the same rank.


The Hessian MatrixExample 5. Suppose F (t) = F (t1, . . . , tn) is a real valued function of n variables

which has continuous 1. and 2. order partial derivatives for t in some domain Ω. For each

t ∈ Ω the gradient and Hessian of F are given by

∇F (t) =

∂F (t)∂t1

...

∂F (t)∂tn

∈ Rn, H(t) =

∂2F (t)∂t1∂t1

. . . ∂2F (t)∂t1∂tn

......

∂2F (t)∂tn∂t1

. . . ∂2F (t)∂tn∂tn

∈ Rn,n.

It is shown in advanced calculus texts that under suitable conditions on the domain Ω the

matrix H(t) is symmetric for each t ∈ Ω. Moreover if ∇F (t∗) = 0 and H(t∗) is

positive definite then t∗ is a local minimum for F . This can be shown using second-order

Taylor approximation of F . Moreover, t∗ is a local maximum if ∇F (t∗) = 0 and H(t∗)

is negative definite.


When is a Matrix Positive Definite?Not all symmetric matrices are positive definite, and sometimes wecan tell just by glancing at the matrix that it cannot be positivedefinite. Examples:

A1 =

0 1

1 1

, A2 =

1 2

2 2

, A3 =

−2 1

1 2

.

A1: If a diagonal entry aii ≤ 0 for some i then eTi Aei = aii ≤ 0 and

A is not positive definite.

A2: if the absolute value of the largest entry of A is not (only) on thediagonal then A is not positive definite.

To show this suppose aij ≥ aii and aij ≥ ajj for some i 6= j. Since A

is symmetric we obtain (ei − ej)T A(ei − ej) = aii + ajj − 2aij ≤ 0

which implies that xT Ax ≤ 0 for some x 6= 0.


SubmatricesTo give citeria for positive definiteness we start with twolemmas.Lemma 10. The leading principal submatrices of a positive definitematrix are positive definite and hence nonsingular.

Proof. Consider a leading principal submatrix Ak of the positive definite

matrix A ∈ Rn,n. Clearly Ak is symmetric. Let x ∈ R

k be nonzero, set

y = [ x0 ] ∈ R

n, and partition A conformally with y as A =[

Ak Bk

Ck Dk

]

,

where Dk ∈ Rn−k,n−k. Then

0 < yTAy = [xT0

T ]

[

Ak Bk

Ck Dk

] [

x

0

]

= xTAkx.


LU -factorizationLemma 11. A matrix is positive definite if and only if it has an LDLT -factorization with

positive diagonal entries in D.

Proof : It follows from Lemma 10 and 9 that A has an LDLT -factorizationA = LDLT . We need to show that the diagonal entries dii of D arepositive. With ei the ith unit vector we find

dii = eTi Dei = eT

i L−1AL−T ei = xTi Axi,

where xi = L−T ei is nonzero since L−T is nonsingular. Since A ispositive definite we see that dii = xT

i Axi > 0 for i = 1, . . . , n.


Proof of the converseConversely, if A has an LDLT -factorization with positive diagonal entriesin D then we can write

A = RT R, RT := LD1/2, (10)

where D1/2 = diag(d1/211 , . . . , d

1/2nn ). Since L and D1/2 are nonsingular it

follows that R is nonsingular and A is positive definite by Example 4.


Cholesky FactorizationThe factorization A = RT R in (10) is quite useful and has a special name.

Definition 12. A factorization A = RT R where R is upper triangular with positive

diagonal entries is called a Cholesky-factorization.

Example 6. The matrix A =[

2 −1−1 2

]

has an LDLT - and Cholesky-factorization given

by

2 −1

−1 2

=

1 0

−1/2 1

2 0

0 3/2

1 −1/2

0 1

=

√2 0

−1/√

2√

3/2

√2 −1/

√2

0√

3/2

.


Positive EigenvaluesLemma 13. A matrix is positive definite if and only if it is symmetric and has positive

eigenvalues.

Proof. x

If A is positive definite then by definition A is symmetric.

Suppose Ax = λx with x 6= 0. Multiplying both sides by xT and solving for λ we

find λ = xT

Ax

xT x> 0.

Suppose conversely that A ∈ Rn,n is symmetric with positive eigenvalues

λ1, . . . , λn. From the spectral theorem we have UT AU = D, where

UT U = UUT = I and D = diag(λ1, . . . , λn).

Let x ∈ Rn be nonzero and define c := UT x = [c1, . . . , cn]T . Then

cT c = xT UUT x = xT x so c is nonzero. Since x = Uc we find

xT Ax = (Uc)T AUc = cT UT AUc = cT Dc =∑n

j=1 λjc2j > 0 and it

follows that A is positive definite.


Necessary and Sufficient ConditionsTheorem 14. The following is equivalent for a symmetric matrix A ∈ R

n,n

1. A is positive definite.

2. A has only positive eigenvalues.

3.∣

∣

∣

∣

∣

∣

∣

∣

∣

a11 . . . a1k

......

ak1 . . . akk

∣

∣

∣

∣

∣

∣

∣

∣

∣

> 0 for k = 1, . . . , n

4. A = BT B for a nonsingular B ∈ Rn,n

Proof By Lemma 13 we know that 1 ⇔ 2. We show that 1 ⇒ 3 ⇒ 4 ⇒ 1.


Proof1 ⇒ 3: By Lemma 10 the leading principal submatrix Ak of A is positivedefinite, and hence has positive eigenvalues by Lemma 13. Since thedeterminant of a matrix equals the product of its eigenvalues we concludethat det(Ak) > 0 for k = 1, . . . , n.3 ⇒ 4: The condition det(Ak) > 0 implies that Ak is nonsingular fork = 1, . . . , n. By Theorem 9 A has an LDLT -factorization A = LDLT .Let Lk and Dk be the leading principal submatrices of order k of L andD, respectively. By partitioning A, L and D similarly to the proof ofLemma 6 we see that Ak = LkDkLT

k is the LDLT -factorization of Ak fork = 1, . . . , n. Using properties of determinants we finddet(Ak) = det(Lk) det(Dk) det(LT

k ) = det(Dk) = d11 . . . dkk > 0. Sincethis holds for k = 1, . . . , n we conclude that D has positive diagonalentries and we have A = BT B with B := R as in (10).4 ⇒ 1: This follows from the discussion in Example 4.


Useful factsSuppose A ∈ Rn,n is positive definite and let 〈·, ·〉 be the usual innerproduct on Rn.

1. A has a set of eigenvectors u1, . . . , un that form an orthonormalbasis for R

n.

2. For any x ∈ Rn we have x =

∑nj=1 cjuj , where cj = 〈x, uj〉 := xT uj .

3. For any x ∈ Rn we have Ax =

∑nj=1 λjcjuj , where λ1, . . . , λn are

the eigenvalues of A.

4. Furthermore 〈Ax, x〉 =∑n

j=1 λjc2j .

5. cond2(A) := ‖A‖2‖A−1‖2 = λmax

λmin

, where λmax and λmin are thelargest and smallest eigenvalue of A.


Proof1. By the spectral theorem it has a set of n orthonormal eigenvectors

u1, . . . , un that form an orthonormal basis for Rn .

2. Since u1, . . . , un form a basis for Rn we have x =

∑nj=1 cjuj for

some c1, . . . , cn. Taking the inner product with ui we find byorthonormality 〈x, ui〉 =

∑nj=1 cj〈uj , ui〉 = ci for i = 1, . . . , n.

3. We apply A to the equation x =∑n

j=1 cjuj .

4. 〈Ax, x〉 = 〈∑ni=1 λiciui,

∑nj=1 cjuj〉 =

∑nj=1 λjc

2j .

5. The eigenvalues of A are positive and since A is symmetric it isnormal and hence cond2(A) = |λmax|

|λmin| = λmax

λmin

.


ExampleFor a positive integer m and real numbers a, b we consider the m-by-mtridiagonal matrix given by

C := tridiagm(a, b, a) =

b a 0 . . . 0

a b a . . . 0

. . .. . .

. . .

0 . . . a b a

0 . . . a b

. (11)

Note that C is symmetric CT = C. We obtain the second derivative matrixwhen a = −1 and b = 2.


C is positive definite if b > 0 and b ≥ 2|Since C is symmetric it is enough to show that the smallesteigenvalue λmin is positive.

The eigenvalues are λj = b + 2a cos (jπh) for j = 1, . . . , m, whereh = 1/(m + 1).

For C to be positive definite it is necessary that the diagonal entryb > 0.

If b > 0 then λmin = b − 2|a| cos (πh) > b − 2|a| ≥ 0

Thus C is positive definite


Finding the Cholesky factorRTo solve a linear system Ax = b where A is positive definite we firstcompute the Cholesky-factorization A = RT R of A and then solve twotriangular systems RT y = b and Rx = y by forward and backwardsubstitution.Consider finding the Cholesky factorization of A. Since A = RT R and R

is upper triangular we find

akj =n

∑

i=1

rikrij =

min(j,k)∑

i=1

rikrij , j, k = 1, . . . , n. (12)


Cholesky Factorization AlgorithmSolving for rkj we find

rkk =(

akk −k−1∑

i=1

r2ik

)1/2,

rkj =(

akj −k−1∑

i=1

rikrij

)

/rkk j = k + 1, . . . , n.

(13)

for k = 1, 2, . . . , n

s = R(1:k−1, k); R(k, k) = (A(k, k) − sT ∗ s)1/2;

R(k, k+1:n) =(

A(k, k+1:n) − sT ∗ R(1:k−1, k+1:n))

/R(k, k);

end


#flopsThe number of flops needed for the Cholesky-factorization is given by

n∑

k=1

(2k−2+(2k−1)(n−k)) ≈n

∑

k=0

2k(n−k) ≈∫ n

0

2x(n−x)dx = n3/3.

In addition we need to take n square roots.

This is half the number of flops needed on Gaussian elimination of anarbitrary matrix.

We obtain this reduction since the Cholesky factorization takesadvantage of the symmetry of A.


Band MatrixIn many applications the matrix A has a banded structure, and thenumber of flops can be reduced.

We say that A has lower bandwidth p if aij = 0 whenever i > j + p,and upper bandwidth q if aij = 0 whenever j > i + q.

A diagonal matrix has upper and lower bandwidth zero,

a matrix with upper and lower bandwidth one is tridiagonal.

if A is symmetric then p = q.

It is easy to extend the algorithm to band-matrices.


A LemmaWe first show that if A = RT R then R has the same upper bandwidth asA.

Lemma 15. Suppose A is positive definite with Cholesky-factorization A = RT R. If

akj = 0 for j > k + d, then also rkj = 0 for j > k + d.

Proof. We show that if R has upper bandwidth d in its first k − 1 rows then row k also

has upper bandwidth d. The proof then follows by induction on k. Now, if j > k + d, then

akj = 0, and if R has upper bandwidth d in its first k − 1 rows then for i > k + d we

have rji = 0 for j = 1, . . . , k − 1. From(13) it follows that rki = 0 for i > k + d.


Banded Cholesky AlgorithmFull version:

for k = 1, 2, . . . , n

s = R(1:k−1, k); R(k, k) = (A(k, k) − sT ∗ s)1/2;

R(k, k+1:n) =(

A(k, k+1:n) − sT ∗ R(1:k−1, k+1:n))

/R(k, k);

end

Banded version:

for k = 1, 2, . . . , n

km = max(1, k − d); kp = min(n, k + d);

s = R(km:k−1, k); R(k, k) = sqrt(A(k, k) − sT ∗ s);

R(k, k+1:kp) =(

A(k, k+1:kp) − sT ∗ R(km:k−1, k+1:kp))

/R(k, k);

end


RTx = b with R upper bandwidth d

To solve Ax = b where A ∈ Rn,n is positive definite with bandwidth d we

can use the banded Cholesky Algorithm followed by a simple modificationof the forward and backward substitution algorithms. In the forwardsubstitution we use L = RT , but do the calculations using the entries inR:Algorithm 16. x

for k = 1 : n

km = max(1, k − d);

x(k) =(

b(k) − x(km:k−1)T ∗ R(km:k−1, k))

/R(k, k);

end


Rx = y with R upper bandwidth d

Algorithm 17. xfor k = n : −1 : 1

kp = min(n, k + d);

x(k) =(

y(k) − R(k, k+1:kp) ∗ x(k+1:kp))

/R(k, k);

end

The number of flops for these algorithms are:

O(2nd2) for the banded Cholesky

O(4nd) for backward and forward substitution.

When d is small compared to n we see that these numbers areconsiderably smaller than the O(n3/3) and O(2n2) counts for thefactorization of a full matrix.


LU-factorization and Positive Deﬁnite Matrices · Uniqueness Theorem 5. The LU-factorization of a...

Documents

Transcript of LU-factorization and Positive Deﬁnite Matrices · Uniqueness Theorem 5. The LU-factorization of a...