Ch10 LU Decomposition 21
Transcript of Ch10 LU Decomposition 21
-
8/18/2019 Ch10 LU Decomposition 21
1/34
The Islamic University of Gaza
Faculty of Engineering
Civil Engineering Department
Numerical Analysis
ECIV 3306
Chapter 10
LU Decomposition and Matrix Inversion
-
8/18/2019 Ch10 LU Decomposition 21
2/34
Introduction
• Gauss elimination solves [A] {x} ={B}
• It becomes insufficient when solving theseequations for different values of {B}
• LU decomposition works on the matrix [A] and thevector {B} separately.
• LU decomposition is very useful when the vector ofvariables {x} is estimated for different parametervectors {B} since the forward elimination process isnot performed on {B}.
-
8/18/2019 Ch10 LU Decomposition 21
3/34
LU Decomposition
If:L: lower triangular matrix
U: upper triangular matrix
Then,
[A]{X}={B} can be decomposed into two
matrices [L] and [U] such that:
1. [L][U] = [A] ([L][U]){X} = {B}
-
8/18/2019 Ch10 LU Decomposition 21
4/34
LU Decomposition
Consider:
[U]{X} = {D}
So, [L]{D} = {B}
2. [L]{D} = {B} is used to generate an
intermediate vector {D} by forward substitution 3. Then, [U]{X}={D} is used to get {X} by back
substitution
([L][U]){X} = {B}
-
8/18/2019 Ch10 LU Decomposition 21
5/34
Summary of LU Decomposition
-
8/18/2019 Ch10 LU Decomposition 21
6/34
LU Decomposition
As in Gauss elimination, LU decomposition must
employ pivoting to avoid division by zero and to
minimize round off errors. The pivoting is done
immediately after computing each column.
-
8/18/2019 Ch10 LU Decomposition 21
7/34
LU Decomposition
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
11 12 13
/ /
22 23
//
33
[ ] 0
0 0
a a a
U a a
a
21
31 32
1 0 0
[ ] 1 0
1
L l
l l
Step 1: Forward elimination
System of linear equations [A]{x}={B}
][]][[ AU L
11
2121
a
al
11
3131
a
al
'
'
32
22
32
a
al
-
8/18/2019 Ch10 LU Decomposition 21
8/34
LU Decomposition
Step 2: Generate an intermediate vector {D} by forward
substitution
Step 3: Get {X} by back substitution.
3
2
1
3
2
1
3231
21
101
001
bb
b
d d
d
l l l
3
2
1
3
2
1
33
2322
131211
''00
''0
d
d
d
x
x
x
a
aa
aaa
[L]{D} = {B}
[U]{X}={D}
-
8/18/2019 Ch10 LU Decomposition 21
9/34
LU Decomposition-Example
102030
307 10
20103
A
..
..
.. 3 0.1 0.2
0 7.003 0.293
0 0.19 10.02
21 31
\
3232 \
22
0.1 0.30.03333; 0.10003 3
0.190.02713
7.003
l l
al
a
0121000
29300037 0
20103
U
.
..
.. 1 0 0
[ ] 0.03333 1 0
0.1000 .02713 1
L
-
8/18/2019 Ch10 LU Decomposition 21
10/34
LU Decomposition-Example (cont’d)
4.71
3.19
85.7
1
102.03.0
3.071.0
2.01.03
3
2
1
x
x
x
0843.70
5617.1985.7
4.71
3.1985.7
102713.01000.0
010333.0001
3
2
1
3
2
1
d
d d
d
d d
Step 2: Find the intermediate vector {D} by forward substitution
Use previous [L] and {D} matrices to solve the system:
-
8/18/2019 Ch10 LU Decomposition 21
11/34
LU Decomposition-Example (cont’d)
Step 3: Get {X} by back substitution.
00003.7
5.2
3
0843.70
5617.19
85.7
012.1000
2933.00033.70
2.01.03
3
2
1
3
2
1
x
x
x
x
x
x
-
8/18/2019 Ch10 LU Decomposition 21
12/34
Decomposition Step
• % Decomposition Step
for k=1:n-1
[a,o]= pivot(a,o,k,n);
for i = k+1:n
a(i,k) = a(i,k)/a(k,k);a(i,k+1:n)= a(i,k+1:n)-a(i,k)*a(k,k+1:n);
end
end
-
8/18/2019 Ch10 LU Decomposition 21
13/34
Partial Pivoting
• %Partial Pivoting
function [a,o] = pivot(a,o,k,n)
[big piv]=max(abs(a(k:n,k)));
piv=piv+(k-1);
if piv ~= ktemp = a(piv,:);
a(piv,:)= a(k,:);
a(k,:)=temp;
temp = o(piv);
o(piv)=o(k);
o(k)=temp;
end
-
8/18/2019 Ch10 LU Decomposition 21
14/34
Substitution Steps
%Forward Substitution
d(1)=bn(1);
for i=2:n
d(i)=bn(i)-a(i,1:i-1)*(d(1:i-1))';
end
• % Back Substitution
x(n)=d(n)/a(n,n);
for i=n-1:-1:1
x(i)=(d(i)-a(i,i+1:n)*x(i+1:n)')/a(i,i);
end
-
8/18/2019 Ch10 LU Decomposition 21
15/34
Matrix Inverse Using the LU
Decomposition
• LU decomposition can be used to obtain the
inverse of the original coefficient matrix [A].
• Each column j of the inverse is determined by
using a unit vector (with 1 in the jth raw).
-
8/18/2019 Ch10 LU Decomposition 21
16/34
Matrix Inverse: LU Decomposition
0
0
1
}{
1
11
x A
b x A
0
1
0
}{
2
22
x A
b x A
1
0
0
}{
3
33
x A
b x A
1st column
of [A]-12nd column
of [A]-13rd column
of [A]-1
3211
x x x A }{}{}{
[A] [A]-1 = [A]-1[A] = I
-
8/18/2019 Ch10 LU Decomposition 21
17/34
Matrix inverse using LU decomposition Example
102030
307 1020103
A
..
..
..
0121000
29300037 020103
U
.
....1 0 0
[ ] 0.03333 1 0
0.1000 .02713 1
L
1009.0
03333.0
1
0
0
1
102713.01000.0
010333.0
001
3
2
1
3
2
1
d
d
d
d
d
d
1A. [L]{d}1 = {b}1
1B. Then, [U]{X}1={d}1
01008.0
00518.0
33249.0
1009.0
03333.0
1
012.1000
2933.00033.70
2.01.03
3
2
1
3
2
1
x
x
x
x
x
x
1st column
of [A]-1
-
8/18/2019 Ch10 LU Decomposition 21
18/34
2A. [L]{d}2 = {b}2
Matrix inverse using LU decomposition
Example (cont’d)
02713.0
1
0
0
1
0
102713.01000.0
010333.0
001
3
2
1
3
2
1
d
d
d
d
d
d
00271.0
142903.0
004944.0
02713.0
1
0
012.1000
2933.00033.70
2.01.03
3
2
1
3
2
1
x
x
x
x
x
x
2B. Then, [U]{X}2={d}22nd column
of [A]-1
-
8/18/2019 Ch10 LU Decomposition 21
19/34
3A. [L]{d}3 = {b}3
Matrix inverse using LU decomposition
Example (cont’d)
1
0
0
1
0
0
102713.01000.0
010333.0
001
3
2
1
3
2
1
d
d
d
d
d
d
09988.0
004183.0006798.0
1
00
012.1000
2933.00033.702.01.03
3
2
1
3
2
1
x
x x
x
x x
3B. Then, [U]{X}3={d}33rd column
of [A]-1
-
8/18/2019 Ch10 LU Decomposition 21
20/34
Matrix inverse using LU decomposition
Example (cont’d)
09988.000271.001008.0004183.0142903.000518.0
006798.0004944.033249.0
][
1
A
-
8/18/2019 Ch10 LU Decomposition 21
21/34
ERROR ANALYSIS AND SYSTEM CONDITION
The inverse provides a means to discern whethersystems are ill-conditioned.
1. Scale the matrix of coefficients [A] so that the largest
element in each row is 1. Invert the scaled matrix and if
there are elements of [A]−1
that are several orders ofmagnitude greater than one, it is likely that the system is ill-
conditioned.
2. Multiply the inverse by the original coefficient matrix and
assess whether the result is close to the identity matrix. If
not, it indicates ill-conditioning.
3. Invert the inverted matrix and assess whether the result is
sufficiently close to the original coefficient matrix. If not, it
again indicates that the system is ill-conditioned.
-
8/18/2019 Ch10 LU Decomposition 21
22/34
Vector and Matrix Norms
Norm is a real-valued function that provides a measure ofsize or “length” of vectors and matrices.
Norms are useful in studying the error behavior of
algorithms.
y.repectivelaxes,zandy,x,alongdistancesthearecand b,a,where
asdrepresente becanthat
spaceEuclideanldimensiona-in threevectoraisexamplesimpleA
cba F
-
8/18/2019 Ch10 LU Decomposition 21
23/34
Vector and Matrix Norms (cont’d)
-
8/18/2019 Ch10 LU Decomposition 21
24/34
Vector and Matrix Norms (cont’d)
• The length of this vector can be simply computed as
222 cba F e
n
i
ji
n
j
e
n
i
ie
n
a
x
x x x X
1
2
,
1
1
2
21
A
[A]matrixaFor
X
ascomputedisnormEuclideana
Length or Euclidean norm of [F]
• For an n dimensional vector
Frobenius norm
(10.24)
-
8/18/2019 Ch10 LU Decomposition 21
25/34
Vector and Matrix Norms (cont’d)
For vectors, there are alternatives called p norms
that can be represented generally by
1/
1
X
P-Norm p
p
n
i pi
x
We can also see that Euclidean norm and the 2
norm, ||X||2, are identical for vectors.
In contrast to vectors, the 2 norm and Euclidean
norm for a matrix are not the same.
-
8/18/2019 Ch10 LU Decomposition 21
26/34
Vector and Matrix Norms (cont’d)
• Uniform vector norm
• Uniform matrix norm (row sum Norm)
ini
xmax1X
n
j ji
ni a1 ,1maxA
11
,11 1
1-Norm
For Vector
CFor olumn Sum Norm
X
( )Matrix
A max
n
ii
n
i j j n i
x
a
-
8/18/2019 Ch10 LU Decomposition 21
27/34
Vector and Matrix Norms (cont’d)
• Matrix Condition Number Defined as:
• For a matrix [A], this number will be greater than or
equal to 1.
• Relative error of the norm
• For example, if the coefficients of [A] are known to
t -digit precision (rounding errors~10-t) and
Cond [A]=10c, the solution [X] may be valid to only
t-c digits (rounding errors~10c-t).
1 A A ACond
A
A ACond
X
X
-
8/18/2019 Ch10 LU Decomposition 21
28/34
The condition number is considerably greater than
unity suggests that the
system is ill-conditioned.
-
8/18/2019 Ch10 LU Decomposition 21
29/34
Iterative Refinement
• Round-off errors can be reduced by the following procedure:
• Suppose an approximate solution vectors given by
• Substitute the result in the original system
....(Eq.1)
]~~~[~
321 x x x X T
3333232131
2323222121
1313212111
~~~~
~~~~
~~~~
b xa xa xa
b xa xa xab xa xa xa
3333232131
2323222121
1313212111
b xa xa xa
b xa xa xa
b xa xa xa
-
8/18/2019 Ch10 LU Decomposition 21
30/34
Iterative Refinement (cont’d)
• Now, assume the exact solution is:
• Then:
… …….(Eq.2)
333
222
111
~
~
~
x x x
x x x
x x x
3333322321131
2332322221121
1331322121111
)~()~()~(
)~()~()~(
)~()~()~(
b x xa x xa x xa
b x xa x xa x xa
b x xa x xa x xa
-
8/18/2019 Ch10 LU Decomposition 21
31/34
Iterative Refinement (cont’d)
Subtract Eq.2 from Eq.1, the result is a system of linear
equations that can be solved to obtain the correction factors
The factors then can be applied to improve the solution as
specified by the equation:
333333232131
222323222121
111313212111
~
~
~
E bb xa xa xa
E bb xa xa xa
E bb xa xa xa
x
333
222
111
~
~
~
x x x
x x x
x x x
-
8/18/2019 Ch10 LU Decomposition 21
32/34
Iterative Refinement- Example (cont’d)
Solve:
The exact solution is ………
1- Solve the equations using [A]-1, such as {x}=[A]-1{c}
]00.100.100.1[T X
]00.1997.0991.0[~ T X
58.232.211.285.1
22.598.077.653.2
28.511.206.123.4
321
321
321
x x x
x x x
x x x
-
8/18/2019 Ch10 LU Decomposition 21
33/34
Iterative Refinement - Example
(cont’d)
2- Substitute the result in the original system [A]{x}={c}
}
~
{}
~
]{[ C X A
59032.222246.5
24511.5
}~
{C
0103.0
00246.0
0348.0
59032.258.2
22246.522.5
24511.528.5
}~
{}{ C C E
-
8/18/2019 Ch10 LU Decomposition 21
34/34
Iterative Refinement - Example
(cont’d)
3- Solve the system of linear equations
using [A]-1 to find the correction factors to yield
}{}~
]{[ E X A
x
]00000757.000300.000822.0[ T X
00.1~00.1~
999.0~
333
222
111
x x x x x x
x x x