Simultaneous Linear Equations

3/7/2008 http://numericalmethods.eng.usf.edu

1

Topic: Gaussian Elimination

Gaussian Elimination

One of the most popular techniques for solving simultaneous linear equations of the form [ ][ ] [ ]CXA =

Consists of 2 steps

1. Forward Elimination of Unknowns.

2. Back Substitution

Forward Elimination

The goal of Forward Elimination is to transform the coefficient matrix into an Upper Triangular Matrix

1525 ���� 1525

7.00056.18.40

1525

���

�

�

���

�

�

−−→���

�

�

���

�

�

11214418641525

Forward Elimination

Linear EquationsA set of n equations and n unknowns

11313212111 ... bxaxaxaxa nn =++++ 11313212111 nn

22323222121 ... bxaxaxaxa nn =++++

nnnnnnn bxaxaxaxa =++++ ...332211

. .

. .

. .

Forward Elimination

Transform to an Upper Triangular MatrixStep 1: Eliminate x1 in 2nd equation using equation 1 as the pivot equation

1Eqn ��)(

121

11

aa

Eqn ��

���

�

Which will yield

111

211

11

21212

11

21121 ... b

aa

xaaa

xaaa

xa nn =+++

Forward Elimination

Zeroing out the coefficient of x1 in the 2nd equation.Subtract this equation from 2nd equation

121

2121

221221

22 ... baa

bxaaa

axaaa

a nnn −=���

�

�−++��

�

�

�− 1

1121

112212

1122 aaa nnn �

��

��

�

'2

'22

'22 ... bxaxa nn =++

nnn aaa

aa

aaa

aa

111

212

'2

1211

2122

'22

−=

−=

�

Or Where

Forward Elimination

Repeat this procedure for the remaining equations to reduce the set of equations as

11313212111 ... bxaxaxaxa nn =++++ 11313212111 ... bxaxaxaxa nn =++++'2

'23

'232

'22 ... bxaxaxa nn =+++

'3

'33

'332

'32 ... bxaxaxa nn =+++

''3

'32

'2 ... nnnnnn bxaxaxa =+++

. . .

. . .

. . .

Forward Elimination

Step 2: Eliminate x2 in the 3rd equation.Equivalent to eliminating x1 in the 2nd equation using equation 2 as the pivot equation.

)(2

3 3222

aa

EqnEqn �

�

���

�−

Forward Elimination

This procedure is repeated for the remaining equations to reduce the set of equations as

11313212111 ... bxaxaxaxa nn =++++'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"33

"33 ... bxaxa nn =++

""3

"3 ... nnnnn bxaxa =++

. .

. .

. .

Forward Elimination

Continue this procedure by using the third equation as the pivot equation and so on.

At the end of (n-1) Forward Elimination steps, the system of equations will look like:

'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"3

"33 ... bxaxa nn =++

( ) ( )11 −− = nnn

nnn bxa

. .. .. .

11313212111 ... bxaxaxaxa nn =++++

Forward Elimination

At the end of the Forward Elimination steps

���

���

���

���

���

���

2

1

n

n

b

b

x

x

aaa

aaaa'2

1'2

'23

'22

1131211

�

�

�����

������

�

=

�����

������

������

������

�− )-(n

nn

3

2

nnn

n

n

b

b

b

x

x

x

a

aa

aaa

1

"3

2

)1(

"3

"33

22322

����

�

�

Back Substitution

The goal of Back Substitution is to solve each of the equations using the upper triangular matrix.

��

��

��

��

��

�� 11131211 x baaa

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

3

2

1

3

2

1

33

2322

131211

xx

000

b

b

a

aa

Example of a system of 3 equations

Back Substitution

Start with the last equation because it has only one unknown

)1( −

=n

nbx )1( −=

nnn

nn a

bx

Solve the second from last equation (n-1)th

using xn solved for previously.

This solves for xn-1.

Back Substitution

Representing Back Substitution for all equations by formula

( ) ( )11

+=

−− −n

ji

iji

i xab

( )1

1

−+= −

=i

ii

ijjiji

i a

xabx For i=n-1, n-2,….,1

and

)1(

)1(

−

−

=n

nn

nn

n ab

x

Example: Rocket Velocity

The upward velocity of a rocket is given at three different times

Time, t Velocity, v

s m/s

5 106.85 106.8

8 177.2

12 279.2

The velocity data is approximated by a polynomial as:

( ) 12.t5 , 322

1 ≤≤++= atatatv

Find: The Velocity at t=6,7.5,9, and 11 seconds.

Example: Rocket Velocity

Assume( ) 12.t5 ,atatatv ≤≤++= 32

21

��

��

��

��

��

�� 11

21 1 vatt 1

Results in a matrix template of the form:

���

����

�

=���

����

����

����

� 3

2

323

222

11

v

v

a

a

tt

tt

3

2

Using date from the time / velocity table, the matrix becomes:

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

2.2792.1778.106

11214418641525

3

2

1

a

a

a

Example: Rocket Velocity

Forward Elimination: Step 1

=���

��

�− )64(25

12

RowRow

YieldsYields

���

�

�

���

�

�

−=���

�

�

���

�

�

���

�

�

���

�

�

−−2.27921.9681.106

aaa

11214456.18.40

1525

3

2

1

Example: Rocket Velocity

=���

��

�− )144(25

13

RowRow

Yields

Forward Elimination: Step 1

���

�

�

���

�

�

−−=

���

�

�

���

�

�

���

�

�

���

�

�

−−−−

0.33621.968.106

aaa

76.48.16056.18.40

1525

3

2

1

Yields

Example: Rocket Velocity

Yields

=−×���

��

�

−− )8.16(

8.42

3Row

Row

Forward Elimination: Step 2

Yields

���

�

�

���

�

�

−=���

�

�

���

�

�

���

�

�

���

�

�

−−735.0

21.968.106

aaa

7.00056.18.40

1525

3

2

1

This is now ready for Back Substitution

Example: Rocket Velocity

Back Substitution: Solve for a3 using the third equation

735.07.0 3 =a

7350.70

7350.

.a 3 =

0501. a 3 =

Example: Rocket Velocity

Back Substitution: Solve for a2 using the second equation

21.9656.18.4 32 −=−− aa

56.121.96 +− a8.4

56.121.96 32 −

+−=

aa

( )84

05015612196.-

...-a 2

+=

7019.a 2 =

Example: Rocket Velocity

Back Substitution: Solve for a1 using the first equation

8.106525 321 =++ aaa

58.106 32 aaa

−−=

2558.106

321

aaa

−−=

( )25

050.170.1958.1061

−−=a

2900.01 = a

Example: Rocket Velocity

Solution:The solution vector is

���

�

�

���

�

�

=���

�

�

���

�

�

050.170.19

2900.0

3

2

1

a

a

a

The polynomial that passes through the three data points is The polynomial that passes through the three data points is then:

( ) 322

1 atatatv ++=

125 ,050.170.192900.0 2 ≤≤++= ttt

Example: Rocket Velocity

Solution:Substitute each value of t to find the corresponding velocity

( ) ( ) ( )./ 69.129

050.1670.1962900.06 2

sm

v

=++=

( ) ( ) ( ).s/m1.165

050.15.770.195.72900.05.7v 2

=++=

( ) ( ) ( ).s/m8.201

050.1970.1992900.09v 2

=++=

( ) ( ) ( ).s/m8.252

050.11170.19112900.011v 2

=++=

./ 69.129 sm=

Pitfalls

Two Potential Pitfalls-Division by zero: May occur in the forward elimination steps. Consider the set of equations:

7710 32 =− xx

655

901.33099.26

7710

321

321

32

=+−=++

=−

xxx

xxx

xx

- Round-off error: Prone to round-off errors.

Pitfalls: ExampleConsider the system of equations:

Use five significant figures with chopping

���

�

�

���

�

�

−−

−

5156099.230710

���

�

�

���

�

�

2

1

x

x

x

���

�

�

���

�

�

6901.37

=���� − 515 ���� 3x ���� 6

At the end of Forward Elimination

���

�

�

���

�

�

−−

15005006001.000710

���

�

�

���

�

�

3

2

1

x

x

x

���

�

�

���

�

�

15004001.67

=

Pitfalls: Example

Back Substitution

99993.01500515004

3 ==x

��

��

=��

����

��

−−

001.67

6001.000710 1

x

x

5.1 001.0

6001.6 32 −=

−−= x

x

3500.010

077 321 −=

−+=

xxx

���

����

�

=���

����

����

����

�

−15004

001.61500500

6001.00

3

2

x

x

Pitfalls: Example

Compare the calculated values with the exact solution

[ ] ���

���

−=���

���

= 10

2

1

x

x

X exact

[ ]���

�

�

���

�

�

−−

=���

�

�

���

�

�

=99993.0

5.135.0

3

2

1

x

x

x

X calculated

[ ]����

��

−=����

��

=11

3

2

x

xX exact

Improvements

Increase the number of significant digitsDecreases round off error

Does not avoid division by zero

Gaussian Elimination with Partial PivotingAvoids division by zero

Reduces round off error

Partial Pivoting

Gaussian Elimination with partial pivoting applies row switching to normal Gaussian Elimination.

How?

At the beginning of the kth step of forward elimination, find the maximum of

pka

At the beginning of the k step of forward elimination, find the maximum of

nkkkkk aaa .......,,........., ,1+

If the maximum of the values is In the pth row, ,npk ≤≤

then switch rows p and k.

Partial Pivoting

What does it Mean?

Gaussian Elimination with Partial Pivoting ensures that each step of Forward Elimination is performed with the pivoting element |akk| having the largest absolute value.

Partial Pivoting: Example

Consider the system of equations

6x5xx5

901.3x3x099.2x3

7x7x10

321

321

21

=+−=++−

=−

6x5xx5 321 =+−

In matrix form

���

�

�

���

�

�

−−

5156099.230710

���

�

�

���

�

�

3

2

1

x

x

x

���

�

�

���

�

�

6901.37

=

Solve using Gaussian Elimination with Partial Pivoting using five significant digits with chopping

Partial Pivoting: Example

Forward Elimination: Step 1

Examining the values of the first column

|10|, |-3|, and |5| or 10, 3, and 5

The largest absolute value is 10, which means, to follow the rules of Partial Pivoting, we switch row1 with row1.

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

−−

6901.37

5156099.230710

3

2

1

x

x

x

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

−−

5.2001.67

55.206001.000710

3

2

1

x

x

x

�

Performing Forward Elimination

Partial Pivoting: Example

Forward Elimination: Step 2

Examining the values of the first column

|-0.001| and |2.5| or 0.0001 and 2.5

The largest absolute value is 2.5, so row 2 is switched with row 3

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

−−

5.2001.67

55.206001.000710

3

2

1

x

x

x

����

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

�

−

−

001.65.2

7

6001.0055.200710

3

2

1

x

x

x

Performing the row swap

Partial Pivoting: Example

Forward Elimination: Step 2

Performing the Forward Elimination results in:

���

�

�

���

�

�

=���

�

�

���

�

�

���

�

�

���

�

� −

002.65.2

7

002.60055.200710

3

2

1

x

x

x

Partial Pivoting: Example

Back Substitution

Solving the equations through back substitution

1002.6002.6

3 ==x

���

���

=���

���

���

��� −

5.27

55.200710 1

x

x002.6

15.255.2 2

2 =−

=x

x

010

077 321 =

−+=

xxx

���

����

�

=���

����

����

����

� 002.65.2

002.60055.20

3

2

x

x

Partial Pivoting: Example

Compare the calculated and exact solution

The fact that they are equal is coincidence, but it does illustrate the advantage of Partial Pivoting

[ ]���

�

�

���

�

�

−=���

�

�

���

�

�

=11

0

3

2

1

x

x

x

X exact[ ]���

�

�

���

�

�

−=���

�

�

���

�

�

=11

0

3

2

1

x

x

x

X calculated

Summary

-Forward Elimination

-Back Substitution

-Pitfalls

-Improvements

-Partial Pivoting