Post on 08-Apr-2018
8/7/2019 Lecture3(Linear Equation)
1/75
1
System of Linear EquationsSystem of Linear Equations
This presentation covers
explanations for these two topics
shown, together with worked
examples.DefinitionDefinition
Solving Linear EquationsSolving Linear Equations
8/7/2019 Lecture3(Linear Equation)
2/75
2
DefinitionDefinition
8/7/2019 Lecture3(Linear Equation)
3/75
3
DefinitionDefinitionLinear equation: an equation of the form ax+by=0 where a and bare not both zero.
Linear system of equations: a system of equations such as
is a linear system of equations.
Both equations must be considered together.
Linear system can either be of two, three or more variables.
!
!
feydx
cbyax
8/7/2019 Lecture3(Linear Equation)
4/75
4
DefinitionDefinition
Think back to linear equations. For instance,
consider the linear equation
y = 3x 5. A "solution" to this equation was
anyx,y-point that "worked" in the equation.So (2, 1) was a solution because, plugging in
2 forx:
3x 5 = 3(2) 5 = 6 5 = 1 =y
On the other hand, (1, 2) was not a solution,
because, plugging in 1 forx:
3x 5 = 3(1) 5 = 3 5 = 2
8/7/2019 Lecture3(Linear Equation)
5/75
5
Now consider the following two-variable
system of linear equations:y= 3x 2y= x 6
Since the two equations
above are in a system, we
deal with them together at
the same time. In particular,
we can graph themtogether on the same axis
system, like this.
8/7/2019 Lecture3(Linear Equation)
6/75
6
Now consider the following two-variable
system of linear equations:
y= 3x 2y= x 6
Solution for a single equationis any point that lies on the line
for that equation. A solution for
a system of equations is any
point that lies on each line in
the system. For example, thered point at right is not a
solution to the system,
because it is not on either line.
8/7/2019 Lecture3(Linear Equation)
7/75
7
Now consider the following two-variable
system of linear equations:
y= 3x 2y= x 6
The blue point at right is not asolution to the system,
because it lies on only one of
the lines, not on both of them.
8/7/2019 Lecture3(Linear Equation)
8/75
8
Now consider the following two-variable
system of linear equations:
y= 3x 2y= x 6
The purple point at right is asolution to the system,
because it lies on both of the
lines.
8/7/2019 Lecture3(Linear Equation)
9/75
9
Solving Linear System in TwoSolving Linear System in Two
VariablesVariables
Graphing
Substitution method
Addition method (also known as elimination
method)
8/7/2019 Lecture3(Linear Equation)
10/75
8/7/2019 Lecture3(Linear Equation)
11/75
11
GraphingGraphing
The first graph shows two distinct non-parallel lines
that cross at exactly one point. This is called an
"independent" system of equations, and the solution is
always some x,y-point.
8/7/2019 Lecture3(Linear Equation)
12/75
12
GraphingGraphing
The second graph shows two distinct lines that are parallel.Since parallel lines never cross, then there can be nointersection; that is, for parallel lines, there can be nosolution. This is called an "inconsistent" system of
equations, and it has no solution.
8/7/2019 Lecture3(Linear Equation)
13/75
13
GraphingGraphing
The third graph appears to show only one line. Actually, it'sthe same line drawn twice. These "two" lines, really being thesame line, then "intersect" at every point along their length.This is called a "dependent" system, and the "solution" is thewhole line.
8/7/2019 Lecture3(Linear Equation)
14/75
14
Example 1Example 1
Solve the following system by graphing.2x 3y=24x+ y= 24
Solution
8/7/2019 Lecture3(Linear Equation)
15/75
15
Example 1Example 1
Solution (5,4)
8/7/2019 Lecture3(Linear Equation)
16/75
8/7/2019 Lecture3(Linear Equation)
17/75
17
Example 3Example 3
Solve the following system by graphing.7x+ 2y = 16
21x 6y = 24
Solution
8/7/2019 Lecture3(Linear Equation)
18/75
18
SummarySummary
i. Solve both equations fory
ii. Compare the slopes to decide how many solutionsthe system has
iii. If the system has one solutions graph the two linesin the same plane
iv. Identify the point of intersection
v. Check the point in both equations
To solve a linear system in two variable bygraphing
8/7/2019 Lecture3(Linear Equation)
19/75
19
SummarySummary
A linear system in two variables may have one solution,no solution, or infinitely many solutions.
We use the slope andy-intercepts of the givenequations to determine how many solutions a system
has: Different slopes one solution
Same slopes, differenty-int no solutions
Same slopes, samey-int infinite many solutions
8/7/2019 Lecture3(Linear Equation)
20/75
20
SummarySummary
When graphing to determine the solutions tothe system: First solve fory
Second compare the slopes to determine howmany solutions
Third if one solution graph both lines on thesame plane
Fourth identify point of intersection Lastly check solution in both equations
8/7/2019 Lecture3(Linear Equation)
21/75
21
Solving by SubstitutionSolving by SubstitutionT
his method works by solving one of the equations forone of the variables, and then plugging this into the other
equation, "substituting" for the chosen variable and
solving for the other. Then back-solve for the first
variable.
Example 4Example 4
Solve the following system by substitution.
2x 3y= 2
4x+ y= 24
8/7/2019 Lecture3(Linear Equation)
22/75
22
4x+y= 24y= 4x+24
Substitute for "y" in the first equation, and solve forx:
2x 3(4x+24) = 22x+ 12x 72 = 2
14x= 70x= 5
Solution
y= 4(5) + 24 = 20 + 24 = 4
Then the solution is (x, y) = (5, 4).
8/7/2019 Lecture3(Linear Equation)
23/75
23
The addition method is also called the method of
elimination. If we have the equation "x + 6 = 11", you
would write "6" under either side of the equation, and
add down to get "x=
5" as the solution.
x+ 6 =11
6 6
x = 5
Solving by AdditionSolving by Addition
8/7/2019 Lecture3(Linear Equation)
24/75
24
Solve the following system using addition.
2x+ y= 93x y= 16
Example 5Example 5
Solution
2x+ y= 9
3x y=16
5x = 25
with x = 5, and then back solve, using either of the original
equations, to find the value ofy. Using the first equation:2(5) + y= 9
10 + y= 9
y= 1
Then the solution is (x, y)
=(5,
1).
8/7/2019 Lecture3(Linear Equation)
25/75
25
Solve the following using addition.
12x 3y= 6
4x y= 2
Exercise 1Exercise 1
8/7/2019 Lecture3(Linear Equation)
26/75
26
Solving Linear System in Three orSolving Linear System in Three or
More VariablesMore VariablesMethods for solving linear equations of three variables:
Direct methods: find the exact solution in a finitenumber of steps
Iterative methods: produce a sequence a sequenceof approximate solutions hopefully converging to the
exact solution.
Matrix algebra is used to solve a system of linear
equations.
8/7/2019 Lecture3(Linear Equation)
27/75
27
Linear SystemsLinear Systems
/
.
.
.
3333232131
2323222121
1313212111
bxaxaxa
bxaxaxa
bxaxaxa
!
!!
!
//1///
.
.
.
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
8/7/2019 Lecture3(Linear Equation)
28/75
28
Matrix AlgebraMatrix Algebra
System ofm linear equations in n unknowns
Matrix-vector notation Extended coefficient matrix
8/7/2019 Lecture3(Linear Equation)
29/75
29
Solving Linear Systems
Solve Ax=b, where Ais an nvn matrix andb is an nv1 column vector
Can also talk about non-square systems
whereA is mvn, b is mv1, andxis nv1
Overdeterminedifm>n:more equations than unknowns
Underdeterminedifn>m:more unknowns than equationsCan look for best solution using least squares
8/7/2019 Lecture3(Linear Equation)
30/75
30
Solving Linear Systems
Recap from Lecture 2:
1. Inverting matrix
2. Cramers Rule
8/7/2019 Lecture3(Linear Equation)
31/75
31
Solving Linear Systems
Recap from Lecture 2:
Inverting matrix
Usually not a good idea to computex=A-1b
Inefficient
Prone to round off error
8/7/2019 Lecture3(Linear Equation)
32/75
32
Echelon Form of a Matrix
An mxn matrix A is said to be row echelon form, if it satisfiesthe following properties:
1. All zero rows, if there any, appear at the bottom of the
matrix.2. Each leading entry (or the first nonzero entry from the
left) of a row is in a column to the right of the leadingentry of row above it.
3. All entries in a column below a leading entry are zeros.
8/7/2019 Lecture3(Linear Equation)
33/75
33
Echelon Form of a Matrix
Matrices in echelon form:
000000
0000
0
000
0
00
000 *
****
,*
**
,*
**
,
**
x
x
x
x
x
x
x
x
x
(x) may have any nonzero value and the entries (*) may
have any value including zero
8/7/2019 Lecture3(Linear Equation)
34/75
34
Reduced Echelon Form of a Matrix
An mxn matrix A is said to be reduced row echelon form, ifit satisfies the following properties:
1. The first entry from the left of a nonzero row is a 1. Thisentry is called a leading one of its row.
2. All entries above and below a leading 1 are zeros.
000000
010000
010
000
10
01
100
010
001
100
01***
,*
*
,,*
8/7/2019 Lecture3(Linear Equation)
35/75
35
Elementary Row operation Matrix
An elementary row operation on a matrix A is any one ofthe following operations:
1. Type I: Interchange any two rows..2. Type II: Multiply a row by a nonzero number
3. Type III: Add a multiple of one row to another.
An mxn matrix B is said to be row equivalent to an mxn
matrix AifB can be obtained by applying a finite sequenceof elementary row operations to A.
8/7/2019 Lecture3(Linear Equation)
36/75
36
Echelon Matrix
!
*
*
*
*
000000
**0000
*****0
******
A
Free variablesFree variables
8/7/2019 Lecture3(Linear Equation)
37/75
37
Reduced Row Echelon Matrix
Free variables
!
1
00
0
000000
*10000*0**10
*0**01
A
Free variables
8/7/2019 Lecture3(Linear Equation)
38/75
38
Example 6
B
A
!
!
130
580
122
212
580
122
212
226
122
212
113
122
212
122
123
~
~
~
~
R1R2
2R2R2
-3R1+R2R2
R1+R3R3
A and B are row equivalentA and B are row equivalent
8/7/2019 Lecture3(Linear Equation)
39/75
39
Steps in Row Reduction-Pivoting
1. Begin with the leftmost nonzero column. This is called pivotcolumn.
2. Select a nonzero entry (having the smallest absolute value) in
the pivot column as a pivot element. If a pivot element not atpivot position then use interchange row operations to movethis entry into pivot position.
3. Perform row reduction into row echelon form(Obtain 0 below the pivot element using row replacement operations byadding suitable multiple of the top row to the row below that)
and row reduction into reduced row echelon form.(Obtain 0 above and below the pivot element using row replacement by
adding suitable multiple of the top row to the row below it)
4. Repeat (1) to (3) on the matrix consisting of the remainingrows.
8/7/2019 Lecture3(Linear Equation)
40/75
40
Example 7
!
2342
3211
8312
3113
A
Apply elementary row operations to transform thefollowing matrix into echelon form
Solution
1. Compute the vector for checking column.
2. Follow the aforementioned steps.
8/7/2019 Lecture3(Linear Equation)
41/75
41
Example 7
!
0000
4100
8120
3211
A
Row echelon form of matrixA
8/7/2019 Lecture3(Linear Equation)
42/75
8/7/2019 Lecture3(Linear Equation)
43/75
43
The rank is the number of the pivots ofA, which is
also the same as the number of nonzero rows of
an echelon form ofA. To compute it, we reduceA
to echelon form and count the number of nonzero
rows or the number of pivot columns.
Rank of a MatrixRank of a Matrix
Example 8Compute the rank of the following matrix
!
321
010
123
A
8/7/2019 Lecture3(Linear Equation)
44/75
44
Example 8Example 8
Solution
!
800
010
321
840
010
321
123
010
321
321
010
123
~
~
~
A
There are 3nonzero rows,
hence the rank
(A)=3
8/7/2019 Lecture3(Linear Equation)
45/75
45
Exercise 3Exercise 3
!
420
121
210
A
Find the rank of matrixA
8/7/2019 Lecture3(Linear Equation)
46/75
46
Solving Linear Systems
Gaussian Elimination
Gauss Jordan Elimination
8/7/2019 Lecture3(Linear Equation)
47/75
47
Gauss Elimination Method
Form the augmented matrix (Ab) corresponding to Ax=b.
Transform augmented matrix (A b) to row echelon matrix(Ud).
Solution (if any)xto the system obtained by solving the
linear system Ux=dcorresponding to (U d) using backwardsubstitution.
8/7/2019 Lecture3(Linear Equation)
48/75
48
Gauss Elimination Method
Solve the following linear system using Gauss elimination.
a. x1+2x2+3x3=62x1-3x2+2x3=14
3x1+x2-x3=-2
b. x1+2x2+3x3=64x1+5x2+6x3=24
2x1+7x2-12x3=-2
c. 3x1-5x2+2x3=6x1+2x2-x3=1-x1+9x2-4x3=-4
8/7/2019 Lecture3(Linear Equation)
49/75
49
Gauss Elimination Method
Solution:
a. Unique solution (r=n)
b. No solution
c. Infinite solution r
8/7/2019 Lecture3(Linear Equation)
50/75
50
Solving Linear Systems
Gaussian Elimination Method for Solving M x = b
A Direct MethodFinite Termination for exact result (ignoring round off)
Produces accurate results for a broad range of
matrices
Computationally Expensive
8/7/2019 Lecture3(Linear Equation)
51/75
8/7/2019 Lecture3(Linear Equation)
52/75
8/7/2019 Lecture3(Linear Equation)
53/75
53
Gauss-Jordan Elimination Method
Solution:
a. Unique solution (r=n)
(Also unique solution ifA
0)
b. No solution
c. Infinite solution r
8/7/2019 Lecture3(Linear Equation)
54/75
54
Consistency of Solutions
The linear system of equations Ax=b has asolution, or said to be consistent IFF
Rank{A}=Rank{A|b}
A system is inconsistent whenRank{A}
8/7/2019 Lecture3(Linear Equation)
55/75
55
LU Factorization Method
Variant of Gaussian elimination that decomposes a matrix asa product of a lower triangular and an upper triangular matrix.
Widely used method on computer for solving a linear system.
When Uis an upper triangular matrix all of whose diagonal
entries are different from zero, then the linear system UX=B
can be solved without transforming the augmented matrix
[UB] to reduced row echelon form.
This is the preferred general method for solving linearThis is the preferred general method for solving linear
equations.equations.
8/7/2019 Lecture3(Linear Equation)
56/75
56
LU Factorization Method
nb
MMMM
bMMM
buu
buuu
...
...
...
...
...
000
0
3
22322
1131211
The solution is obtained by the following algorithm
.,,...,,, 121
1
11
11
1
!
!
!
!
!
nnju
xub
x
u
xubx
u
bx
jj
j
nk
kjkj
j
nn
nnnnn
nn
nn
This is merely back substitution
8/7/2019 Lecture3(Linear Equation)
57/75
57
LU Factorization Method
nnnnnnb
M
b
b
b
llll
MMMM
lll
ll
l
3
2
1
321
333231
2221
11
0
00
000
...
...
...
...
...
In similar manner, ifL is a lower triangular matrix of whose all diagonalentries are different from zero, then the linear system LX=B can be solved
by forward substitution.
.,...,, njl
xlb
x
lxlbx
l
bx
jj
j
j
j
j 2
1
1
22
1212
2
11
1
1
!
!
!
!
!
The solution is given by
8/7/2019 Lecture3(Linear Equation)
58/75
58
Example 9
Solve the linear system
134
3226
102
428
25
10
21
1
2
1
!
!
!
!
!!
xxx
x
x
x
j
5x1 =10
4x1-2x2 =28
2x1+3x2+4x3 =26
Solution
8/7/2019 Lecture3(Linear Equation)
59/75
8/7/2019 Lecture3(Linear Equation)
60/75
60
LU Factorization Method
!
)(
)(
1
)(
11
)3(
3
)3(
33
)2(
2
)2(
23
)2(
22
)1(
1
)1(
13
)1(
12
)1(
11
4,3,2,1,
3,12,11,1
2,31,3
1,2
0000
000
00
0
1
1
0
001
0001
00001
n
nn
n
nn
n
nn
n
n
n
nnnn
nnn
a
aa
aa
aaa
aaaa
mmmm
mmm
mm
m
A////
.
.
.
.
/.
///
.
.
.
Compact storage: The diagonal entries of L matrix are all 1s,they dont need to be stored. LU is stored in a single matrix.
There are infinitely many different ways to decompose A.Most popular one: U=Gaussian eliminated matrix
L=Multipliers used for elimination
8/7/2019 Lecture3(Linear Equation)
61/75
61
1 0 2 3 1* 2 0 * 0 1* 3 0 * 0.5 2 3
0.5 1 0 0.5 0.5* 2 1* 0 0.5* 3 1* 0.5 1 2
! ! !
LU
LU Factorization Method
Suppose we are given:
Then we can writeA= LU where:
Lets check that:
2 3
1 2
!
A
1 0
0.5 1
!
L
2 3
0 0.5
!
U
8/7/2019 Lecture3(Linear Equation)
62/75
62
Example 10
Solve the linear system
6x1-2x2-4x3+4x4 =2
3x1-3x2 -6x3+x4 =-4
-12x1+8x2+21x3-8x4 =8
-6x1-10x3+7x4 =-43
!
!
1211
0122
0012
10001
71006
821812
1633
4426
L
A
!
8000
2500
1420
4426
U
!
43
8
4
2
B
Solution:
8/7/2019 Lecture3(Linear Equation)
63/75
63
Example 10
Then solve AX=B by writing LUX=B. Let UX=Zand solve LZ=B
!
43
8
4
2
1211
0122
0012
1
0001
4
3
2
1
z
z
z
z
By forward substitution,
32243
2228
52
14
2
3214
213
12
1
!!
!!
!!
!
zzzz
zzz
zz
z
8/7/2019 Lecture3(Linear Equation)
64/75
64
Example 10
Solve UX=Z,
!
32
2
5
2
8000
2500
1420
4426
4
3
2
1
x
x
x
x
Hence,
546
4422
962
45
215
22
48
32
432
1
43
2
1
4
3
4
.
.
.
!
!
!
!
!
!
!
!
xxxx
xxx
xx
x
8/7/2019 Lecture3(Linear Equation)
65/75
65
Decomposition Methods
Solve UX=Z,
Hence,
Doolittle decomposition
Crout decomposition
Cholesky decomposition (for symmetric
matrices)
8/7/2019 Lecture3(Linear Equation)
66/75
66
Crout Decomposition
Solve UX=Z,
Hence,
!
!!
!
!
!
3323321331223212311131
2313212212211121
131211
333231
232221
131211
33
2322
131211
3231
21
3
2
1
3
2
1
333231
232221
131211
00
0
1
01
001
uululululul
uuluululuuu
aaa
aaaaaa
u
uu
uuu
ll
lLUA
x
x
x
X
b
b
b
B
aaa
aaa
aaa
A ,,
8/7/2019 Lecture3(Linear Equation)
67/75
67
Crout Decomposition
Solve UX=Z,
Hence,
233213
11
31
3333
12
11
21
2212
11
31
3232
11
31
31
13
11
21
232312
11
21
2222
11
21
21
1313
1212
1111
ulaa
aau
aa
aaa
a
aau
a
al
aaaaua
aaau
aal
au
au
au
!
!!
!!!
!
!
!
,/,
,,
8/7/2019 Lecture3(Linear Equation)
68/75
68
Crout Decomposition
Solve UX=Z,
Hence,
Thus the matrices L and Ubecome known. NowAX=B
becomes
LUX=B LY=B, where Y= UX
8/7/2019 Lecture3(Linear Equation)
69/75
69
Example 11
Solve the linear system
3x1+2x2+7x3 =4
2x1+3x2 +x3 =5
3x1+4x2+x3 =7
!
!!
3323321331223212311131
2313212212211121
131211
33
2322
131211
3231
21
143
132
723
00
0
1
01
001
uululululul
uuluulul
uuu
u
uu
uuu
ll
lLA
Solution:
8/7/2019 Lecture3(Linear Equation)
70/75
70
Example 11
713
11
5
61
521433
3
77
212
3
23
3
2
7
2
3
33
3231
232221
13
12
11
!
!!
!!!
!
!
!
u
ul
uul
u
u
u
,/,
,,
8/7/2019 Lecture3(Linear Equation)
71/75
71
Example 11
!
5
8
00
3
11
3
50
723
15
6
1
013
2
001
A
Write UX=Ywhich gives LY=B
!
!
5
1
3
74
7
5
4
15
61
013
2021
3
2
1
3
2
1
y
yy
y
yy
8/7/2019 Lecture3(Linear Equation)
72/75
72
Example 11
!
!
8
7
8
9
8
7
5
1
3
7
4
5
800
3
11
3
50
723
3
2
1
3
2
1
x
x
x
x
x
x
Hence the original system reduces to
!
!
5
1
3
74
7
5
4
15
61
013
2021
3
2
1
3
2
1
x
x
x
x
x
x
8/7/2019 Lecture3(Linear Equation)
73/75
73
Steps in LU Decomposition
8/7/2019 Lecture3(Linear Equation)
74/75
74
Iterative Methods
If systems of linear equations are very large,the computational effort of direct methods is
prohibitively expensive
Three common classical iterative techniques forlinear systems
The Jacobi method
Gauss-Seidel method
8/7/2019 Lecture3(Linear Equation)
75/75