MATH-1030 Matrix Theory & Linear Algebra I · MATH-1030 Matrix Theory & Linear Algebra I LinJiu...

MATH-1030Matrix Theory & Linear Algebra I

Lin Jiu

Dalhousie University

May 7th, 2019

Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 1 / 14

Syllabus

1 1st assignment of today: read the syllabus posted on Brightspace.Lin Jiu, Chase 307, [email protected] Office hours: Mondays and Tuesdays 12:00--1:30 PM or by appointmentOffice hours THIS WEEK: Wednesday (May 8th) and Thursday (9th) 12:00–1:30 PMLearning Centre: Chase 119. Tutors are available Monday–Friday 1–5pmHomework (10%), Classwork (10%), Midterm I (20%), Midterm II (20%), Final Exam (40%)

You need to pass the final (50% or higher) to pass the course.Midterm I: May 21, Midterm II: June 6th, 6:00-7:30 PM, Dunn 117Final Exam: June 20, 6:00-9:00 PM, Dunn 117Homework on BrightSpace (WeBWork) (five problem sets with 2 points for each)Classwork: on each lecture days (not including today and days with exams, i.e., 10 times), there will be one problem in the middle of thelecture. You can form a group of 2 or 3 to turn your classwork in. Each problem is worth 1 point

2 2nd assignment of the today: find one or two partners.3 For midterms and final, you will expect questions of

True of False without explanations; multiple choice; ←−NO PARTIAL CREDITSTu-re of False WITH explanations, shorted or detailed answers: ←−PARTIAL CREDITS

4 A missed midterm cannot be written at another time. If you miss the midterm without prior permission, then it will count as a0. Exceptions are made in two cases: (1) if you obtain the instructor’s prior permission to miss a midterm, or (2) if you havean officially valid excuse such as a medical doctor’s note. In these cases, the weight of the missed midterm will be shifted tothe final exam (e.g., the final exam will then count 60% instead of 40%). There is no make-up option for the final examexcept in cases of an officially valid excuse such as a medical doctor’s note.


Linear Algebra

1 LINEAR:Example of a linear equation:

2x = 8

Example of a non-linear equation:2x2 = 8

Example of another linear equation:2x + 3y + 10z − 5w = 13 (∗)

2 SOLUTION(S) of (∗)Solution 1: x = 2, y = 3, z = w = 0.

2 · 2+ 3 · 3 = 4+ 9 = 13.

Solution 2: x = 0, y = 1, z = 1, w = 0.3 · 1+ 10 · 1 = 13.

Solution 3: x = 132 , y = z = w = 0.

2 · 132

= 13.

Fact: There are infinitely many solutions of (∗).


Tuples

2x + 3y + 10z − 5w = 13 (∗)

NotationWe shall use the tuple notation for the solutions.

(x , y , z ,w) = (2, 3, 0, 0)

Tuple:

(x , y) -pair 2-tuple(x , y , z) -triple 3-tuple

· · · etc.

RemarkNote that tuples are ordered, and therefore different from sets. For example,

(2, 3) 6= (3, 2) while {2, 3} = {3, 2}.

The set of solutions of (∗) can be written(/listed) as{(2, 3, 0, 0), (0, 1, 1, 0),

(132, 0, 0, 0

), . . .

}


Systems of linear equations

Example

2x + 3y + 10z − 5w = 13 (∗)x + 2y + 3z + 4w = 5

y − z + 2w = 0

DefinitionA solution of a system of equations is an assigned of numbers to the variables, making all the equation true.

Example

(2, 3, 0, 0) is not a solution, since 1 · 2+ 2 · 3 = 8 6= 5.

(0, 1, 1, 0) is a solution:2+ 3 = 51− 1 = 0


Linear equation

DefinitionA linear equation is an equation of the form

a1x1 + a2x2 + . . .+ anxn = b. (1)

Here,

x1, . . . , xn − variables/unknownsa1, . . . , an − coefficients

b − constant term/right-hand side (RHS)

If an equation is written in the form of (1), it is in standard form.

Example

2x1 + 3x2 + 0x3 = 5

is a linear equation in standard form.

Example

2x − 3y + 5 = 7z + 2y

It is a linear equation (x1, x2, x3) = (x , y , z), but not in standard form. We can rewrite it to standard form by algebra:

2x + (−5)y + (−7)z = −5,

or2x − 5y − 7z = −5.


System of linear equations

DefinitionA system of linear equations is a list of equations

a11x1 + a12x2 + . . .+ a1nxn = b1a21x1 + a22x2 + . . .+ a2nxn = b2

...am1x1 + am2x2 + . . .+ amnxn = bm

(2)

Similarly, this is the standard form, and

x1, . . . , xn − variables/unknowns; a11, . . . , a1n, a21, . . . , amn − coefficients; b1, . . . , , bm − constant term

A system of equations is inconsistent if it has no solution. It is called consistent if it has one or more solutions.

Example2x − 7w = 52y + 3x = 15

x + w = y + 5a system of linear equations, but not in standard form⇒

2x + 0y − 7w = 53x + 2y + 0w = 15

x − y + w = 5

Solution

(x , y ,w) =

(18539

,513

,2539

). consistent

ExampleAre the following systems consistent or inconsistent?

(1)

{x + y = 52x + 2y = 11

(2)

{x = 0x = 1

(3)0x = 1 (4)

x + y = 7x − y = 3x + 2y = 1

All are inconsistent.


Geometric interpretation

When there are 2 or 3 variables, we can visualize the system.

Example {x + y = 7x − y = 3

Each equation represents a line.

This system has a unique solution(x , y) = (5, 2).



Example {x + y = 52x + 2y = 10

This system has infinity many solutions{(5, 0), (4, 1), (0, 5), . . .}



Example x + y = 7x − y = 3x + 2y = 1

This system has no solution (inconsistent).

Example x + y + z = 12x + y − z = 5x − y + 2z = 7

It has a unique solution (SageMath). This particular example involves three planes intersection in a point. There are many otherpossibilities for how 3 planes intersect (e.g., two or all three planes could be parallel).


Solving systems of equations algebraically

Elementary operations:Interchange (swap) two equations.Multiply an equation by a non-zero number. For example,

x + 2y = 5 ∼ 2x + 4y = 10.

Add a multiple of one equation to another. For example,

E1 :2x + y = 5E2 :x − 2y = 0

E1←E1−2E2∼ E1 :0x + 5y = 5E2 :x − 2y = 0

More precisely,

LHS : (2x + y)− 2(x − 2y) = 5yRHS : 5− 2 · 0 = 5.

From this, we can easily see y = 1 and then x = 2.Performing an elementary operation does NOT change the set of solutions of the system of equations, but (hopefully) it will makethe system simpler.

RemarkThe symbol “∼” means that tow systems of equations are equivalent, i.e., they have the SAME solutions.


Example

Example

E1 :x + 2y + z = 8E2 :2x + y + z = 7E3 :x − y − z = −4

E2←E2−2E1∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 :x − y − z = −4

E3←E3−E1∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : −3y − 2z = −12

E3←E3−E2∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : − z = −3

E3←−E3∼E1 :x + 2y + z = 8E2 : − 3y − z = −9E3 : z = 3

Back Substitution:z = 3 E2

=⇒ −3y − 3 = −9 =⇒ −3y = −6⇒ y = 2

(y , z) = (2, 3) E1=⇒ x + 4+ 3 = 8 =⇒ x = 1

This system has the unique solution(x , y , z) = (1, 2, 3)

It is consistent.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 7th, 2019 12 / 14

Matrix Notation

DefinitionThe augmented matrix of the system of linear equations


...am1x1 + am2x2 + . . .+ amnxn = bm

is a11 · · · a1n b1...

......

am1 · · · amn bm

.

ExampleE1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4

⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

NotationThe following shorter notation of elementary operations will apply:

Swap rows Ri and Rj : Ri ←→ Rj .Multiply Ri by a nonzero number a: Ri ←− aRi .Add a times Ri to Rj : Rj ←− Rj + aRi .


Example

Example

E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4

=⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

R2 ← R2 − 2R1

∼R1R2R3

1 2 1 80 −3 −1 −91 −1 −1 −4

R3 ← R3 − R1

∼R1R2R3

1 2 1 80 −3 −1 −90 −3 −2 −12

R3 ← R3 − R2

∼R1R2R3

1 2 1 80 −3 −1 −90 0 −1 −3

R3 ← −R3

∼R1R2R3

1 2 1 80 −3 −1 −90 0 1 3

Substitution as before. Or, we can continue as

R1R2R3

1 2 1 80 −3 −1 −90 0 1 3

R2 ← R2 + R3

∼R1R2R3

1 2 1 80 −3 0 −60 0 1 3

R2 ← −R2

3

∼R1R2R3

1 2 1 80 1 0 20 0 1 3

R1 ← R1 − 2R2 − R3 ∼R1R2R3

1 0 0 10 1 0 20 0 1 3

⇒ (x , y , z) = (1, 2, 3).



Lin Jiu


May 9th, 2019


Matrix Notation

DefinitionThe augmented matrix of the system of linear equations


...am1x1 + am2x2 + . . .+ amnxn = bm

is a11 · · · a1n b1...

......

am1 · · · amn bm

.

NotationThe following shorter notation of elementary operations will apply:

Swap rows Ri and Rj : Ri ←→ Rj .Multiply Ri by a nonzero number a: Ri ←− aRi .Add a times Ri to Rj : Rj ←− Rj + aRi .



⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

R2 ← R2 − 2R1

2 1 1 7−2 · ( 1 2 1 8 )

2 1 1 7− 2 4 2 16

0 −3 −1 −9

E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4

⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

R2←R2−2R1∼R1R2R3

1 2 1 80 −3 −1 −91 −1 −1 −4

R1R2R3

1 2 1 80 −3 −1 −91 −1 −1 −4

R3 ← R3 − R1

1 −1 −1 −4− 1 2 1 8

0 −3 −2 −12E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4

⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

R2←R2−2R1∼R1R2R3

1 2 1 80 −3 −1 −91 −1 −1 −4

R3←R3−R1∼R1R2R3

1 2 1 80 −3 −1 −90 −3 −2 −12

R1R2R3

1 2 1 80 −3 −1 −90 −3 −2 −12

R3 ← R3 − R2

0 −3 −1 −9− 0 −3 −2 −12

0 0 1 3

R3←R3−R2∼R1R2R3

1 2 1 80 −3 −1 −90 0 1 3



⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

∼∼∼∼ R1R2R3

1 2 1 80 −3 −1 −90 0 1 3

⇒ E1 : x + 2y + z = 8E2 : −3y − z = −9E3 : z = 3

Back Substitution:

z = 3 E2=⇒ −3y − 3 = −9 =⇒ y = 2

(y , z) = (2, 3) E1=⇒ x + 2 · 2+ 3 = 8 =⇒ x = 1

Or, we can continue as follows

R1R2R3

1 2 1 80 −3 −1 −90 0 1 3

R2 ← R2 + R3

0 −3 −1 −9+ 0 0 1 3

0 −3 0 −6

R2←R2+R3∼R1R2R3

1 2 1 80 −3 0 −60 0 1 3

R1R2R3

1 2 1 80 −3 0 −60 0 1 3

R2←− R23∼

R1R2R3

1 2 1 80 1 0 20 0 1 3

R2 ← R1 − R3

1 2 1 8

− 0 0 1 3

1 2 0 5

∼R1R2R3

1 2 0 50 1 0 20 0 1 3

R1R2R3

1 2 0 50 1 0 20 0 1 3

R1 ← R1 − 2R2

1 2 0 5

−2 · ( 0 1 0 2)

1 0 0 1

∼R1R2R3

1 0 0 10 1 0 20 0 1 3

E1 : x + 2y + z = 8E2 : 2x + y + z = 7E3 : x − y − z = −4

⇒R1R2R3

1 2 1 82 1 1 71 −1 −1 −4

∼∼∼∼ R1R2R3

1 0 0 10 1 0 20 0 1 3

⇒ E1 : x = 1E2 : y = 2E3 : z = 3


Example

E1 : x + 2y + z = 7E2 : 2x + 2y − 3z = 0E3 : 3x + 4y − 2z = 6

=⇒R1R2R3

1 2 1 72 2 −3 03 4 −2 6

R2 ← R2 − 2R1

∼R1R2R3

1 2 1 70 −2 −5 −143 4 −2 6

R3 ← R3 − 3R1

∼R1R2R3

1 2 1 70 −2 −5 −140 −2 −5 −15

R3 ← R3 − R2

∼R1R2R3

1 2 1 70 −2 −5 −140 0 0 −1

This an inconsistent system. The last indicates an equation

0x + 0y + 0z = 0 = −1.

If the augmented matrix contains a row of all 0’s on the LHS (coefficients) and a non-zero value on the RHS (constant term), as[0 0 · · · 0 b( 6= 0)

],

then the system is inconsistent.[Question] Any other possible cases? (unique solution, no solution, more solutions (infinitely many))


ExampleExample

E1 : x + 2y + z = 7E2 : 2x + 2y − 3z = 0E3 : 3x + 4y − 2z = 7

=⇒R1R2R3

1 2 1 72 2 −3 03 4 −2 7

R2 ← R2 − 2R1

∼R1R2R3

1 2 1 70 −2 −5 −143 4 −2 7

R3 ← R3 − 3R1

∼R1R2R3

1 2 1 70 −2 −5 −140 −2 −5 −14

R3 ← R3 − R2 ∼R1R2R3

1 2 1 70 −2 −5 −140 0 0 0

We were expecting an equation involving only z . However, there is no such equation, which means z can be any number. We let

z = s ,

where s is called a parameter(free variable), i.e., s is any number. We substitute back:

R2 : −2y − 5z = −14⇒ y = 7− 52z = 7− 5

2s

R1 : x + 2y + z = 7⇒ x = 7− 2y − z = 7− 2(7− 5

2s

)− z = −7+ 4s.

General solution (parametrized solution):

(x , y , z) =

(4s − 7, 7− 5

2s, s

)for any number s

Special solutions:s 0 1 2 3

(x , y , z) (−7, 7, 0)(−3, 9

2 , 1)

(1, 2, 2)(5,− 1

2 , 3)


Example

Example

E1 : x + 2y + 3z = 6E2 : 2x + 4y + 6z = 12

E3 : −x − 2y − 3z = −6(E2 = 2E1, E3 = −E1) =⇒

R1R2R3

1 2 3 62 4 6 12−1 −2 −3 −6

R2 ← R2 − 2R1

∼R1R2R3

1 2 3 60 0 0 0−1 −2 −3 −6

R3 ← R3 + R1

∼R1R2R3

1 2 3 60 0 0 00 0 0 0

The last two rows are trivial, which indicates that both y and z can be any number. Let

z = s, y = t,

for parameters s and t.R1 : x + 2y + 3z = 6⇒ x = 6− 2y − 3z = 6− 2t − 3s.

The (2-parameter) general solution is(x , y , z) = (6− 2t − 3s, t, s) .

Special solutions aret\s 0 1 2 30 (6, 0, 0) (3, 0, 1) (0, 0, 2) (−3, 0, 3)1 (4, 1, 0) (1, 1, 1) (−2, 1, 2) (−5, 1, 3)2 (2, 2, 0) (−1, 2, 1) (−4, 2, 2) (−7, 2, 3)3 (0, 3, 0) (−3, 3, 1) (−6, 3, 2) (−9, 3, 3)


Echelon Form

DefinitionAn entry of an augmented matrix is called a leading entry or pivot entry if it is the leftmost non-zero entry of a row. An augmentedmatrix is in echelon form (also called row echelon form) if

1 All rows of zeros are below all non-zero rows.2 Each leading entry of a row is in a column to the right of the leading entry of any row above it.

A column containing a pivot entry is also called a pivot column.

Example ∗ ∗ ∗ ∗ ∗ ∗ ∗0 ∗ ∗ ∗ ∗ ∗ ∗0 0 0 ∗ ∗ ∗ ∗0 0 0 0 0 0 00 0 0 0 0 0 0

{∗− nonzero entry;∗− leading/pivot entry.

Echelon forms: 0 5 2 1 3 50 0 0 0 1 60 0 0 0 0 00 0 0 0 0 0

,

1 4 0 0 5 00 0 1 0 2 00 0 0 1 4 00 0 0 0 0 1

,

3 0 6 20 1 4 00 0 2 10 0 0 0

Not in echelon form

0 0 0 01 2 3 30 1 0 20 0 0 10 0 0 0

,

1 2 32 4 −64 0 7

,

0 2 3 31 5 0 20 0 1 00 0 0 0


Gaussian Elimination

Goal: Use elementary row operations to transform the system of equations to echelon form.Input: Augmented matrix

Output: Augmented matrix in echelon form row equivalent to the input.

Algorithm: Gaussian EliminationThis algorithm provides a method for using row operations to take a matrix to its echelon form. We begin with the matrix in itsoriginal form.

1 Starting from the left, find the first non-zero column. This is the first pivot column, and the position at the top of this columnwill be the position of the first pivot entry. Switch rows if necessary to place a non-zero number in the first pivot position.

2 Use row operations to make the entries below the first pivot entry (in the first pivot column) equal to zero.3 Ignoring the row containing the first pivot entry, repeat steps 1 and 2 with the remaining rows. Repeat the process until there

are no more non-zero rows left.

Example

0 0 0 2 3 50 2 1 0 1 40 0 0 1 2 30 1 2 3 4 5

=

0 0 0 2 3 50 2 1 0 1 40 0 0 1 2 30 1 2 3 4 5

R1 ↔ R2 ∼

0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 1 2 3 4 5

R4 ← R4 −12

∼

0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 0 3

2 3 72 3

=

0 2 1 0 1 40 0 0 2 3 50 0 0 1 2 30 0 3

2 3 72 3

R2 ↔ R4

0 2 1 0 1 40 0 3

2 3 72 3

0 0 0 1 2 30 0 0 2 3 5

R4 ← R4 − 2R3

∼

x y z w v0 2 1 0 1 40 0 3

2 3 72 3

0 0 0 1 2 30 0 0 0 −1 −1

E1 2y + z + v = 4E2 3

2z + 3w + 72v = 3

E3 w + 2v = 3E4 −v = −1


Terminology1 Any column containing a pivot entry is called a pivot column.2 The corresponding variables are called pivot variables.3 Any variable that is not a pivot variable is called a free variable.

In the previous example, x is a free variable and y , z ,w , v are pivot variables.

Back substitutionEvery free variable becomes a parameter.Every pivot/leading variable has an equation, which we can use to find its value.

Examplex y z w v0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0

We have

v : free variable v = t

w : from R2, pivot variable w − 2v = 4 =⇒ w = 2v + 4 = 2t + 4z : free variable z = s

y : from R1, pivot variable y + 2z + 3v = 5 =⇒ y = 5− 2z − 3v = 5− 2s − 3tx : free variable x = r

General solution(x , y , z ,w , v) = (r , 5− 2s − 3t, s, 2t + 4, t).


Rank

Remark1 If a system is inconsistent, then the RHS is a pivot column. 1 2 0 3

0 0 1 50 0 0 7

−→ inconsistent, no solution.[0 0 · · · 0 b( 6= 0)

]2 If the system is consistent, the number of free variables determines the number of solutions:

no free variables=⇒no parameters=⇒unique solution;at least one free variable =⇒ infinitely many solution.

DefinitionThe rank of a matrix is the number of pivot entries of its echelon form.

Example

A =

0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0

B =

1 0 0 10 1 0 20 0 1 3

rank(A) = 2 rank(B) = 3


Rank

RemarkIf a system of m equations in n variables has rank r and is consistent, then there are n − r free variables, i.e., n − r parameters inthe general solution.

n = r ⇒ n − r = 0⇒unique solutionn > r ⇒ n − r > 0⇒infinitely many solutions.

Examplex y z w v0 1 2 0 3 50 0 0 1 −2 40 0 0 0 0 00 0 0 0 0 0

A system of 4 equations in 5 variables. rank= 2. m = 4, n = 5 and r = 2.

5− 2 = 3 free variables: (x , z , v) = (r , s, t)

v : = t

w : = 2t + 4z : = s

y : = 5− 2s − 3tx : = r

General solution(x , y , z ,w , v) = (t, 5− 2s − 3t, s, 2t + 4, t).


Classwork

Consider system

x + 2y + z = 5x + 3y + 2z = 9

2x + 5y + 3z = 14

and its corresponding matrix form 1 2 1 51 3 2 92 5 3 14

Find its rank.

Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 9th, 2019 13 / 18

Example

Solve the linear system

x + 2y + z = 5x + 3y + 2z = 92x + 5y + 3z = 14

1 2 1 51 3 2 92 5 3 14

R2 ← R2 − R1 ∼

1 2 1 50 1 1 42 5 3 14

R3 ← R3 − 2R1

∼

1 2 1 50 1 1 40 1 1 4

R3 − R2 ∼

1 2 1 50 1 1 40 0 0 0

x + 2y + z = 5y + z = 4

pivot variables x and y ; free variable z

z = tR2=⇒ y = 4− z = 4− tR1=⇒ x = 5− 2y − z = 5− 2 (4− t))− t = t − 3

General solution:(x , y , z) = (−3+ t, 4− t, t).

It is also written as column form: xyz

=

−3+ t4− tt

=

−340

+ t

1−11

.

We shall come back to this form in Chapter 2.


Gauss-Jordan elimination

It is similar to Gaussian elimination, but withmore elementary row operators;and less back substitution.

DefinitionA matrix B is in reduced echelon form if it is in echelon form and additionally

all pivot entries are equal to 1;all entries above pivot entries are equal to 0.

Examplex + 2y + z = 5x + 3y + 2z = 9

2x + 5y + 3z = 14−→

1 2 1 51 3 2 92 5 3 14

∼ 1 2 1 5

0 1 1 40 0 0 0

︸︷︷︸

Echelon Form

R1 ← R1 − 2R2 ∼

1 0 −1 −30 1 1 40 0 0 0

︸︷︷︸

Reduced Echelon Form

pivot variables x and y ; free variable z

z = t Equations:

{x − z = −3y + z = 4

=⇒

{y = 4− z = 4− t

x = −3+ z = −3+ t=⇒ (x , y , z) = (−3+ t, 4− t, t)

xyz

=

−340

+ t

1−11

=

−340

+

t−tt

=

−3+ t4− tt

.


Example

2x + y + w = 8x + y + z = 6

3x + 2y + z + v = −2→

2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2

R1 ↔ R2 ∼

1 1 1 0 0 62 1 0 1 0 83 2 1 0 1 −2

R2 ← R2 − 2R1

R3 ← R3 − 3R1

∼

1 1 1 0 0 60 −1 −2 1 0 −40 −1 −2 0 1 −20

R2 ← −R2 ∼

1 1 1 0 0 60 1 2 −1 0 40 −1 −2 0 1 −20

R1 ← R1 − R2

R3 ← R3 + R2

∼

1 0 −1 1 0 20 1 2 −1 0 40 0 0 −1 1 −16

R3 ← −R3 ∼

1 0 −1 1 0 20 1 2 −1 0 40 0 0 1 −1 16

R1 ← R1 − R3

R2 ← R2 + R3

∼

1 0 −1 0 1 −140 1 2 0 −1 200 0 0 1 −1 16

free variables z = s and v = t

R1 :x = −14+ z − v = −14+ s − t

R2 :y = 20− 2z + v = 20− 2s + t

R3 :w = 16+ v = 16+ t

General solution xyzwv

=

−14200160

+ t

−11011

+ s

1−2100

=

−14200160

+

−tt0tt

+

s−2ss00

=

−14− t + s20+ t − 2s

s16+ t

t


Uniqueness

TheoremReduced echelon forms are unique.

DefinitionTwo matrices are called row equivalent, if one can be obtained from the other by elementary row operations.

ExampleLet

A =

1 2 1 30 3 3 31 1 0 2

, B =

2 4 2 62 5 3 71 6 5 7

BR1← 1

2R1∼ 1 2 1 3

2 5 3 71 6 5 7

R2←R2−2R1∼

1 2 1 30 1 1 11 6 5 7

R3←R3−5R2∼

1 2 1 30 1 1 11 1 0 2

R2←3R2∼

1 2 1 30 3 3 31 1 0 2

= A

A and B are row equivalent.


Example

TheoremTwo matrices are row equivalent if and only if (iff) they have the same reduced echelon form.

Example

A =

1 2 1 30 3 3 31 1 0 2

, B =

2 4 2 62 5 3 71 6 5 7

AR3←R3−R1∼

1 2 1 30 3 3 30 −1 −1 −1

R2← 1

3R2∼ 1 2 1 3

0 1 1 10 −1 −1 −1

R3←R3+R2∼

1 2 1 30 1 1 10 0 0 0

R1←R1−2R2∼

1 0 −1 10 1 1 10 0 0 0

BR1← 1

2R1∼ 1 2 1 3

2 5 3 71 6 5 7

R2←R2−2R1∼

1 2 1 30 1 1 11 6 5 7

R3←R3−R1∼

1 2 1 30 1 1 10 4 4 4

R3←R3−4R2∼

1 2 1 30 1 1 10 0 0 0

R1←R1−2R2∼

1 0 −1 10 1 1 10 0 0 0

Thus, we confirm that A and B are row equivalent.



Lin Jiu


May 14th, 2019


Homogeneous systems of equations

DefinitionA system of equations is called homogeneous if each of the constantterms is equal to 0. A homogeneous system therefore has the form

a11x1 + a12x2 + . . .+ a1nxn = 0a21x1 + a22x2 + . . .+ a2nxn = 0

...am1x1 + am2x2 + . . .+ amnxn = 0

aij—coefficients

xj—variables

Example2x + 3y + 4z = 7

x − 2y = 0x + y + z = 3

Non-homogenous

2x + 3y + 4z = 0x − 2y = 0

x + y + z = 0

HomogenousLin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 2 / 33

Example

DefinitionA system of equations is called homogeneous if each of the constantterms is equal to 0. A homogeneous system therefore has the form

a11x1 + a12x2 + . . .+ a1nxn = 0a21x1 + a22x2 + . . .+ a2nxn = 0

...am1x1 + am2x2 + . . .+ amnxn = 0

RemarkHomogeneous systems are always consistent. They always have thefollowing (trivial) solution (x1, x2, . . . , xn) = (0, 0, . . . , 0).Therefore, we want to know, for a given homogeneous system, doesit have non-trivial solutions? Infinitely many solutions.


Example

2x + 3y + 4z = 0x − 2y = 0

x + y + z = 0→

2 3 4 01 −2 0 01 1 1 0

R1↔R2∼ 1 −2 0 0

2 3 4 01 1 1 0

R2←R2−2R1R3←R3−R1∼

1 −2 0 00 7 4 00 3 1 0

R2←R2−2R3∼ 1 −2 0 0

0 1 2 00 3 1 0

R1←R1+2R2R3←R3−3R2∼

1 0 4 00 1 2 00 0 −5 0

R3←−R3

5∼ 1 0 4 0

0 1 2 00 0 1 0

R1←R1−4R3R2←R2−2R3∼

1 0 0 00 1 0 00 0 1 0

=⇒ (x , y , z) = (0, 0, 0)


Example2x + 3y + 4z = 0

x − 2y = 0x + y + z = 0

−→

2 3 4 01 −2 0 01 1 1 0

∼∼∼∼ 1 0 0 0

0 1 0 00 0 1 0

Thus, it only has the trivial solution

(x , y , z) = (0, 0, 0)[Question] What is the rank of the augmented matrix (or its reducedechelon form)? (Elementary row operations do not change the rank)

rank = 3 = # of unknowns.

FactIf a homogeneous system of n variables has rank n, then it only hasthe trivial solution.

# of free variables/parameters = # of unknowns− rank


Example

x + 3y − 2z = 02x + 4y − 2z = 0

y − z = 0→

1 3 −2 02 4 −2 00 1 −1 0

R2←R2−2R1∼ 1 3 −2 0

0 −2 2 00 1 −1 0

R2↔R3∼

1 3 −2 00 1 −1 00 −2 2 0

R1←R1−3R2R3←R3+2R2∼

1 0 1 00 1 −1 00 0 0 0

3 variables with rank 2:

x y z 1 0 1 00 1 −1 00 0 0 0

=⇒

{pivot variables x , y

free variable z = t


Example

x + 3y − 2z = 02x + 4y − 2z = 0

y − z = 0→

1 3 −2 02 4 −2 00 1 −1 0

∼∼x y z 1 0 1 00 1 −1 00 0 0 0

z = t ⇒

{R1 : x + z = 0R2 : y − z = 0

⇒

{x = −ty = t

(x , y , z) = (−t, t, t)

General solution:xyz

=

−ttt

= t ·

−111

basic solution(t = 1)−

−111


TheoremTheoremGiven a homogeneous system with n variables and of rank r , thesystem is always consistent:

1 if r = n, it only has the unique, trivial solution; (all zeros)2 if r < n, there are non-trivial solutions. In this case, the number

of parameters is n − r and there are n − r basic solutions.

Example

x + 4y + 3z = 03x + 12y + 9z = 0

→[1 4 3 03 12 9 0

]R2←R2−3R1∼

[1 4 3 00 0 0 0

]parameters: z = s, y = t =⇒ x = 0− 4y − 3z = −4t − 3sxy

z

=

−4t − 3sts

= t

−410

+s

−301

basicsolutions:

−410

and

−301


TheoremTheoremIf the general solution has the formx1

...xn

= s1

•...•

+ · · ·+ sr

•...•

basicsolutions:

•...•

, . . . ,

•...•

DefinitionLet A be a system of linear equations. The associated homogeneoussystem is the system B , which has the same left-hand side as A, butall 0’s on the right-hand side.Example

x + 4y + 3z = 23x + 12y + 9z = 6

associated homogeneous system−−−−−−−−−−−−−−−−−−−−−→

x + 4y + 3z = 03x + 12y + 9z = 0


Comparison

x + 4y + 3z = 23x + 12y + 9z = 6

(A)→[1 4 3 23 12 9 6

]R2←R2−3R1∼

[1 4 3 20 0 0 0

]free variables: z = s, y = t =⇒ x = 2− 4y − 3z = 2− 4t − 3sxy

z

=

2− 4t − 3sts

=

200

+ t

−410

+ s

−301

x + 4y + 3z = 0

3x + 12y + 9z = 0(B)→

[1 4 3 03 12 9 0

]R2←R2−3R1∼

[1 4 3 00 0 0 0

]xyz

=

−4t − 3sts

= t

−410

+ s

−301

(x , y , z) = (2, 0, 0)is a solution to A


ComparisonTheoremLet A be a system of equations, and let B be the associatedhomogeneous system. Then

the general solution of A(GA) =a particular solution of A(PA)+ the general solution of B(GB).

Example

x + 4y + 3z = 23x + 12y + 9z = 6

(A) −→x + 4y + 3z = 0

3x + 12y + 9z = 0(B) ∼

[1 4 3 00 0 0 0

]xyz

︸︷︷︸GA

=

200

︸︷︷︸PA

+ t

−410

+ s

−301

︸︷︷︸

GB


Summary of Chpt. 1

Given a system (or just one equation), tell whether it is linear:

2x + 4y = 0linear

{3x + 5y = 32xy = 4

NOT linear

Given a linear system, write down the augmented matrix:

x + 2y + z = 0x + 3y + 2z = 32x + 5y + 3z = 3

=⇒

1 2 1 01 3 2 32 5 3 3

and vice versa.


Summary of Chpt. 1

The most important eliminations:Gaussian elimination and Gauss-Jordan elimination

Gaussian Elimination↓

echelon form

Gauss-Jordan Elimination↓

redued echelon form

↓ ∗ ∗ ∗ ∗0 0 ∗ ∗0 0 0 0

↓ 1 ∗ 0 ∗

0 0 1 ∗0 0 0 0

through THREE elementary row operations

Ri ←→ Rj Rj ←− aRj Rj ←− Rj + aRi (i 6= j , a 6= 0)


Summary of Chpt. 1

(reduced) echelon form: 1 2 1 01 3 2 32 5 3 3

∼ 1 2 1 0

0 1 1 30 0 0 0

⇒ {x + 2y + z = 0y + z = 3

1 Consistent or inconsistent? For the example, it is consistent.

Inconsistent example:

1 2 0 30 0 4 50 0 0 7

2 Pivot variable(s): x and y ;3 Free variable(s): z = t4 rank=# of pivot entries= 2


Summary of Chpt. 1

(reduced) echelon form: 1 2 1 01 3 2 32 5 3 3

∼ 1 2 1 0

0 1 1 30 0 0 0

⇒ {x + 2y + z = 0y + z = 3

5 Solution z = t free, x , y pivotechelon form =⇒ back substitution y = 3− z = 3− tx = −z − 2y = −6 + treduced echelon form 1 2 1 0

0 1 1 30 0 0 0

∼ 1 0 −1 −6

0 1 1 30 0 0 0

xyz

=

•••

+t

•••

=

••0

+t

••1

=

−630

+t

1−11

=

−6+ t3− tt


Summary of Chpt. 1

6 Determine whether two systems have the same solutions.Compare the reduced echelon forms of the two systems

x + 2y + z = 33y + 3z = 3x + y = 2

2x + 4y + 2z = 62x + 5y + 3z = 7x + 6y + 5z = 7

A =

1 2 1 30 3 3 31 1 0 2

, B =

2 4 2 62 5 3 71 6 5 7

1 2 1 3

0 3 3 31 1 0 2

∼ 1 0 −1 1

0 1 1 10 0 0 0

∼ 2 4 2 6

2 5 3 71 6 5 7

So they have the same solutions. (Two matrices are rowequivalent.)


Summary of Chpt. 1

Some important facts:1 A system of linear equations can have 0, 1 and infinitely many

solutions.0 solution—inconsistent 1 or infinitely many—consistent

2 A consistent system of m linear equations, n variables and rankr . # of free variables/parameters = n − r

n = r unique solutionn > r infinitely many

Hard & Tricky Problem: What happens if r > n? For example, asystem of 3 equations with 2 variables and rank 3.The answer is that the system if inconsistent.

# of free variables‖

2− 3 = −1

• • •• • •• • •

∼ ∗ ∗ ∗0 ∗ ∗

0 0 ∗


Classwork

Solve the homogeneous system

x + 2y + z = 0x + 3y + 2z = 0

2x + 5y + 3z = 0

Lecture resumes at 7:45.


Solution

x + 2y + z = 0x + 3y + 2z = 02x + 5y + 3z = 0

−→

1 2 1 01 3 2 02 5 3 0

∼

1 2 1 00 1 1 00 0 0 0

∼

1 0 −1 00 1 1 00 0 0 0

z is free: z = t. xy

z

= t ·

1−11


Coordinate systemOn the plane (R2):

The coordinate system allows us to describe every point in the planeby a pair of real numbers, called the coordinates of the point.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 20 / 33

VectorsUnlike a point, which describes a location in a coordinate system, avector describes an offset or a distance and direction.

v =

[21

]2 and 1 of v are called components. w =

[3−1

]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 21 / 33

Space

Zero vector

0 =

[00

].

More generally,a point in an n-dimensional space is given by n coordinates andwritten as

P = (x1, . . . , xn)

a vector in an n-dimensional space is given by n components andwritten as

v =

x1...xn

0 =

0...0

.


Space

DefinitionThe set of n-dimensional column vectors is denoted by

Rn :=

x1...xn

x1, . . . , xn ∈ R

and is called n-dimensional Euclidean space.

ExampleR2 : Euclidean planeR3 : vectors in 3-dimensional Euclidean spaceRn : vectors in n-dimensional Euclidean space


Points VS Vectors1 Given two points P and Q, we can define a vector v =

−→PQ

2 In n-dimensional space, the coordinate vector of a pointP = (x1, . . . , xn) is the vector

−→OP =

x1...xn

.

O = (0, 0, . . . , 0) is the origin.

Example (n = 2)


Addition of vectors

If

v =

x1

x2...xn

, w =

y1

y2...yn

=⇒ v + w =

x1 + y1

x2 + y2...

xn + yn

.

Slide w so that the tail of w is on the tip of v. Then draw the arrowwhich goes from the tail of v to the tip of w . This arrow representsthe vector v + w.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 14th, 2019 25 / 33

Addition of vectors

Example

[23

]+

[−14

]=

[17

] 123−45

+

110−11

=

233−56

Negation

v =

x1

x2...xn

=⇒ −v =

−x1

−x2...−xn

Rotate v by 180◦


Properties of AdditionProperties

(A1) u + v = v + u(A2) (u + v) + w = u + (v + w)

(A3) u + 0 = u(A4) u + (−u) = 0

Exampleu + v + w + (−u) + w + (−w)

(A2) = u + v + w + (−u) + (w + (−w))

(A4) = u + v + w + (−u) + 0(A3) = u + v + w + (−u)(A1) = u + (−u) + v + w(A4) = 0 + v + w(A3) = v + w.


Properties of AdditionProperties

(A1) u + v = v + u(A2) (u + v) + w = v + (u + w)

(A3) u + 0 = u(A4) u + (−u) = 0

Remark(A1)


Scalar Multiplication

a ·

x1

x2...xn

=

ax1

ax2...

axn

Example

3 ·

12−1

=

36−3

‘

Properties(SM1) k (u + v) = ku + kv(SM2) (k + `)u = ku + `u(SM3) k (`u) = (k`)u(SM4) 1 · u = u

Example

u + 2v − 3u− 5v + 6u(A1) = (u− 3u + 6u) + (2v − 5v)

(SM2) = (1− 3 + 6)u + (2− 5) v= 4u− 3v


Example

2x + y + w = 8x + y + z = 6

3x + 2y + z + v = −2→

2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2

∼ 1 0 −1 0 1 −14

0 1 2 0 −1 200 0 0 1 −1 16

free variables z = s and v = tGeneral solution:xyzwv

=

−14200160

+t

−11011

+s

1−2100

=

−14200160

+−tt0tt

+

s−2ss00

=

−14− t + s20 + t − 2s

s16 + t

t


ExampleConsider four points A(2, 3), B(−3, 4), C (−8,−1) and D (−3,−2)

−→AB =

[−3− 24− 3

]=

[−51

]−→DC =

[−8− (−3)−1− (−2)

]=

[−51

]=−→AB

−→AD =

[−3− 2−2− 3

]=

[−5−5

]−→CB =

[−3− (−8)4− (−1)

]=

[55

]= −−→AD =

−→DA

A,B ,C ,D form a parallelogram


A(2, 3), B(−3, 4), C (−8,−1) D (−3,−2)

−→AB +

−→BC =

−→AB +

−→BC =

−→AC

−→AB+

−→BC =

[−51

]+

[−5−5

]=

[−10−4

]−→AC =

[−8− 2−1− 3

]=

[−10−4

]−→AB−

−→BC =? in terms of

−→XY ,

for X and Y from A, B , C andD.


A(2, 3), B(−3, 4), C (−8,−1) D (−3,−2)

−→AB−

−→BC =

−→AB+

−→CB =

−→DC+

−→CB =

−→DB

−→AB −

−→BC =

[−51

]−[−5−5

]=

[−51

]+

[55

]=

[06

]−→DB =

[−3− (−3)4− (−2)

]=

[06

]



Lin Jiu


May 16th, 2019


Recall

v =

x1x2...xn

, w =

y1y2...yn

=⇒ v + w =

x1 + y1x2 + y2

...xn + yn

. a ·

x1x2...xn

=

ax1ax2...

axn

Example 123−45

+

110−11

=

233−56

3 ·

12−1

=

36−3


Linear Combination

DefinitionA linear combination of vectors v1, . . . , vn is an expression of the form

w = a1v1 + a2v2 + · · ·+ anvn

(=

n∑k=1

akvk

).

Example

Is the vector w =

231

a linear combination of v1 =

121

and v2 =

352

? By

definition, we need to determine whether there exist scalars a1 and a2 such that

w = a1v1 + a2v2231

= a1

121

+ a2

352

⇐⇒231

=

a1 + 3a22a1 + 5a2a1 + 2a2

⇐⇒a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 1


Linear Combination

w =

231

, v1 =

121

, v2 =

352

w = a1v1 + a2v2 ⇐⇒

a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 1

A system of linear equations. Of course E3− E1 gives a2 but we would like toreview the Gauss-Jordan elimination. 1 3 2

2 5 31 2 1

R2←R2−2R1R3←R3−R1∼

1 3 20 −1 −10 −1 −1

R3←R3−R2R2←−R2∼

1 3 20 1 10 0 0

R1←R1−3R2∼

1 0 −10 1 10 0 0

⇒ [a1a2

]=

[−11

]⇒ w = −v1 + v2

So the answer is yes, w is a linear combination of v1 and v2.


Example

Is the vector u =

232

a linear combination of v1 =

121

and v2 =

352

?

u = a1v1 + a2v2 ⇔

232

= a1

121

+ a2

352

=

a1 + 3a22a1 + 5a2a1 + 2a2

⇔a1 + 3a2 = 22a1 + 5a2 = 3a1 + 2a2 = 2 1 3 2

2 5 31 2 2

R2←R2−2R1R3←R3−R1∼

1 3 20 −1 −10 −1 0

R3←R3−R2∼ 1 3 2

0 1 10 0 1

The system is inconsistent. Such a1 and a2 do not exist. u is NOT a linearcombination of v1 and v2.


Geometrically....

The set of all linear combinations of u and v is the set of (position vectors of)points in the plane spanned by u and v. Here, u and v are not parallel, i.e.,v 6= ku for some k . (Otherwise au + bv = au + bku = (a+ bk)u —a line)


Length/Norm

Definition

The norm of a vector u =

x1...xn

is given by

‖u‖ :=√x21 + · · ·+ x2

n

Example

u =

[34

]By definition, ‖u‖ =

√32 + 42 =

√9+ 16 =

√25 = 5

v =

123−2

=⇒ ‖v‖ =√

12 + 22 + 32 + (−2)2 =√1+ 4+ 9+ 4 =

√18 = 3

√2



By Pythagorean’s Theorem, let the length of u bec , then

c2 = 32 + 42 ⇒ c =√

32 + 42 = 5.

FactGeometrically, ‖u‖ is the length of the vector u.

RemarkWe do not distinguish the norm of a vector andthe length of a vector.


Unit vector

DefinitionA vector is called a unit vector if it has length 1.

ExampleThe following vectors are unit vectors:

100

,010

,001

,−1000

,[

1√2

1√2

]

√(1√2

)2

+

(1√2

)2

=

√12+

12= 1

u =

[34

]is NOT a unit vector. ‖u‖ = 5 6= 1.


Unit vector

DefinitionIf u is any non-zero vector, then 1

‖u‖u is a unit vector, which is called the result ofnormalizing the vector u.

Remark1 If u 6= 0 (non-zero), ‖u‖ is a positive number (length) , so is 1

‖u‖ .

1‖u‖

u is the scalar multiplication of1‖u‖

and u

2 If u = 0 =

0...0

, ‖0‖ =√02 + · · ·+ 02 = 0 , so that 1

‖0‖ does not exist.

3 Sometimes we write u‖u‖


Unit vector

Example

Normalize the vector u =

[34

]. Since ‖u‖ =

√32 + 42 = 5,

1‖u‖

u =15

[34

]=

15 · 3

15 · 4

=

35

45

.Let v = 1

‖u‖u: ‖v‖ =√( 3

5

)2+( 4

5

)2=√

925 + 16

25 = 1 v is a unit vector.

Laws of norm/length‖u‖ ≥ 0. ‖u‖ = 0 if and only if (iff) u = 0‖ku‖ = |k| ‖u‖Triangle inequality ‖v + w‖ ≤ ‖v‖+ ‖w‖


Unit vector

Laws of norm/length1 ‖u‖ ≥ 0. ‖u‖ = 0 if and only if (iff) u = 02 ‖ku‖ = |k| ‖u‖3 Triangle inequality ‖v + w‖ ≤ ‖v‖+ ‖w‖

Example

Let u =

23−6

‖u‖ =√

22 + 32 + (−6)2 =√4+ 9+ 36 = 7

−2u =

−4−612

=⇒ ‖2u‖ =√(−4)2 + (−6)2 + 122 = 14 = |−2| ‖u‖

Remark

If u is non-zero∥∥∥∥ 1‖u‖u

∥∥∥∥ =∣∣∣ 1‖u‖

∣∣∣ · ‖u‖ 1= 1‖u‖ · ‖u‖ = 1⇒ 1

‖u‖u is a unit vector


Dot Product

DefinitionLet

u =

u1...un

, and v =

v1...vn

The dot product of u and v is defined by

u · v :=

u1...un

•v1...vn

= u1v1 + u2v2 + · · ·+ unvn

(=

n∑k=1

ukvk

).

Example 1−10

•200

= 1 · 2+ (−1) · 0+ 0 · 0 = 2+ 0+ 0 = 2.


Examples

Let u =

[11

], v =

[−12

]and w =

[1−1

].

u · v = 1 · (−1) + 1 · 2 = −1+ 2 = 1v ·w = (−1) · 1+ 2 · (−1) = −3u ·w = 1 · 1+ 1 · (−1) = 0

u ·w = 0, u · v > 0, v ·w < 0

We can observe from the picture that:u and w is perpendicular (= 90◦);the (smaller) angle between u and v is acute (< 90◦);the (smaller) angle between v and w is obtuse (> 90◦);



We only consider the smaller angle (the one≤ 180◦) between two vectors.

u · v = ‖u‖ · ‖v‖ cos θ

Recall that if both u and v are non-zero,

‖u‖ > 0, ‖v‖ > 0 and cos θ

> 0, θ < 90◦

= 0, θ = 90◦

< 0, θ > 90◦

If the angle θ is acute (θ < 90◦), then u · v > 0;If the angle θ is obtuse (θ > 90◦), then u · v < 0;If two vectors are perpendicular (θ = 90◦), thenu · v = 0;Vice versa: u · v gives certain information for theangle between u and v


Example

Find the angle between

u =

1−10

and v =

200

Directly, 1

−10

•200

= 1 · 2+ (−1) · 0+ 0 · 0 = 2.

In addition,

‖u‖ =√

12 + (−1)2 + 02 =√2 ‖v‖ =

√22 + 02 + 02 = 2.

Thus, let θ be the angle between u and v.

cos θ =u · v‖u‖‖v‖

=2√2 · 2

=1√2⇒ θ = 45◦.

How about sin θ? sin 45◦ = 1√2sin2 θ + cos2 θ = 1 =⇒ sin2 θ = 1− cos2 θ = 1

2

0◦ ≤ θ ≤ 180◦ =⇒ sin θ ≥ 0 =⇒ sin θ =

√12=

1√2


Laws

u · v = v · u (Be careful, it is not associative! (u · v)w 6= u(v ·w))The dot product gives a scalar(number). We can NOT compute u · v ·w! Inaddition,

u =

[10

], v =

[11

], w =

[01

]=⇒

{u · vv ·w

= 1= 1

=⇒

{(u · v)w(v ·w)u

= w= u

u · u = ‖u‖2 ≥ 0, u · u = 0 if and only if (iff) u = 0

u · u = ‖u‖‖u‖ cos 0◦ = ‖u‖2

(u + v) ·w = u ·w + v ·w, u · (v + w) = u · v + u ·w(ku) · v = k(u · v) = u · (kv)|u · v| ≤ ‖u‖‖v‖

|u · v| = |‖u‖ · ‖v‖ cos θ| = ‖u‖ · ‖v‖ |cos θ| ≤ ‖u‖ · ‖v‖


Example

1 u · v = v · u2 u · u = ‖u‖2 ≥ 0, u · u = 0 if and only if (iff) u = 03 (u + v) ·w = u ·w + v ·w, u · (v + w) = u · v + u ·w4 (ku) · v = k(u · v) = u · (kv)5 |u · v| ≤ ‖u‖‖v‖

Problem. Using the properties of the dot product, prove the parallelogramidentity:

‖a + b‖2 + ‖a− b‖2 = 2‖a‖2 + 2‖b‖2

Proof.

‖a + b‖2 + ‖a− b‖22= (a + b) · (a + b) + (a− b) · (a− b)3= a · a + a · b + b · a + b · b + a · a + (a) · (−b) + (−b) · a + (−b) · (−b)4= a · a + a · b + b · a + b · b + a · a− a · b− b · a + (−1)2 b · b1= a · a + a · b + a · b + b · b + a · a− a · b− a · b + b · b= 2a · a + 2b · b2= 2‖a‖2 + 2‖b‖2


Example

‖a + b‖2 + ‖a− b‖2 = 2‖a‖2 + 2‖b‖2

Given a parallelogram ABCD, if wedenote the length of the line segmentAC by |AC | and similarly for othersegments, we have|AC |2 + |BD|2 = |AB|2 + |CD|2 +

|BC |2 + |AD|2 = 2 |AB|2 + 2 |BC |2

−→AC =

−→AB +

−→BC = a + b−→

DB =−→DA+

−→AB = −b + a = a− b

‖−→AB‖ = ‖

−→DC‖ = ‖a‖

‖−→AD‖ = ‖

−→BC‖ = ‖b‖

Definition ((This will not appear in the tests))Read section 2.6.5 for

compv(u) :=u · v‖v‖

(a scalar) and projv(u) =u · v‖v‖2

v (a vector)


Classwork

Let θ be the angle between vectors

u =

102

and v =

1−20

.Find cos θ and sin θ


Solution

u · v =

102

· 1−20

= 1 · 1+ 0 · (−2) + 2 · 0 = 1

Meanwhile

‖u‖ =√

12 + 02 + 22 =√5, ‖v‖ =

√12 + (−2)2 + 02 =

√5

Thus,

cos θ =u · v‖u‖‖v‖

=1√

5 ·√5=

15

From cos2 θ + sin2 θ = 1, we have

sin2 θ = 1− cos2 θ = 1− 125

=2425

Notice that θ ≤ 180◦, which indicates that sin θ ≥ 0.

sin θ =

√2425

=2√6

5.


Cross Product ONLY in R3

DefinitionGiven

u =

u1u2u3

and v =

v1v2v3

,the cross product of u and v is defined by

u× v =

u1u2u3

×v1v2v3

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1

.Example

u =

341

v =

201

=⇒ u× v =

4 · 1− 1 · 01 · 2− 3 · 13 · 0− 4 · 2

=

4−1−8


Geometric Meaning

u× v is orthogonal(perpendicular) to both uand v.‖u× v‖ = ‖u‖ · ‖v‖ sin θ is the area of theparallelogram spanned by u and v.u, v, and u× v form a right-hand system.

Use your right handthumb uindex finger vthen the middle finger givesyour the direction of u× v.


Right-hand coordinate system

i =

100

, j =

010

, k =

001

Any vector in R3 can be uniquely written as a linear combination of i, j, andk: xy

z

= x i + y j + zk = x

100

+ y

010

+ z

001

Table

i× i = 0 i× j = k i× k = −jj× i = −k j× j = 0 j× k = ik× i = j k× j = −i k× k = 0


Laws

i× i = 0 i× j = k i× k = −jj× i = −k j× j = 0 j× k = ik× i = j k× j = −i k× k = 0

1 u× v = −v × u2 u× u = 0 (sin 0◦ = 0)3 u× (v + w) = u× v + u×w,4 (u + v)×w = u×w + v ×w,5 (ku)× v = k(u× v) = u× (kv)

Example

u =

341

v =

201

u× v = (3i + 4j + k)× (2i + k)3,4,5= 6i× i + 3i× k + 8j× i + 4j× k + 2k× i + k× k

Table = 60− 3j− 8k + 4i + 2j + 0= 4i− j− 8k

=

4−1−8

=

4 · 1− 1 · 01 · 2− 3 · 13 · 0− 4 · 2


Area

1. Find the area of the parallelogram spanned by

u =

341

v =

201

.Answer is

‖u× v‖ =∥∥∥∥ 4−1−8

∥∥∥∥ =√

42 + (−1)2 + (−8)2 =√16+ 1+ 64 =

√81 = 9.

2. Find the area of the triangle that has

u =

341

v =

201

as two of its sides.Answer is

12‖u‖‖v‖ sin θ = 1

2‖u× v‖ = 92


Box product

Addition— u + v vector+vector=vector;Scalar Multiplication— ku scalar·vector=vector;Dot product— u · v vector·vector=scalar/number;Cross product— u× v vector×vector=vector;Norm/Length— ‖u‖ ‖vector‖ =scalar/number;Normalization— 1

‖u‖u =unit vector;

DefinitionThe box product of three vectors u, v, and w is (u× v) ·w. It is equal to thesigned volume of the parallelepiped spanned by u, v, and w. The sign is

positive, if u, v, and w form a right-handed system;negative, if u, v, and w form a left-handed system;

The actual, unsigned volume is

volume = |(u× v) ·w| .


Box product

Volume = Base Area × HeightBase Area = ‖u× v‖Height = ‖w‖ cos θVolume = ‖u× v‖‖w‖ cos θ = (u× v) ·w

Fact(u× v) ·w = (v ×w) · u = (w × u) · v See Proposition 2.51 of the textbook.

ExampleFind the volume of the parallelepiped determined by the vectors 1

−7−5

, 1−2−6

, and −32

3


Box product

1−7−5

, 1−2−6

, and −32

3

1−7−5

× 1−2−6

=

(−7) (−6)− (−5) (−2)(−5) 1− 1 (−6)1 (−2)− (−7) 1

=

3215

3215

· −32

3

= 32 · (−3) + 1 · 2+ 5 · 3 = −79

Volume = |−79| = 79.


Summary of Chpt. 2

1 Points and Vectors2 Operations:

Addition—u + v

Scalar Multiplication—ku

Dot product—u · vCross product—u× v

Norm/Length—‖u‖;Normorlization— 1

‖u‖u

Box Product (u× v) · w

Definition (Calculations)

Laws/Properties

Geometric Interpretations

3 Linear combinations ⇐⇒ Linear systems



Lin Jiu


May 21st, 2019

Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 1 / 17

A system of linear equations

Standard form3x + 2y + 4z = 7

x − y + z = 22y + 3z = 5

Vector form x

310

+ y

2−12

+ z

413

=

725

=

3x + 2y + 4zx − y + z

0x + 2y + 3z

Goal of the chapter: Matrix form3 2 4

1 −1 10 2 3

·xyz

=

725

←− GOAL

Augmented matrix: 3 2 4 71 −1 1 20 2 3 5


DefinitionDefinitionAn m × n-matrix is an array of scalars with m rows and n columns.

m rows

a11 a12 a13 · · · a1na21 a22 a23 · · · a2n...

......

. . ....

am1 am2 am3 · · · amn

︸︷︷︸

n columns

The dimension of the matrix is m × n.The scalars in the matrix are called entries. An entry at the ith row and jthcolumn is an (i , j)-entry.

Example

If A =

[2 3 10 0 2

], 2 is the (1, 1)-entry; both (2, 1) and (2, 2) entries are 0. The

dimension of A is 2× 3.


Equality, Addition

Two matrices are equal if they have the same dimension (i.e., the same number ofrows and columns) and the same entries.

Example [2 3 10 1 1

]=

[2 3 10 1 1

] [0 0 00 0 0

]6=

0 00 00 0

DefinitionWe can add two m × n matrices (same dimension) and get an m × n matrix.a11 · · · a1n

.... . .

...am1 · · · amn

+

b11 · · · b1n...

. . ....

bm1 · · · bmn

=

c11 · · · c1n...

. . ....

cm1 · · · cmn

where

cij = aij + bij , 1 ≤ i ≤ m, 1 ≤ j ≤ n


Example

1 2 3−1 05 0

+

2 21 10 −1

=

2+ 2 3+ 2−1+ 1 0+ 15+ 0 0+ (−1)

=

4 50 15 −1

2 2 3

−1 05 0

+

[2 1 02 1 −1

]not defined

RemarkRecall the addition of vectors:u1

...un

+

v1...vn

=

u1 + v1...

un + vn


Scalar multiplication

Definition

k ·

a11 · · · a1n...

. . ....

am1 · · · amn

=

ka11 · · · ka1n...

. . ....

kam1 · · · kamn

Example

2 ·[1 10 3

]=

[2 20 6

]

RemarkRecall the scalar multiplication of vectors:

a

u1...un

=

au1...

aun


Negation, Zero Matrix

Definition

1 If A =

a11 · · · a1n...

. . ....

am1 · · · amn

, then −A := (−1) · A =

−a11 · · · −a1n...

. . ....

−am1 · · · −amn

−

2 3−1 05 0

=

−2 −31 0−5 0

2 The zero matrix of dimension m × n is the m × n matrix with all entries 0:

0 =

0 0 · · · 00 0 · · · 0...

.... . .

...0 0 · · · 0

︸︷︷︸

n columns

m rows

RemarkIt depends on the context that whether 0is the scalar or the zero matrix of acertain dimension.


Laws

Let A, B, C , and 0 have the same dimen-sion.

Addition1 (A+ B) + C = A+ (B + C)2 A+ B = B + A3 A+ 0 = A = 0 + A4 A+ (−A) = 0

Scalar Multiplication1 (k + `)A = kA+ À2 k(A+ B) = kA+ kB3 k(À) = (k`)A4 1A = A, (−1)A = −A, 0A = 0

For vectors:(A1) u + v = v + u(A2) (u + v) + w =

u + (v + w)

(A3) u + 0 = u(A4) u + (−u) = 0

(SM1) k (u + v) = ku + kv(SM2) (k + `)u = ku + ù(SM3) k (ù) = (k`)u(SM4) 1 · u = u

FactA column vector in Rn is a special n × 1 matrix.

For example, v =

[12

]is a 2× 1 matrix.


MultiplicationFact (Row and Column Vectors)

1 A column vector

x1...xn

is an n × 1 matrix.

2 A row vector[x1, · · · , xn

]is a 1× n matrix.

3 A scalar x is, sometimes, usefully, considered as a 1× 1 matrix[x]. If the

matrix contains only 1 entry, the square brackets “[]” can be considered asredundant.

Definition (Matrix Multiplication)We can multiply an m × n matrix by a k × ` matrix ONLY if n = k . In this case,the product is an m × ` matrix. In generala11 · · · a1n

.... . .

...am1 · · · amn

b11 · · · b1`

.... . .

...bn1 · · · bn`

=

c11 · · · c1`...

. . ....

cm1 · · · cm`

,

where cij = ai1b1j + ai2b2j + · · ·+ ainbnjLin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 9 / 17

Multiplication

Remember this: ci j is determined by the i-th row of the first matrix and the j-thcolumn of the second matrix. More precisely,

the i-th row of the first matrix is the row vector:

[ai1, ai2, . . . , ain

] column vector=⇒

ai1ai2...ain

∈ Rn

the j-th column of the second matrix is

b1jb2j...bnj

∈ Rn

compute the dot productai1...ain

·b1j

...bnj

= ai1b1j + ai2b2j + · · ·+ ainbnj = cij .


Example

Example

[1 3 52 4 6

]·

1 23 42 42 6

=? dimensions: 2×3 and 4×2 3 6= 4 =⇒ not defined.

RemarkThis is why we have dot and cross product for vectors but not regular“multiplication” .

u =

u1...un

v =

v1...vn

Both have dimension n × 1.

# of colums in u = 1 6= n = # of rows in v


Example

[2 1 1 −10 2 1 4

]1 0 10 1 21 0 01 1 0

=?

First of all, the dimensions are 2× 4 and 4× 3. The result is a matrix ofdimension 2× 3. Now, we write down the two rows of the first matrix:

R1 :[2 1 1 −1

]R2 :

[0 2 1 4

] column vectors=⇒ v1 =

211−1

v2 =

0214

and the three column vectors from the second matrix

w1 =

1011

, w2 =

0101

, w3 =

1200


Example

v1 =

211−1

v2 =

0214

w1 =

1011

, w2 =

0101

, w3 =

1200

[2 1 1 −10 2 1 4

]1 0 10 1 21 0 01 1 0

=

[a11 a12 a13

a21 a22 a23

]

a11 = v1 · w1 = 2 · 1 + 1 · 0 + 1 · 1 + (−1) · 1 = 2

a12 = v1 · w2 = 2 · 0 + 1 · 1 + 1 · 0 + (−1) · 1 = 0

a13 = v1 · w3 = 2 · 1 + 1 · 2 + 1 · 0 + (−1) · 0 = 4

a21 = v2 · w1 = 0 · 1 + 2 · 0 + 1 · 1 + 4 · 1 = 5

a22 = v2 · w2 = 0 · 0 + 2 · 1 + 1 · 0 + 4 · 1 = 6

a23 = v2 · w3 = 0 · 1 + 2 · 2 + 1 · 0 + 4 · 0 = 4

[2 1 1 −10 2 1 4

]1 0 10 1 21 0 01 1 0

‖[

v1 · w1 v1 · w2 v1 · w3

v2 · w1 v2 · w2 v2 · w3

]‖[

2 0 45 6 4

]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 21st, 2019 13 / 17

Alternative calculation

column-method: Let A be an m × n matrix and v ∈ Rn is a vector, which can betreated as an n × 1 matrix. Av is an m × 1 matrix as well as a vector in Rm.row-method: Let A be an m × n matrix and u a 1×m matrix. Namelyu =

[u1 · · · um

]is a row vector. uA is a 1× n matrix as well as a row vector

in Rn.

Example

[2 1 1 −10 2 1 4

]1011

=

[2 1 1 −10 2 1 4

]1011

= 1

[20

]+ 0

[12

]+ 1

[11

]+ 1

[−14

]=

[20

]+

[00

]+

[11

]+

[−14

]=

[25


Alternative calculation

[2 1 1 −10 2 1 4

]1011

= 1[20

]+ 0

[12

]+ 1

[11

]+ 1

[−14

]=

[25

]

[2 1 1 −10 2 1 4

]0101

= 0[20

]+ 1

[12

]+ 0

[11

]+ 1

[−14

]=

[12

]+

[−14

]=

[06

]

[2 1 1 −10 2 1 4

]1200

= 1[20

]+ 2

[12

]+ 0

[11

]+ 0

[−14

]=

[20

]+

[24

]=

[44

]

[2 1 1 −10 2 1 4

]︸︷︷︸

A

1 0 10 1 21 0 01 1 0

↑ ↑ ↑w1 w2 w3

=[Aw1 Aw2 Aw3

]=

[2 0 45 6 4

]


Row and column methods

Row-method and column-method: Read Proposition 4.23 and 4.29. Let Am×n andBn×` be matrices such that

A =

a1...

am

and B =[b1 · · · b`

], for

{row vectors a1, . . . , ancolumn vectors b1, . . . ,b`

in Rn

AB =

colume method︷︸︸︷[Ab1 · · · Ab`

]=

row method︷︸︸︷a1B...

amB

=

a1b1 · · · a1b`

.... . .

...amb1 · · · amb`

Example (Column method)

[2 1 1 −10 2 1 4

]A

1 0 10 1 21 0 01 1 0

w1 w2 w3

= [Aw1 Aw2 Aw3] Aw1 = 1[20

]+ 0

[12

]+ 1

[11

]+ 1

[−14

]=

[25

]


Example (row method)

[2 1 1 −10 2 1 4

]1 0 10 1 21 0 01 1 0

=

[2 0 45 6 4

]

[2 1 1 −1

] 1 0 10 1 21 0 01 1 0

= 2[1 0 1

]+ 1

[0 1 2

]+ 1

[1 0 0

]− 1

[1 1 0

]=

[2 0 2

]+[0 1 2

]+[1 0 0

]−[1 1 0

]=

[2 0 4

][0 2 1 4

] 1 0 10 1 21 0 01 1 0

= 0[1 0 1

]+ 2

[0 1 2

]+ 1

[1 0 0

]+ 4

[1 1 0

]=

[0 0 0

]+[0 2 4

]+[1 0 0

]+[4 4 0

]=

[5 6 4



Lin Jiu


May 23rd, 2019

Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 1 / 33

Multiplication

A =

a11 · · · a1n...

. . ....

am1 · · · amn

m×n

, B =

b11 · · · b1`...

. . ....

bn1 · · · bn`

k×`

AB exists ONLY if n = k .

AB =

c11 · · · c1`...

. . ....

cm1 · · · cm`

m×`

, cij = ai1b1j + · · ·+ ainbnj

formula, row method, column method

A =

a1...

am

B =

[b1 · · · b`

]⇒ AB =

a1B...

amB

=[Ab1 · · · Ab`

]=

a1b1 · · · a1b`

.... . .

...amb1 · · · amb`


Example[1 23 4

] [1 01 1

]=?

First of all the dimensions are 2× 2 and 2× 2. ⇒ Result: a 2× 2-matrix.

A =

[1 23 4

]⇒{

a1 = [1 2]a2 = [3 4]

B =

[1 01 1

]⇒ b1 =

[11

], b2 =

[01

](I) Formula:

a1b1 =

[12

]·[11

]= 1 · 1+ 2 · 1 = 3

a1b2 =

[12

]·[01

]= 1 · 0+ 2 · 1 = 2

a2b1 =

[34

]·[11

]= 3 · 1+ 4 · 1 = 7

a2b2 =

[34

]·[01

]= 3 · 0+ 4 · 1 = 4

Thus,

AB =

[a1b1 a1b2a2b1 a2b2

]=

[3 27 4

]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 3 / 33

Example

A =

[1 23 4

]⇒{

a1 = [1 2]a2 = [3 4]

B =

[1 01 1

]⇒ b1 =

[11

], b2 =

[01

](II) Column method: AB = A [b1 b2] = [Ab1 Ab2]

Ab1 =

[1 23 4

] [11

]= 1

[13

]+ 1

[24

]=

[37

]Ab2 =

[1 23 4

] [01

]= 0

[13

]+ 1

[24

]=

[24

] ⇒ AB =

[3 27 4

]

(III) Row method: AB =

[a1a2

]B =

[a1Ba2B

]a1B = [1 2]

[1 01 1

]= 1 [1 0] + 2 [1 1] = [1 0] + [2 2] = [3 2]

a2B = [3 4][1 01 1

]= 3 [1 0] + 4 [1 1] = [3 0] + [4 4] = [7 4]

⇒ AB =

[3 27 4


Remarks

1 Let

v =

a1...an

, w =

b1...bn

∈ Rn =⇒ v ·w = a1b1 + · · ·+ anbn

[a1 · · · an

]︸︷︷︸1×n

b1...bn

︸︷︷︸n×1

= [v ·w]

We identify a 1× 1 matrix and a scalar.2 Recall our goal:

3x + 2y + 4z = 7x − y + z = 2

0x + 2y + 3z = 5⇔

3 2 41 −1 10 2 3

·xyz

=

725

3 2 41 −1 10 2 3

·

xyz

= x

310

+ y

2−12

+ z

413

(columnmethod

)=

3x + 2y + 4zx − y + z

0x + 2y + 3z


Identity matrix

DefinitionThe n-th identity matrix is a square matrix of dimension n × n, given by

In =

1 0 0 · · · 00 1 0 · · · 00 0 1 · · · 0...

......

. . ....

0 0 0 · · · 1

n×n

Example

I2 =

[1 00 1

]I3 =

1 0 00 1 00 0 1

I4 =

1 0 0 00 1 0 00 0 1 00 0 0 1


Laws

1 (AB)C = A(BC )

2 Most of the times,AB 6= BA

3 A(B + C ) = AB + AC

4 (A+ B)C = AC + BC

5 k(AB) = (kA)B = A(kB)

6 If A has dimension m × n,AIn = A = ImA.

ALWAYS remember that sincematrix multiplication is not com-mutative, we need to specify mul-tiplying a matrix to the LEFT orto the RIGHT.

Example

1.[0 10 0

][1 23 4

]=

[3 40 0

][1 23 4

] [0 10 0

]=

[0 10 3

]2.[a b

] [cd

]=[ac + bd

][cd

] [a b

]=

[ac bcad bd

]3.[0 10 0

] [13

]=

[30

][13

] [0 10 0

]Not defined

4.[

1 00 1

] [1 2 32 4 1

]=

[1 2 32 4 1

][1 2 32 4 1

] 1 0 00 1 00 0 1

=

[1 2 32 4 1


Remark

Please REMEMBER: matrix multiplication is totally different from themultiplication of numbers/scalars.

3 · 4 = 12 = 4 · 3 but most of the times AB 6= BA.x2 = 0⇒ x = 0, but A2 = AA = 0 does not mean A = 0.

A =

[0 10 0

]⇒ A2 =

[0 · 0+ 1 · 0 0 · 1+ 1 · 00 · 0+ 0 · 0 0 · 1+ 0 · 0

]=

[0 00 0

]This actually

shows we can have A 6= 0 6= B, but AB = 0Recall that, for numbers, if xy = 0 then either x = 0 or y = 0.ab = ac ⇒ b = c if a 6= 0 but AB = AC does not imply that B = C .AB = AC ⇒ AB − AC = 0 ⇒ A (B − C ) = 0

Let A =

[0 10 0

], B =

[0 30 0

], C =

[0 20 0

]⇒ B − C = A

and A(B − C ) = AA = A2 = 0.Exercise 4.4.12–Exercise 4.4.18


Simplification

(AB)C = A(BC ) A(B + C ) = AB + AC k(AB) = (kA)B = A(kB)AB 6= BA (A+ B)C = AC + BC AIn = A = ImA

Example

Let X be 2× 2, satisfying 3X[2 13 4

]+

[23

] [1 4

]= X

[1 23 4

] [−1 53 −1

]X =?

3X[2 13 4

]= X

(3[2 13 4

])= X

[6 39 12

][23

] [1 4

]=

[2 · 1 2 · 43 · 1 3 · 4

]=

[2 83 12

][1 23 4

] [−1 53 −1

]=

[1 · (−1) + 2 · 3 1 · 5+ 2 · (−1)3 · (−1) + 4 · 3 3 · 5+ 4 · (−1)

]=

[5 39 11

]X

[6 39 12

]+

[2 83 12

]= X

[5 39 11

]⇒ X

[6 39 12

]− X

[5 39 11

]= −

[2 83 12

]⇒[−2 −8−3 −12

]= X

([6 39 12

]−[5 39 11

])= X

[1 00 1

]= X .


Further question

3X[2 13 4

]+

[23

] [1 4

]= X

[1 23 4

] [−1 53 −1

]⇒ X = −

[2 83 12

]Problem

How about 3X[2 03 4

]+

[23

] [1 4

]= X

[1 23 4

] [−1 53 −1

]?

3X[2 03 4

]− X

[1 23 4

] [−1 53 −1

]= X

[6 09 12

]− X

[5 39 11

]= X

[1 −30 1

]Namely, it holds that X

[1 −30 1

]= −

[23

] [1 4

]= −

[2 83 12

]Of course, we can set X =

[a bc d

]and observe that

−[2 83 12

]=

[a bc d

] [1 −30 1

]=

[a −3a+ bc −3c + d

]Use the FOUR equations to solve for a, b, c , and d . Or, we can introduce theinverse matrix.


Matrix Inverses

Definition1. Let A be a matrix. We say that A is left invertible if there exists a matrix Bsuch that

BA = I (Here we do not specify the dimension)

Then B is called a left inverse of A. Am×n ⇒ Bn×m2. A is right invertible if there exists a matrix B such that

AB = I .

Then B is called a right inverse of A.3. A is invertible if it has both left and right inverses. In this case, A is a squarematrix (Theorem 4.48) and both left and right inverses coincide. Namely, thereexist a (square) matrix B (with the same dimension as A) such that

AB = BA = I . We write B = A−1.

FactNot every matrix is invertible. Not every square matrix is invertible.


Examples

1 Let A =

[1 0 00 1 0

]and B =

1 00 10 0

. We can compute that

AB =

[1 0 00 1 0

] 1 00 10 0

=

[1 00 1

]= I

and

BA =

1 00 10 0

[ 1 0 00 1 0

]=

1 0 00 1 00 0 0

6= I

B is a right inverse of A, but not a left inverse.

2 Let A =

[1 −30 1

]and B =

[1 30 1

].

AB =

[1 −30 1

] [1 30 1

]=

[1 · 1− 3 · 0 1 · 3− 3 · 10 · 1+ 1 · 0 0 · 3+ 1 · 1

]=

[1 00 1

]= I = BA.


Example

[1 −30 1

] [1 30 1

]=

[1 30 1

] [1 −30 1

]= I ⇒

[1 30 1

]=

[1 −30 1

]−1

3X[2 03 4

]+

[23

] [1 4

]= X

[1 23 4

] [−1 53 −1

]⇒ X

[1 −30 1

]= −

[2 83 12

]Now, multiplying both sides by

[1 30 1

], to the right

X

[1 −30 1

] [1 30 1

]= −

[2 83 12

] [1 30 1

]The left-hand side is X

[1 −30 1

] [1 30 1

]= XI = X

The right-hand side is

−[2 83 12

] [1 30 1

]= −

[2 · 1+ 8 · 0 2 · 3+ 8 · 13 · 1+ 12 · 0 3 · 3+ 12 · 1

]= −

[2 143 21

]X =

[−2 −14−3 −21


Find the inverses

Given A =

1 2 10 1 12 4 3

, check if A is invertible. If so, find the inverse.

Method: Make an augmented system[A | I

]. Attempt row reductions to get[

I | B]. If this succeeds, then A is invertible and B = A−1.

[A | I

]=

1 2 1 1 0 00 1 1 0 1 02 4 3 0 0 1

R3←R3−2R1∼ 1 2 1 1 0 0

0 1 1 0 1 00 0 1 −2 0 1

R1←R1−2R2∼

1 0 −1 1 −2 00 1 1 0 1 00 0 1 −2 0 1

R1←R1+R3R2←R2−R3∼

1 0 0 −1 −2 10 1 0 2 1 −10 0 1 −2 0 1

A is invertible with A−1 =

−1 −2 12 1 −1−2 0 1

Exercise: check AA−1 = A−1A = I


Find the inverses

Given A =

[1 −30 1

], check if A is invertible. If so, find the inverse.

[A | I

]=

[1 −3 1 00 1 0 1

]R1←R1+3R2∼

[1 0 1 30 1 0 1

]⇒ A−1 =

[1 30 1

]

RemarkAfter Midterm II, we will study an alternative method to compute the inverse,which sometimes is simpler for 2× 2 matrices.

Example

Let A =

1 −2 21 0 22 −2 4

. Find A−1 if it exists.


Example

[A | I

]=

1 −2 2 1 0 01 0 2 0 1 02 −2 4 0 0 1

R2←R2−R1R3←R3−2R1∼

1 −2 2 1 0 00 2 0 −1 1 00 2 0 −2 0 1

R3←R3−R2∼

1 −2 2 1 0 00 2 0 −1 1 00 0 0 −1 −1 1

From here, we see rank(A) = 2 < 3. So the (unique) reduced echelon form of A is1 0 0

0 1 00 0 0

6=1 0 00 1 00 0 1

.

There is no way to obtain I on the left-hand side of this augmented matrix.Hence, there is no way to complete the algorithm, and the inverse of A does notexist. Namely, A is NOT invertible.Remark (A being n × n )If rank(A) = n,

[A | I

]∼[I | B

]succeeds to get B = A−1. (full rank)

If rank(A) < n, then A is not invertible.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 23rd, 2019 16 / 33

Laws & Properties

1 Inverse is unique. If both B and C are inverses of A, then B = C .

Proof.C = IC = (BA)C = B(AC ) = BI = B.

2 I−1 = I .3 If both A and B are invertible, (AB)−1 = B−1A−1. (B−1A−1AB = I .)

(1/(xy) = (1/x) (1/y))4 If A is invertible, so is A−1. Moreover, (A−1)−1 = A.5 If A is invertible and k 6= 0. (kA)−1 = k−1A−1 = 1

kA−1.

Theorem1 If A is invertible, then A is a square matrix.2 If A is a square matrix, then the following are equivalent:

A is invertible;A has a left inverse;A has a right inverse;

3 Let A be an m × n matrix.If A is left invertible, then m ≥ n.If A is right invertible, then m ≤ n.If A is invertible, then m = n.

rank(A) ≤ m, rank(A) ≤ nleft invertible iff rank(A) = nright invertible iff rank(A) = m


Classwork

Let

A =

1 0 11 −1 11 1 −1

Determine whether A is invertible. If A isinvertible, find A−1.

Lecture resumes at 7:50


Solution

[A | I

]=

1 0 1 1 0 01 −1 1 0 1 01 1 −1 0 0 1

R2←R2−R1R3←R3−R1∼

1 0 1 1 0 00 −1 0 −1 1 00 1 −2 −1 0 1

R2←−R2∼

1 0 1 1 0 00 1 0 1 −1 00 1 −2 −1 0 1

R3←R3−R2∼

1 0 1 1 0 00 1 0 1 −1 00 0 −2 −2 1 1

R3←− 1

2R3∼ 1 0 1 1 0 0

0 1 0 1 −1 00 0 1 1 − 1

2 − 12

R1←R1−R3∼

1 0 0 0 12

12

0 1 0 1 −1 00 0 1 1 − 1

2 − 12

A−1 =

0 12

12

1 −1 01 − 1

2 − 12


Solve system of equations

Consider the systemx + z = 1

x − y + z = 3x + y − z = 2

. We can write it as Aa = b where

A =

1 0 11 −1 11 1 −1

, a =

xyz

, b =

132

.

From the previous slide, we have

A−1 =

0 12

12

1 −1 01 − 1

2 − 12

Multiplying both sides of Aa = b by A−1 to the LEFT:

A−1Aa = A−1b⇐⇒ a = A−1b

a =

xyz

= A−1b =

0 12

12

1 −1 01 − 1

2 − 12

132

=

0 · 1+ 12 · 3+

12 · 2

1 · 1− 1 · 3+ 0 · 21 · 1− 1

2 · 3−12 · 2

=

52−2− 3

2

.


Solve system of equations

How about another systemx + z = 0

x − y + z = 1x + y − z = 3

?

013

xyz

= A−1

013

=

0 12

12

1 −1 01 − 1

2 − 12

013

=

2−1−2

This illustrates that for a system Ax = b where A−1 exists, it is easy to find thesolution when the vector b is changed.

x + z = 0x − y + z = 0x + y − z = 0

(homogeneous)

⇒

xyz

= A−1

000

=

0 12

12

1 −1 01 − 1

2 − 12

000

=

0 · 0+ 12 · 0+

12 · 0

1 · 0− 1 · 0+ 0 · 01 · 0− 1

2 · 0−12 · 0

=

000


Application (Solve matrix equation)

Find a matrix X such that [1 11 2

]X =

[1 23 4

]

Let A =

[1 11 2

]and C =

[1 23 4

].

AX = C =⇒ A−1AX = A−1C = IX = X .[1 1 1 01 2 0 1

]R2←R2−R1∼

[1 1 1 00 1 −1 1

]R1←R1−R2∼

[1 0 2 −10 1 −1 1

]Thus,

X = A−1C =

[2 −1−1 1

] [1 23 4

]=

[2 · 1− 1 · 3 2 · 2− 1 · 4−1 · 1+ 1 · 3 −1 · 2+ 1 · 4

]=

[−1 02 2

].


Application (Solve matrix equation)

Find a matrix X such that[1 11 2

]X =

[1 23 4

]A =

[1 11 2

]

Let X =

[a bc d

]. Recall the column method:

[1 11 2

] [ac

]=

[13

] [1 11 2

] [bd

]=

[24

][13

]=

[1 11 2

] [ac

]= a

[11

]+ c

[12

]=

[a+ ca+ 2c

]⇒ c = 2, a = −1

[24

]=

[1 11 2

] [bd

]= b

[11

]+ d

[12

]=

[b + db + 2d

]⇒ d = 2, b = 0


Brief Explanation

Consider the example of A =

1 2 10 1 12 4 3

. Let B =

a b cd e fg h i

such that

AB = A

a b cd e fg h i

= I =

1 0 00 1 00 0 1

.

By the column method,

A

adg

=

100

A

beh

=

010

A

cfi

=

001

it is equivalent to three systems (with the same LHS A): Recall the reducedechelon form 1 2 1 1

0 1 1 02 4 3 0

Row operations∼∼∼∼∼∼

1 0 0 a0 1 0 d0 0 1 g


Brief Explanation

Similarly 1 2 1 00 1 1 12 4 3 0


1 0 0 b0 1 0 e0 0 1 h

1 2 1 0

0 1 1 02 4 3 1


1 0 0 c0 1 0 f0 0 1 i

KEY: The row operations for the three systems are the same, completelydetermined by A (the coefficients) 1 2 1 1 0 0

0 1 1 0 1 02 4 3 0 0 1


1 0 0 a b c0 1 0 d e f0 0 1 g h i

A−1 =

a b cd e fg h i


Symbolic matrix equation

Assume all matrices A, B, C and X are n × n and also assume all matrices areinvertible as needed. If X satisfies AX (X + C )−1 = B, solve for X .Solution. Multiplying both sides by X + C to the right

AX (X + C )−1 (X + C ) = B (X + C ) (∗)

AX = BX + BC =⇒ AX − BX = BC =⇒ (A− B)X = BC

Multiply both sides by (A− B)−1 to the left

(A− B)−1 (A− B)X = (A− B)−1BC = X . (∗∗)

X = (A− B)−1BC

If you multiply matrices to the wrong side, you will have either in (∗)

(X + C )AX (X + C )−1 = (X + C )B

or in (∗∗)(A− B)X (A− B)−1 = BC (A− B)−1


Transpose

DefinitionThe transpose of an m × n matrix A, denoted by AT , is an n ×m matrix, whose(i , j)-entry is just the (j , i)-entry of A. Namely, it is obtained from A by switchingrows and columns, , i.e., the i-th column of AT is the i-th row of A. Moreprecisely,

A =

a11 · · · a1na21 · · · a2n...

. . ....

am1 · · · amn

=⇒ AT =

a11 a21 · · · am1...

.... . .

...a1n a2n · · · amn

A =

1 2 34 5 67 8 9

=⇒ AT =

1 4 72 5 83 6 9


Transpose

Example

B =

[1 0 3−2 1 0

]=⇒ BT =

1 −20 13 0

v =

1024

=⇒ vT =[1 0 2 4

] [1 0 2 4

]T=

1024

= v

RemarkNow we see that the transpose of a row ( resp. column) vector is a column(resp. row) vector.


Dot product

Recall that for v =

v1...vn

and w =

w1...wn

v ·w = v1w1 + · · ·+ vnwn =[v1 · · · vn

] w1...wn

= vTw.

RemarkTo save space, we sometimes use the transpose of a row vector to express thecorresponding column vector. Namely,12

3

=[1 2 3

]T.


Laws of transposes

(A+ B)T = AT + BT (rA)T = rAT

(AT )T = A (AB)T = BTAT

0T = 0 IT = I (A−1)T = (AT )−1

ExampleLet

A =

1 30 1−1 0

B =

[1 23 −1

]=⇒ C = AB =

10 −13 −1−1 −2

BT =

[1 32 −1

]AT =

[1 0 −13 1 0

]=⇒ BTAT =

[10 3 −1−1 −1 −2

]= CT

Namely, Let v =[0 1

]Tand w =[1 3

]T .v ·w = 3 = w · v ⇔ vTw = 3 = wTv.

v ·w = vTw = 3 = [3] = [3]T =(vTw

)T= wT

(vT)T

= wTv = w · v


(Anti-)Symmetric matrices

DefinitionA matrix A is called symmetric if AT = A. It is said to be antisymmetric(sometimes also called skew symmetric) if AT = −A.

Example

A =

1 2 32 4 103 10 15

I4 =

1 0 0 00 1 0 00 0 1 00 0 0 1

0 =

[0 00 0

]are symmetric.

B =

0 2 3−2 0 10−3 −10 0

C =

0 1 3−1 0 2−3 −2 0

0 =

[0 00 0

]are antisymmetric.

[1 20 1

]is neither.


Trace (*)

DefinitionLet A be an n× n-matrix (square matrix). The trace, denoted by tr(A) is the sumof its diagonal entries.

A =

a11 · · · a1n...

. . ....

an1 · · · ann

=⇒ tr(A) = a11 + a22 + · · ·+ ann

Example

A =

1 2 32 4 103 10 15

=⇒ tr(A) = 1+ 4+ 15 = 20.


Summary of Chpt. 3

The only object is matrix.1 dimension (check before calculation), entry, row vectors & column vectors.2 addition & scalar multiplication similar to those for vectors3 multiplication:

requirement for dimensionscalculation formula, row & column methods

4 inversedefinition and laws/propertiesfind the inverseapplications: solving linear systems matrix equations (including symbolicequations )

5 transpose



Lin Jiu


May 28th, 2019


Span

DefinitionGiven vectors v1, . . . , vk in Rn, the span of v1, . . . , vk is the set of all their linearcombinations. In symbols,

span {v1, . . . , vk} = {a1v1 + a2v2 + · · ·+ akvk | a1, . . . , ak ∈ R} .

Example

Which of the following vectors are in the span of v1 =

122

and v2 =

202

?(a) u =

524

(b)w =

323

(c) t =

1−20

What do we want?

Determine whether there exist a1 and a2 such that

u = a1v1 + a2v2 w = a1v1 + a2v2 t = a1v1 + a2v2


Solution

v1 = [1 2 2]T , v2 = [2 0 2]T , u = [5 2 4]T , w = [3 2 3]T , t = [1 − 2 0]T .

a1v1 + a2v2 = a1

122

+ a2

202

=

1 22 02 2

[a1a2

]

(a)

1 2 52 0 22 2 4

(b)

1 2 32 0 22 2 3

(c)

1 2 12 0 −22 2 0

1 2 5 3 1

2 0 2 2 −22 2 4 3 0

R2←R2−2R1R3←R3−2R1∼

1 2 5 3 10 −4 −8 −4 −40 −2 −6 −3 −2

R2←− 1

4 R2R3←− 1

2 R3∼ 1 2 5 3 1

0 1 2 1 10 1 3 3

2 1

R3←R3−R2∼

1 2 5 3 10 1 2 1 10 0 1 1

2 0


Solution

v1 = [1 2 2]T , v2 = [2 0 2]T , u = [5 2 4]T , w = [3 2 3]T , t = [1 − 2 0]T . 1 2 5 3 12 0 2 2 −22 2 4 3 0

∼∼∼ 1 2 5 3 1

0 1 2 1 10 0 1 1

2 0

Only the system of (c) is consistent. Therefore, only t is the in the span of v1 andv2. t ∈ span {v1, v2}What is the solution? (a1, a2) = (−1, 1)

v2−v1 = [2 0 2]T−[1 2 2]T =([1 2 2]− [2 0 2]

)T= [1 −2 0]T = t

Is this solution important?No, the key is to know whether the system is consistent.


Geometric Interpretation of Span

Let v1 and v2 be non-parallel vectors. Then, span {v1, v2}, the set of all linearcombinations, is a plane through the origin.


Geometric Interpretation of Span

Remark

The span of 1 vector is usually a line. (span{0} is just the origin.)The span of 2 vectors is usually a plane. (It could be a line if both areparallel) ( If v = ku for some k (parallel), then au + bv = au + bku= (a+ bk)u — a line)The span of 3 vectors is usually a 3-dimensional space. (It could be a planeor a line)Higher-dimensional spaces:

Example (in R6)

v1 = [ 1 0 1 2 3 4 ]T , v2 = [ 1 1 1 2 2 2 ]T ,

v3 = [ 0 0 1 3 7 4 ]T , v4 = [ 1 0 1 0 3 0 ]T .

span {v1, v2, v3, v4}Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 6 / 34

Redundancy

Example

v1 =

110

, v2 =

012

, v3 =

122

.

They are not mutually parallel. However, it is not hard to see the fact:

v1 + v2 =

110

+

012

=

122

= v3

Thus, v3 is “redundant”, i.e., span {v1, v2, v3} = span {v1, v2}.span {v1, v2} ⊆ span {v1, v2, v3}: a1v1 + a2v2 = a1v1 + a2v2 + 0v3

span {v1, v2, v3} ⊆ span {v1, v2}:a1v1 + a2v2 + a3v3 = a1v1 + a2v2 + a3 (v1 + v2) = (a1 + a3) v1 +(a2 + a3) v2

RemarkThis is an example that a span of 3 vectors is a plane.


Redundancy

DefinitionLet v1, . . . , vk be vectors and consider span {v1, . . . , vk}. We say that the vectorvj is redundant if vj is a linear combination of the vectors preceding it, i.e., vj isa linear combination of v1, . . . , vj−1. In this case, vj contributes nothing to thespan, i.e., span {v1, . . . , vk} = span {v1, . . . , vj−1, vj+1, . . . vk} .

ExampleFind the redundant vectors in the sequence

v1 =

110

, v2 =

220

, v3 =

101

, v4 =

−1−21

, v5 =

111

v1 is not redundant. There is no v0. In fact, v1 is never redundant unlessv1 = 0. span∅ = {0}v2 = 2v1, so it is redundant.v3 is not a linear combination of v1 and v2, so it is not redundant. why?


Example

v1 =

110

, v2 =

220

, v3 =

101

, v4 =

−1−21

, v5 =

111

v1 is not redundant. v2 is redundant. v3 is not redundant. 3rd component

v4? 1 2 1 −11 2 0 −20 0 1 1

R2←R2−R1∼ 1 2 1 −1

0 0 −1 −10 0 1 1

R3←R3+R2∼ 1 2 1 −1

0 0 −1 −10 0 0 0

consistent=⇒ redundant.v5? 1 2 1 −1 1

1 2 0 −2 10 0 1 1 1

R2←R2−R1∼ 1 2 1 −1 1

0 0 −1 −1 00 0 1 1 1

R3←R3+R2∼ 1 2 1 −1 1

0 0 −1 −1 00 0 0 0 1

inconsistent=⇒not redundant.

span {v1, v2, v3, v4, v5} = span {v1, v3, v5}


Casting-out algorithm

Input: A finite sequence of vectors v1, . . . , vk in Rn

Output: Redundant vectorsSteps: (1) Form an n × k matrix whose columns are the k given vectors.

(2) Reduce it to an echelon form.(3) Non-pivot columns correspond to redundant vectors.

ExampleFind the redundant vectors in the sequence

v1 =

110

, v2 =

220

, v3 =

101

, v4 =

−1−21

, v5 =

111

1 2 1 −1 1

1 2 0 −2 10 0 1 1 1

R2←R2−R1∼1 2 1 −1 1

0 0 −1 −1 00 0 1 1 1

R3←R3+R2∼1 2 1 −1 1

0 0 −1 −1 00 0 0 0 1

Non-pivot vectors are v2 and v4 , which are redundant.span {v1, v2, v3, v4, v5} = span {v1, v3, v5}


Example

Find the redundant vectors in the sequence

v1 =

0000

, v2 =

0110

, v3 =

0220

, v4 =

1111

, v5 =

1331

, v6 =

1234

0 0 0 1 1 10 1 2 1 3 20 1 2 1 3 30 0 0 1 1 4

R1↔R2∼

0 1 2 1 3 20 0 0 1 1 10 1 2 1 3 30 0 0 1 1 4

R3←R3−R1∼

0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 1 1 4

R4←R4−R2∼

0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 3


Example

Find the redundant vectors in the sequence

v1 =

0000

, v2 =

0110

, v3 =

0220

, v4 =

1111

, v5 =

1331

, v6 =

1234

0 0 0 1 1 10 1 2 1 3 20 1 2 1 3 30 0 0 1 1 4

∼∼∼0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 3

R4←R4−3R3∼0 1 2 2 3 20 0 0 1 1 10 0 0 0 0 10 0 0 0 0 0

Non-pivot vectors are v1, v3 and v5 , which are redundant.

span {v1, v2, v3, v4, v5, v6} = span {v2, v4, v6}

Remarkspan {v1, v2, v3} = {a1v1 + a2v2 + a3v3 | a1, a2, a3 ∈ R}

span {v1, v2} = {a1v1 + a2v2 | a1, a2 ∈ R}span {v1} = {a1v1 | a1 ∈ R}

span∅ = {0} the set containing only the zero vector.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 12 / 34

Linear (in)dependence

DefinitionA set of vectors {v1, . . . , vk} is called

linearly independent if there are no redundant vectors;linearly dependent if it is not linearly independent, i.e., if there are redundantvectors.

ExampleAre the following vectors linearly independent?

v1 =

110

, v2 =

022

, v3 =

10−1

1 0 11 2 00 2 −1

R2←R2−R1∼ 1 0 1

0 2 −10 2 −1

R3←R3−R2∼ 1 0 1

0 2 −10 0 0

Since v3 is redundant, we see they are linearly dependent.


Linear (in)dependence

DefinitionA set of vectors {v1, . . . , vk} is called

linearly independent if there are no redundant vectors;linearly dependent if it is not linearly independent, i.e., if there are redundantvectors.

ExampleAre the following vectors linearly independent?

v1 =

110

, v2 =

022

, v3 =

10−1

1 0 1

1 2 00 2 −1

R2←R2−R1∼ 1 0 1

0 2 −10 2 −1

R3←R3−R2∼ 1 0 1

0 2 −10 0 0

Since v3 is redundant, we see they are linearly dependent.


Extended casting-out algorithm

The ordinary casting-out algorithm.Except we go to the reduced echelon form.The entries of the non-pivot columns determine the coefficients for writingeach redundant vector as a linear combination of preceding non-redundantvectors.

ExampleConsider the vectors

v1 =

1000

, v2 =

1111

, v3 =

0222

, v4 =

3111

(a) Is the set {v1, v2, v3, v4} linearly independent? v4 = 2v1 + v2(b) If not, show how each redundant vector can be written as a linear combinationof preceding vectors.


Example

v1 =

1000

, v2 =

1111

, v3 =

0222

, v4 =

3111

Solution(a) By the casting-out algorithm,

1 1 0 30 1 2 10 1 2 10 1 2 1

R3←R3−R2R4←R4−R2∼

1 1 0 30 1 2 10 0 0 00 0 0 0

⇒ v3 and v4

are redundant.

(b) Proceed to the reduced echelon form:1 1 0 30 1 2 10 0 0 00 0 0 0

R1←R1−R2∼1 0 −2 20 1 2 10 0 0 00 0 0 0

⇒{

v3 = (−2)v1 + 2v2

v4 = 2v1 + v2


Remark

You might (or might not) get confused, by comparing with an early example

2x + y + w = 8x + y + z = 6

3x + 2y + z + v = −2→

2 1 0 1 0 81 1 1 0 0 63 2 1 0 1 −2

∼ 1 0 −1 0 1 −14

0 1 2 0 −1 200 0 0 1 −1 16

z = s and v = t =⇒

xyzwv

=

−14200160

+ t

−11011

+ s

1−2100

We have to switch the signs for the entries, except for those on the right-handside.

1 1 0 30 1 2 10 1 2 10 1 2 1

∼1 0 −2 20 1 2 10 0 0 00 0 0 0

⇒{

v3 = (−2)v1 + 2v2

v4 = 2v1 + v2


Remark

The reason is: when expressing v3 and v4 as linear combinations of v1 and v2,they ARE on the right-hand side.

v1 =

1000

, v2 =

1111

, v3 =

0222

, v4 =

3111

v3 = a1v1 + a2v2 = a1

1000

+ a2

1111

⇔1 10 10 10 1

[a1a2

]=

0222

→

1 1 00 1 20 1 20 1 2

1 1 00 1 20 1 20 1 2

R3←R3−R2R4←R4−R2∼

1 1 00 1 20 0 00 0 0

R1←R1−R2∼

1 0 −20 1 20 0 00 0 0

⇒v3‖

(−2)v1+2v2


Classwork

v1 =

213

, v2 =

112

, v3 =

011

, v4 =

100

, v5 =

001

(a) Is the set {v1, v2, v3, v4, v5} linearly independent?

(b) If not, show how each redundant vector can be written

as a linear combination of preceding vectors.


Solution

v1 =

213

, v2 =

112

, v3 =

011

, v4 =

100

, v5 =

001

2 1 0 1 0

1 1 1 0 03 2 1 0 1

R1↔R2∼ 1 1 1 0 0

2 1 0 1 03 2 1 0 1

R2←R2−2R1R3←R3−3R1∼

1 1 1 0 00 −1 −2 1 00 −1 −2 0 1

R2←−R2∼

1 1 1 0 00 1 2 −1 00 −1 −2 0 1

R3←R3+R2R1←R1−R2∼

1 0 −1 1 00 1 2 −1 00 0 0 −1 1


Solution

v1 =

213

, v2 =

112

, v3 =

011

, v4 =

100

, v5 =

001

2 1 0 1 0

1 1 1 0 03 2 1 0 1

∼ 1 0 −1 1 0

0 1 2 −1 00 0 0 −1 1

R3←−R3∼

1 0 −1 1 00 1 2 −1 00 0 0 1 −1

R2←R2+R3R1←R1−R3∼

1 0 −1 0 10 1 2 0 −10 0 0 1 −1

v3 = −v1 + 2v2 = −

213

+ 2

112

=

011

, v5 = v1 − v2 − v4


Alternative characterization

TheoremThe vectors v1, . . . , vk are linearly independent iff (if and only if) the vectorequation

a1v1 + a2v2 + · · ·+ akvk = 0

has only the trivial solution. Namely it only has the solution a1 = · · · = ak = 0.

Example

v1 =

110

v2 =

123

v3 =

−3−23

The equation above (k = 3) becomes

a1v1 + a2v2 + a3v3 = 0⇐⇒

1 1 −31 2 −20 3 3

a1a2a3

=

000

Homogeneous

Check the rank of the coefficient matrix. Find its (reduced) echelon form:Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 21 / 34

Example

v1 =

110

v2 =

123

v3 =

−3−23

1 1 −31 2 −20 3 3

R2←R2−R1∼1 1 −30 1 10 3 3

R3←R3−3R1∼1 1 −30 1 10 0 0

R1←R1−R2∼1 0 −40 1 10 0 0

The rank is 2 < 3, so the vector equation has non-trivial solutions

The three vectors, v1, v2 and v3 are linearly dependent.In particular,

v3 = −4v1 + v2 ⇐⇒ 4v1 − v2 + v3 = 0⇐⇒ (a1, a2, a3) = (4,−1, 1)

This is the basic solution. General solution: t (4,−1, 1)

t (4v1 − v2 + v3) = 4tv1 − tv2 + tv3 = t0 = 0


ProofTheoremThe vectors v1, . . . , vk are linearly independent iff the vector equation

a1v1 + a2v2 + · · ·+ akvk = 0

has only the trivial solution.

Proof.We need to prove two directions.

“if part”

If a1v1 + a2v2 + · · ·+ akvk = 0 has only the trivial solution, thenv1, . . . , vk are linearly independent.

By contradiction. Suppose the statement is false, i.e., v1, . . . , vk are linearlydependent. WLOG (=without loss of generality), we let vj be redundant.This means, there are a1, . . . , aj−1 such thata1v1 + a2v2 + · · ·+ aj−1vj−1 = vj =⇒ a1v1 + a2v2 + · · ·+ aj−1vj−1 − vj = 0.

(a1, . . . , aj−1,−1, 0, . . . , 0) is a non-trivial solution.

This is a contradiction. Therefore, the statement is true, i.e., v1, . . . , vk arelinearly independent.


ProofTheoremThe vectors v1, . . . , vk are linearly independent iff the vector equation

a1v1 + a2v2 + · · ·+ akvk = 0

has only the trivial solution.

Proof (Cont’d).

“only if part”

If v1, . . . , vk are linearly independent, then a1v1 + a2v2 +· · ·+ akvk = 0 only has the trivial solution.

Again, to prove it by contradiction, we assumea1v1 + a2v2 + · · ·+ akvk = 0

has a non-trivial solution, say (a1, . . . , ak) 6= (0, . . . , 0). Let j be the largestindex such that aj 6= 0 , i.e.,

(a1, . . . , ak) = (a1, . . . , aj , 0, . . . , 0) =⇒ a1v1 + a2v2 + · · ·+ ajvj = 0

Thus, vj = −a1

ajv1 −

a2

ajv2 − · · · −

a1

ajvj−1.

This contradicts that v1, . . . , vk are linearly independent.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 24 / 34

Properties of linear independence

1. ReorderingIf v1, . . . , vk are linearly independent, then any reordering of this sequence ofvectors is also linearly independent.

ExampleIf v1, v2, v3 are linearly independent, then v3, v1, v2 are linearlyindependent.

2.SubsetIf v1, . . . , vk are linearly independent, so are v1, . . . , vj for any j < k .

RemarkThis seems to work only for the first j(< k) vector. Combiningwith the “Reordering” property, we see any j vectors chosen fromv1, . . . , vk are linearly independent.


Properties of linear independence

3. DimensionLet v1, . . . , vk be linearly independent in Rn. Then, k ≤ n.

Proof.The casting-out algorithm involves an n × k matrix. If k > n,[

v1 · · · vn vn+1 · · · vk]

i.e., there are more columns than rows, there must be at least onenon-pivot column. This implies v1, . . . , vk must be linearlydependent if k > n.

RemarkThere will be a general version of this property for subspaces.


Example

Assume v1, v2, and v3 are linearly independent in Rn.[Question] Determine whether the following vectors are linearly independent:

w1 = v1 + v2, w2 = v2 + 2v3, w3 = 2v1 + v2 − v3

We use the alternative characterization of linear independence.Consider the vector equation

a1w1 + a2w2 + a3w3 = 0

and ask whether this equation has a non-trivial solution.If “yes”, then w1, w2 and w3 are linearly dependent.If “no”, then w1, w2 and w3 are linearly independent.a1w1 + a2w2 + a3w3 = a1 (v1 + v2) + a2 (v2 + 2v3) + a3 (2v1 + v2 − v3)

Simplify(a1 + 2a3) v1 + (a1 + a2 + a3) v2 + (2a2 − a3) v3 = 0

Because v1, v2 and v3 are linearly independent, we know that

b1v1 + b2v2 + b3v3 = 0 =⇒ (b1, b2, b3) = (0, 0, 0) =⇒a1 + 2a3 = 0

a1 + a2 + a3 = 02a2 − a3 = 0


Example

a1w1 + a2w2 + a3w3 = 0⇔a1 + 2a3 = 0

a1 + a2 + a3 = 02a2 − a3 = 0 1 0 2

1 1 10 2 −1

∼ 1 0 2

0 1 −10 2 −1

∼ 1 0 2

0 1 −10 0 1

This system only has the trivial solution (a1, a2, a3) = (0, 0, 0).w1,w2 and w3 are linearly independent.

Remark (Linear transformation)

[w1 w2 w3

]=

1 0 21 1 10 2 −1

[v1 v2 v3]

After Midterm II, we shall learn an alternative method.


Uniqueness of linear combinations

TheoremAssume v1, . . . , vk are linearly independent and w ∈ span {v1, . . . , vk}. Then, wcan be written as a linear combination of v1, . . . , vk in a unique way.

Proof.Assume there are two ways:

w = b1v1 + b2v2 + · · ·+ bkvkw = c1v1 + c2v2 + · · ·+ ckvk

Subtraction yields

0 = (b1 − c1) v1 + (b2 − c2) v2 + · · ·+ (bk − ck) vk

Since v1, . . . , vk are linearly independent, the only solution to the equation is

b1 − c1 = b2 − c2 = · · · = bk − ck = 0⇐⇒ b1 = c1, b2 = c2, . . . , bk = ck .

They are the same linear combination.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 28th, 2019 29 / 34

Subspaces in Rn

DefinitionA subset V of Rn is called a subspace of Rn if the following 3 conditions hold:

0 ∈ V ;“V is closed under addition” : Whenever u,w ∈ V , then u + w ∈ V .“V is closed under scalar multiplication” : Whenever u ∈ V and k ∈ R, thenku ∈ V .

ExampleLet V be a plane through the origin in 3-dimensional space. Then, V is asubspace of R3.


Examples

Let W be a line through the origin. Then, it is a subspace.

Let A be the closed unit disc. A ={[

x y]T ∈ R2 | x2 + y2 ≤ 1

}.

This is not a subspace:0 ∈ A

u =

[10

],w =

[01

]∈ A but u + w =

[11

]/∈ A

(12 + 12 = 2 > 1

);

2u =

[20

]/∈ A


Solutions to a homogeneous system

ExampleThe solution space of a system of homogeneous equations. Let A be an m × nmatrix and V be the set of solutions of the homogeneous system

Au = 0 i.e., V = {u ∈ Rn | Au = 0} .

Then, V is a subspace of Rn.1 0 ∈ V , ⇐= A0 = 0 the trivial solution;2 for any u,w ∈ V , =⇒ Au = 0 = Aw =⇒ A (u + w) = Au+Aw = 0+ 0 = 0

=⇒ u + w ∈ V ;3 for any u ∈ V , k ∈ R, =⇒ Au = 0 =⇒ A (ku) = k (Au) = k0 = 0

=⇒ ku ∈ V

RemarkThe solution space of a homogeneous system = span {basic solutions}


Span

ExampleLet v1, . . . , vk be vectors in Rn. Then, span {v1, . . . , vk} is a subspace.

1 0 ∈ span {v1, . . . , vk} ⇐= 0 = 0v1 + · · ·+ 0vk ;2 Addition: Assume u,w ∈ span {v1, . . . , vk}

u = a1v1 + · · ·+ akvkw = b1v1 + · · ·+ bkvk

=⇒ u + w = (a1 + b1) v1 + · · ·+ (ak + bk) vk

=⇒ u + w ∈ span {v1, . . . , vk}3 Scalar multiplication:

u ∈ span {v1, . . . , vk} =⇒ u = a1v1 + · · ·+ akvk

=⇒ ru = (ra1) v1 + · · ·+ (rak) vk ∈ span {v1, . . . , vk}


SpanTheoremEvery subspace V of Rn is equal to a span. Moreover, we can writeV = span {u1, . . . ,uk}, where u1, . . . ,uk are linearly independent.

Proof.If V = {0}, then V = span∅ and we are done.If V 6= {0}, then V contains some non-zero vector. Pick any non-zero vectoru1 ∈ V . If V = span{u1}, we are done.Otherwise, pick u2 in V but not in span{u1}. If V = span{u1,u2}, we aredone.Otherwise, we pick u3 ∈ V \span{u1,u2} (A\B = {x ∈ A | x /∈ B}) andcheck V

?= span{u1,u2,u3} etc.

Note that:1 After j steps, vectors u1, . . . ,uj are linearly independent.2 Since there are at most n linearly independent vectors in Rn , this process

must stop after finitely many steps.



Lin Jiu


May 30th, 2019


Recall

1 Linearly (in)dependent(extended) casting-out algorithmhomogeneous system

2 Span of a set of vectorsRedundant vectors can be eliminated.For example, a span of three vectors, may not be a space.

3 Subspaces of Rn

lines, planes containing the originthe solution space of homogeneous systemTheorem. Every subspace V of Rn is equal to a span. Moreover, we can writeV = span {u1, . . . , uk}, where u1, . . . , uk are linearly independent.


Bases

DefinitionLet V be a subspace of Rn. {u1, . . . ,uk} is a basis of V if the following twoconditions hold:

1 V = span {u1, . . . ,uk}2 u1, . . . ,uk are linearly independent.

RemarkThe space V is “exactly” the span of the basis. None of the vectors in a basis isredundant.

Example1. A basis of R3:

e1 =

100

e2 =

010

e3 =

001

is called the standard basis of R3. Check the right-hand system: i, j, k in Chpt. 2.


Examples

R3 = span {e1, e2, e3} ⇐= every vector v =

xyz

∈ R3: v = xe1 + ye2 + ze3

It is obvious that they are linearly independent. I3 has rank 3.

PropositionLet ei be the vector in Rn whose i-th component is 1 and all of whose othercomponents are 0. In other words, ei is the i-th column of the identity matrix.

e1 =

100...0

, e2 =

010...0

, e3 =

001...0

, . . . , en =

000...1

Then, {e1, . . . , en} is a basis for Rn. It is called the standard basis of Rn.

rank(In) = n.


Examples

For the standard basis of Rn:

e1 =

100...0

, e2 =

010...0

, e3 =

001...0

, . . . , en =

000...1

This is also an orthonormal basis (orthogonal+normal)

Orthogonal: For i 6= j , ei · ej = 0Normal: ‖ei‖ = 1

You will study more in MATH-2040 (§11)


Examples

2. A non-standard basis of R3: u1 =

111

u2 =

110

u3 =

100

xyz

= a1u1 + a2u2 + a3u3 =

1 1 11 1 01 0 0

a1a2a3

→ 1 1 1 x

1 1 0 y1 0 0 z

rank = 3⇒ consistent & unique solution

1 1 1 x1 1 0 y1 0 0 z

R2←R2−R1R3←R3−R1∼

1 1 1 x0 0 −1 y − x0 −1 −1 z − x

R2↔R3∼ 1 1 1 x

0 −1 −1 z − x0 0 −1 y − x

=⇒ R3 = span {u1,u2,u3}u1,u2,u3 are linearly independent : x = y = z = 0 1 1 1 0

1 1 0 01 0 0 0

R2←R2−R1R3←R3−R1∼

1 1 1 00 0 −1 00 −1 0 0

R2↔R3∼ 1 1 1 0

0 −1 0 00 0 −1 0

rank= 3 =⇒unique trivial solution =⇒linearly independent.


Examples

RemarkRecall Theorem 1.26 Consider a system of m equations in n variables, and assumethat the coefficient matrix has rank r . Assume further that the system isconsistent.

If r = n , then the system has a unique solution.If r < n , then the system has infinitely many solutions, with n − rparameters.

Now, suppose we do not assume the system is consistent, but let m = n = r .The system is consistent, since there are no zero rows for the coefficient matrix inits echelon form (m = n = r). Moreover, this system has a unique solution.

# of equations = # of variables = rank =⇒ unique solution

Example3. The columns of an invertible matrix. Let A be an invertible n × n matrix and{u1, . . . ,un} be the columns of A.

A =[u1 · · · un

]⇒[A I

]∼[I A−1 ] =⇒ rank(A) = n


Examples

Ax = v⇐⇒[

u1 · · · un v]has a unique solution.

v =[x1 · · · xn

]T ∈ Rn: v = a1u1 + · · ·+ anun =⇒ Rn = span {u1, . . . ,un}v = 0 =⇒ u1, . . . ,un are linearly independent.

The columns of an n × n invertible matrix form a basis of Rn.4. Find a basis for the space

V = span

120

,

240

,

101

,

442

,

341

.

It is obvious that the second vector is redundant. u2 = 2u1It suffices to cast out the redundant vectors.

The result will only contain linearly independent vectors.And this does not change the span (still V ).


Examples

4. Find a basis for the space V = span

120

,

240

,

101

,

442

,

341

.

Solution. Use the casting-out algorithm:1 2 1 4 32 4 0 4 40 0 1 2 1

R2←R2−2R1∼1 2 1 4 30 0 −2 −4 −20 0 1 2 1

R2←− 1

2R2∼1 2 1 4 30 0 1 2 10 0 1 2 1

R3←R3−R2∼

1 2 1 4 30 0 1 2 10 0 0 0 0

This means u2, u4 and u5 are redundant. Therefore,

V = span

120

,

240

,

101

,

442

,

341

= V = span

basis︷︸︸︷120

,

101

.


Examples

5. Find a basis of the solution space of the system of homogeneous equations

1 2 1 4 32 4 0 4 40 0 1 2 1

xyzwv

=

000

⇔ 1 2 1 4 3 0

2 4 0 4 4 00 0 1 2 1 0

Solution. 1 2 1 4 3 02 4 0 4 4 00 0 1 2 1 0

R2←R2−2R1∼ R2←− 12R2∼

1 2 1 4 3 00 0 1 2 1 00 0 1 2 1 0

R3←R3−R2∼ R1←R1−R2∼

1 2 0 2 2 00 0 1 2 1 00 0 0 0 0 0


Examples

1 2 1 4 32 4 0 4 40 0 1 2 1

xyzwv

=

000

∼ 1 2 0 2 2 0

0 0 1 2 1 00 0 0 0 0 0

x and z are pivot while y , w and v are free.

y = t

w = s

v = r

⇒x =

z =

−2y − 2w − 2v−2w − v

= −2t − 2s − 2r= −2s − r

=⇒

xyzwv

=

−2t − 2s − 2r

t−2s − r

sr

xyzwv

=

−2t − 2s − 2r

t−2s − r

sr

= t

−21000

+ s

−20−210

+ r

−20−101


Examples

The solution space is span

−21000

,

−20−210

,

−20−101

. The vectors are linearly

independent due to the 1’s in red. Basis:{[−2 1 0 0 0]T , [−2 0 − 2 1 0]T , [−2 0 − 1 0 1]T

}FactThe solution space of a homogeneous system is the span of basic solution(s).Basic solutions are linearly independent. (Parameters)

6. Consider the subspace V of R3, consisting of all vectors[x y z

]T withx = z . In symbols,

V =

xyz

∈ R3 x = z

.

Find a basis for V .Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 12 / 34

Examples

Solution. Solve the homogeneous system x − z = 0. V is the solution space ofthe equation x = z , the homogeneous system x − z = 0

[1 0 −1 0

]⇒

z = t

y = s

⇒ x = z = t⇒

xyz

= s

010

+ t

101

Basis:

010

,

101

V =

xyz

x = z

= span

010

,

101


Coordinate

DefinitionLet B = {u1, . . . ,uk} be a basis of some subspace V . Then, every vector w ∈ Vcan be uniquely written as a linear combination of u1, . . . ,uk :

w = a1u1 + · · ·+ akuk .

We say a1, . . . , ak are the coordinates of w with respect to the basis B. We shalluse the notation

[w]B =[a1 a2 · · · ak

]T.

Example1. Let V = R3.

With respect to the standard basis e1 =

100

, e2 =

010

, e3 =

001

every vector v =

xyz

= xe1 + ye2 + ze3 ⇒

xyz

are the coordinates.


Examples

Recall a non-standard basis of R3: B =

u1 =

111

,u2 =

110

,u3 =

100

1 1 1 x1 1 0 y1 0 0 z

∼ 1 1 1 x

0 −1 −1 z − x0 0 −1 y − x

∼ 1 0 0 z

0 1 0 y − z0 0 1 x − y

Every vector v =

xyz

= zu1 + (y − z)u2 + (x − y)u3

⇒ [v]B =

zy − zx − y

= A

xyz

=⇒ A =

0 0 10 1 −11 −1 0

Exercise:

A−1 =

1 1 11 1 01 0 0

=[u1 u2 u3

]xyz

= xe1 + ye2 + ze3 = v = zu1 +(y − z)u2 +(x − y)u3 =

1 1 11 1 01 0 0

zy − zx − y


Example

2. Let B = {u1,u2,u3} =

1010

,

1120

,

0011

be a basis for a certain

3-dimensional subspace of R4. Let w =[3 2 6 1

]T . Find the coordinates ofw with respect to (w. r. t. ) B, i.e, [w]B .Solution. We need to solve a1u1 + a2u2 + a2u3 = w.

1 1 0 30 1 0 21 2 1 60 0 1 1

R3←R3−R1∼

1 1 0 30 1 0 20 1 1 30 0 1 1

R3←R3−R2∼

1 1 0 30 1 0 20 0 1 10 0 1 1

R4←R4−R3∼

1 1 0 30 1 0 20 0 1 10 0 0 0

R1←R1−R2∼

1 0 0 10 1 0 20 0 1 10 0 0 0

[w]B =

a1a2a3

=

121

⇒ w = u1 + 2u2 + u3.


Example

3. Let B = {u1,u2,u3} =

1010

,

1120

,

0011

.

Find the vector v whose coordinates w. r. t. B are[2 −1 1

]T .Solution.

v = 2u1+(−1)u2+1u3 = 2

1010

+(−1)

1120

+1

0011

=

2020

+−1−1−20

+0011

=

1−111

Factvector⇔coordinates One direction is easier than the other, which is due to that“vector⇒coordinates” needs to calculate the (left) inverse.


Classwork

Let

S =

xyzw

x + y = z + w

x + z = y + w

.

(a) Find a basis B for S .

(b) Obviously, v = [3 3 3 3]T ∈ S . Find [v]B



Solution

S =

xyzw

x + y = z + w

x + z = y + w

⇒x + y − z − w = 0

x − y + z − w = 0⇒[1 1 −1 −11 −1 1 −1

]xyzw

=

[00

]

(a) [1 1 −1 −1 01 −1 1 −1 0

]R2←R2−R1∼

[1 1 −1 −1 00 −2 2 0 0

]R2←− 1

2R2∼[

1 1 −1 −1 00 1 −1 0 0

]R1←R1−R2∼

[1 0 0 −1 00 1 −1 0 0

]

z = t

w = s⇒

xyzw

= t

0110

+ s

1001


Solution

S =

xyzw

x + y = z + w

x + z = y + w

= span

0110

,

1001

⇒ B =

0110

,

1001

(b) Easily by observation, [v]B =

[33

]. Or, we can try to solve

3333

= v = a

0110

+ b

1001

=

0 11 01 00 1

[ab]

0 1 31 0 31 0 30 1 3

∼

1 0 30 1 31 0 30 1 3

∼

1 0 30 1 30 0 00 0 0


Properties of Bases

TheoremSuppose u1, . . . ,ur are linearly independent elements of span {v1, . . . , vs}. Then,r ≤ s.

Proof.For j = 1, . . . , r , since uj ∈ span {v1, . . . , vs}, uj = a1jv1 + a2jv2 + · · ·+ asjvs .Let

A =

a11 · · · a1r...

. . ....

as1 · · · asr

an s × r matrix.

Assume r > s. The system Ax = 0 has a non-trivial solution x 6= 0 for the systema11 · · · a1r...

. . ....

as1 · · · asr

x1...xr

=

0...0

for all j = 1, . . . , s

x1aj1 + · · ·+ xrajr = 0Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 21 / 34

Properties of Basesx1u1 + · · ·+ xrur = x1 (a11v1 + a21v2 + · · ·+ as1vs) + · · ·

+ xr (a1rv1 + a2rv2 + · · ·+ asrvs)= (x1a11 + · · ·+ xra1r ) v1 + · · ·

+ (x1ar1 + · · ·+ xrarr ) vr= 0v1 + · · ·+ 0vr = 0

We have a non-trivial linear combination of {u1, . . .ur} giving the zero vector 0.This violates that u1, . . . ,ur are linearlly independent ⇒ r ≤ s.

TheoremLet V be a subspace of Rn. Then, any two bases of V have the same numberof elements.

Proof.Let {u1, . . . ,us} and {v1, . . . , vr} be two bases of V .

V = span {u1, . . . ,us} and {v1, . . . , vr} are linearly independent. =⇒ r ≤ s.V = span {v1, . . . , vr} and {u1, . . . ,us} are linearly independent. =⇒ s ≤ r .

Therefore, r = s.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 22 / 34

DimensionDefinitionLet V be a subspace of Rn with basis v1, . . . , vk . Then we say V has dimensionk . dimV = k .

Properties1 Every subspace V of Rn has a basis and therefore a dimension.2 Every linearly independent set of vectors u1, . . . ,ur ∈ V can be extended to

a basis.3 Every spanning set v1, . . . , vr of V can be shrunk to a basis of V .4 If V and W are subspaces of Rn with V ⊆W (V is a subset of W ), then

dimV ≤dimW .5 Let V be a k-dimensional space. Then,

every linearly independent set has ≤ k vectors;every spanning set has ≥ k vectors.

6 Let u1, . . . ,uk be vectors in a k-dimensional space V .

V = span {u1, . . . ,uk} if and only if u1, . . . ,uk are linearly independent.

Namely, u1, . . . ,uk form a basis iff they are linearly independent.


Proofs1 Every subspace V of Rn has a basis and therefore a dimension. (See the

slides on Tuesday that V = span {u1, . . . ,uk})2 Every linearly independent set of vectors u1, . . . ,ur ∈ V can be extended to

a basis.Proof. By (1), V has a basis v1, . . . , vs . Apply the casting-out algorithm tou1, . . . ,ur , v1, . . . , vs .

The output form a basis forspan {u1, . . . , ur , v1, . . . , vs} = span {v1, . . . , vs} = V ;u1, . . . , ur are linearly independent, so they “survived”.

3 Every spanning set v1, . . . , vr of V can be shrunk to a basis of V .Proof. Casting-out algorithm removes redundant vectors.

4 If V and W are subspaces of Rn with V ⊆W then dimV ≤dimW .Proof. Let v1, . . . , vk be a basis of V . (2) allows us to extend it to a basis ofW . Thus, dimV ≤dimW .

5 Let V be a k-dimensional space. Then,every linearly independent set has ≤ k vectors;every spanning set has ≥ k vectors.Proof. Apply the first property of bases: Suppose u1, . . . , ur are linearly

independent elements of span {v1, . . . , vk}. Then, r ≤ k.

If V = span{u1, . . . , ur} with r < k, then dimV ≤ r < k, which is acontradiction.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I May 30th, 2019 24 / 34

Proofs

6. Let u1, . . . ,uk be vectors in a k-dimensional space V .

V = span {u1, . . . ,uk} if and only if u1, . . . ,uk are linearly independent.

Namely, u1, . . . ,uk form a basis iff they are linearly independent.

Proof.We show both directions.

If u1, . . . ,uk are linearly independent, by (2) we extend it to a basis of V .dimV = k =# of elements in the basis, {u1, . . . ,uk} is A basis.V = span {u1, . . . ,uk}.If V = span {u1, . . . ,uk}, (3) allows us to shrunk to a basis of V , containingk elements. . No one is redundant. u1, . . . ,uk are linearly independent.


Example

Extend {u1,u2} to a basis of R4, where u1 =

11−12

and u2 =

12−24

.Solution. Recall the standard basis of R4 that {e1, e2, e3, e4}. We apply thecasing-out algorithm to {u1,u2, e1, e2, e3, e4}

1 1 1 0 0 01 2 0 1 0 0−1 −2 0 0 1 02 4 0 0 0 1

R2←R2−R1R3←R3+R1R4←R4−2R1∼

1 1 1 0 0 00 1 −1 1 0 00 −1 1 0 1 00 2 −2 0 0 1

R3←R3+R2R4←R4−2R2∼

1 1 1 0 0 00 1 −1 1 0 00 0 0 1 1 00 0 0 −2 0 1

R4←R4+2R3∼

1 1 1 0 0 00 1 −1 1 0 00 0 0 1 1 00 0 0 0 2 1

Basis: {u1,u2, e2, e3}


Example

Let V =

xyzw

x + 2y + z − w = 0

. Note u1 =

1−110

,u2 =

−2111

∈ V .

Extend {u1,u2} to a basis of V .Solution. First of all, solve for V . [1 2 1 − 1 | 0] y = t, z = s, w = rx = −2t − s + r

xyzw

= t

−2100

︸︷︷︸

v1

+ s

−1010

︸︷︷︸

v2

+ r

1001

︸︷︷︸

v3

Casting-out algorithm is applied to {u1,u2, v1, v2, v3}1 −2 −2 −1 1−1 1 1 0 01 1 0 1 00 1 0 0 1

∼

1 0 0 1 −10 1 0 0 10 0 1 1 −20 0 0 0 0

Basis: {u1,u2, v1}


Column, row and null spaces of a matrix

Definition

Let A =

a11 · · · a1n...

. . ....

am1 · · · amn

be an m × n-matrix.

The column space of A is the subspace of Rm spanned by the columns of A.

col(A) = span

a11

...am1

,

a12...

am2

, . . . ,

a1n...

amn

.

The row space of A is the subspace of Rn spanned by the rows of A.

row(A) = span{[

a11 · · · a1n]T

,[a21 · · · a2n

]T, · · · ,

[am1 · · · amn

]T}.

The null space of A is the subspace of Rn given by the solutions to thehomogeneous system Av = 0. null(A) = {v ∈ Rn | Av = 0} .


Column, row and null spaces of a matrix

TheoremIf A is an m × n matrix of rank r , then, dim(col(A)) = dim(row(A)) = r , anddim(null(A)) = n − r .

DefinitionThe dimension of the null space is also called the nullity of A, i.e.,nullity(A) = dim(null(A)).

Theorem (Rank-Nullity Theorem)For an m × n matrix A: rank(A) + nullity(A) = n.

ExampleFind a basis and the dimension of each of (a) the column space, (b) the row space(c) and the null space of the matrix

A =

1 2 3 2 33 2 5 4 75 0 5 5 10


ExampleFind the reduced echelon form:1 2 3 2 3

3 2 5 4 75 0 5 5 10

R2←R2−2R1R3←R3−5R1∼

1 2 3 2 30 −4 −4 −2 −20 −10 −10 −5 −5

R2←− 1

4 R2R3←− 1

10 R3∼1 2 3 2 30 1 1 1

212

0 1 1 12

12

R3←R3−R2R1←R1−2R2∼

1 0 1 1 20 1 1 1

212

0 0 0 0 0

1 Column space: pivot vectors are the first two columns.

Basis :

135

,

220

dim(col(A)) = rank(A) = 2.

2 Row space: pivot vectors are the first two rows.Basis :

{[1 2 3 2 3

]T,[3 2 5 4 7

]T}dim(row(A)) = 2.


Example

3 Null space: Av = 0 1 2 3 2 3 03 2 5 4 7 05 0 5 5 10 0

∼ 1 0 1 1 2 0

0 1 1 12

12 0

0 0 0 0 0 0

x3 = s

x4 = t

x5 = r

=⇒x1 = −x3 − x4 − 2x5

x2 = −x3 −12x4 −

12x5

= −s − t − 2r

= −s − 12t − 1

2r

x1x2x3x4x5

= s

−1−1100

+ t

−1− 1

2010

+ r

−2− 1

2001

Basis:

−1−1100

,

−1− 1

2010

,

−2− 1

2001

dim(null(A)) = 3.


RemarkBe careful when interchanging rows.

A =

1 1 11 1 01 1 01 0 0

∼1 1 10 0 −10 0 −10 −1 −1

∼1 1 10 −1 −10 0 −10 0 −1

∼1 1 10 −1 −10 0 −10 0 0

Basis for the row space: {R1,R3,R4} .

Midterm IIThere will be 20 problems.The True/False problems will NOT require explanations.True/False, Multiple choice, Short answer

RemarkPlease be aware of that: in the final exam, short answer problems will be replacedby detailed answer problems. Namely, to obtain full credits, you need to show allthe steps, i.e., only providing the final answer is NOT sufficient.


Summary of Chpt. 5

span linearly (in)dependent vectors basis

1 span: set of linear combinations span {u1, . . . ,u`}Natural question: Do we need all the u1, . . . ,u` ?To answer: casting-out algorithm By casting out the redundant vectors, theremaining vectors are linearly independent.Every subspace is a span of a set of vectors.

2 linearly (in-)dependent vectors: Besides (extended) casting-out algorithm, wehave another criterion to determine whether a set of vectors. Thecorresponding homogeneous system:

a1u1 + · · ·+ a`u` = 0.

check whether it ONLY has the trivial solution.3 basis: a set of linearly independent vectors, whose span gives the subspace

The number of vectors in a basis is fixed. dimension of a subspaceShrink and extend to obtain a basis.Bases for the row, column and null spaces of a matrix


Summary of Chpt. 5

There are lots of examples in this chapter. The key is the elementary rowoperation.

Read the slides/textbook carefully, by focusing on the examples.Make sure, for different type of problems, you understand the steps &reasons.



Lin Jiu


June 4th, 2019

Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 1 / 35

Vector functions

Example (Function y = f (x))

f (x) = x2 =⇒ f (0) = 0, f (1) = 1, f (2) = 4, . . .

Example (Vector function w = f (v), with both input and outputvectors)

f

xyz

=

[x + zy2 − xz

]=⇒ f

100

=

[1+ 0

02 − 1 · 0

]=

[10

]

f

010

=

[0+ 0

12 − 0 · 0

]=

[01

]

f

111

=

[1+ 1

12 − 1 · 1

]=

[20

]Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 2 / 35

Examples

RemarkIf f inputs an n-dimensional vector and outputs an m-dimensional vector, we write

f : Rn −→ Rm.

Example(a) The example above is f : R3 −→ R2.

f

xyz

=

[x + zy2 − xz

]

(b)g

([xy

])=

x + yx − y2y

is an example g : R2 −→ R3


Linear Transformation

Recall that at the very beginning of this class, we quickly introduced linearequations and non-linear equations. We shall also classify the vector functions.

DefinitionA vector function T : Rn −→ Rm is called linear or a linear transformation if

1 T (v + w) = T (v) + T (w);2 T (kv) = kT (v) .

Namely, T is linear iff T preserves the addition and scalar multiplication of thevectors. Note that the addition and scalar multiplication on both sides of the twoidentities are different, since the vectors on both sides usually do not have thesame dimension. v,w ∈ Rn T (v) ,T (w) ∈ Rm

ExampleWhich of the following vector functions are linear?

(a) T1

xyz

=

[x + 2zy − z

], (b) T2

xyz

=

[2

x − z

], (c) T3

xyz

=

[x2

yz


Solution

(c) T3

xyz

=

[x2

yz

]is obviously not linear since it involves x2 and yz .

T3

100

=

[12

0 · 0

]=

[10

]; T3

200

=

[40

].

200

= 2

100

However, 2T3

100

= 2[10

]=

[20

]6=[40

]= T3

2

100

.

It violates the second condition. Namely, the scalar multiplication is notpreserved.

(a) T1

xyz

=

[x + 2zy − z

]is linear. Let v =

xyz

,w =

x ′y ′z ′

∈ R3 and k ∈ R.

We need to checkT1 (v + w)

???= T1 (v) + T1 (w)

T1 (kv) ???= kT1 (v)


Example

(a) T1

xyz

=

[x + 2zy − z

], v =

xyz

, w =

x ′y ′z ′

Check T1 (v + w)

???= T1 (v) + T1 (w);

T1 (v) = T1

xyz

=

[x + 2zy − z

]T1 (w) =

[x ′ + 2z ′

y ′ − z ′

]

=⇒ T1 (v) + T1 (w) =

[x + x ′ + 2z + 2z ′

y ′ + y ′ − z − z ′

]On the other hand,

v+w =

x + x ′

y + y ′

z + z ′

⇒ T1 (v + w) =

[x + x ′ + 2 (z + z ′)y + y ′ − (z + z ′)

]=

[x + x ′ + 2z + 2z ′

y ′ + y ′ − z − z ′

]

=⇒ T1 (v) + T1 (w) = T1 (v) + T1 (w)


Example

(a) T1

xyz

=

[x + 2zy − z

], v =

xyz

, w =

x ′y ′z ′

Check T1 (kv)

???= kT1 (v);

T1 (v) =[x + 2zy − z

]kT1 (v) =

[k (x + 2z)k (y − z)

]=

[kx + 2kzky − kz

]Meanwhile,

kv =

kxkykz

=⇒ T1 (kv) =[kx + 2kzky − kz

]= kT1 (v)

RemarkIt is not surprising that both components of T1 (x + 2z , y − z) are linear in x , y ,and z . If a vector function is linear, all its components are linear functions.


Example

(b) T2

xyz

=

[2

x − z

]Apparently, both components of T2 are linear functions. However, T2 is notlinear, due to the constant 2.

T2

100

=

[2

1− 0

]=

[21

], T2

200

=

[22

]

⇒ 2T

100

6= T2

2

100

In general, due to this 2, T2 (v) =

[2∗

]and T2 (kv) =

[2∗∗

]so that

T2 (v) 6= T2 (kv).

RemarkFor a vector function, even if all the components are linear functions, this doesguarantee the function to be linear. (KEY: Without constant terms)

Proposition (Alternative characterization, i.e., T is linear iff)T (av+ bw) = aT (v) + bT (w). (a = b = 1 addition, b = 0 scalar multiplication )


Linear Transformation

LemmaEvery linear transformation T satisfies T (0) = 0.

Example

Recall T2

xyz

=

[2

x − z

]. T2 (0) = T2

000

=

[20

]6= 0.

Proof.Let v be any vector in Rn. Then, T (v) ∈ Rm

0 = 0 · T (v)(2)= T (0 · v) = T (0)

Remark

This is only necessary,but not sufficient.

Recall T3

xyz

=

[x2

yz

]⇒ T3

000

=

[00

] butNOTlinear.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 9 / 35

Matrix transformation

TheoremLet A be an m × n matrix. The function

T : Rn → Rm

v 7→ Av (T (v) = Av)

is always linear. Such a function is called a matrix transformation.

Proof.Just check the two conditions.

1 T (v + w) = A (v + w) = Av + Aw = T (v) + T (w)

2 T (kv) = A (kv) = kAv = kT (v)

RemarkThe dimensions are compatible.

v ∈ Rn — an n × 1 matrix;Am×n —Av is well defined and has dimension m × 1 — a vector in Rm


ExampleRecall the previous example

T1

xyz

=

[x + 2zy − z

]

It is actually a matrix transformation. T1 : R3 −→ R2

T1

xyz

=

[x + 2zy − z

]2×1

=

[]2×3

xyz

3×1

=

[1 0 20 1 −1

]xyz

PropositionEvery linear transformation is a matrix transformation. (Proof comes later)

RemarkGiven a vector function, if all the components are linear without the constantterm, we can write it as a matrix transformation. Therefore, it is linear.


Example

Let T

xyzw

=

x − 2w + zx − y2y + z

. Is T linear? If yes, write it as a matrix

transformation.

Solution. All the three components are linear without the constant term. So, Tis linear.

Since T : R4 −→ R3, we should find a 3× 4 matrix A such that T (v) = Av.x − 2w + zx − y2y + z

=

xx0

+

0−y2y

+

z0z

+

−2w00

= x

110

+ y

0−12

+ z

101

+w

−200

=

1 0 1 −21 −1 0 00 2 1 0

xyzw


Linear combination

RemarkIf T is a linear transformation, then T preserves not only addition and scalarmultiplication, but all linear combinations:

T (a1v1 + a2v2 + · · ·+ akvk)(1)= T (a1v1) + T (a2v2) + · · ·+ T (akvk)(2)= a1T (v1) + a2T (v2) + · · ·+ akT (vk)

Example

Assume T (u1) =

[10

], T (u2) =

[02

], and T (u3) =

[11

]. Find T (4u1 + 2u2 − u3).

Solution.T (4u1 + 2u2 − u3) = 4T (u1) + 2T (u2)− T (u3)

= 4[10

]+ 2

[02

]−[11

]=

[40

]+

[04

]−[11

]=

[33


Example

Let T : R3 −→ R2 be linear. If

T

121

=

[01

], T

133

=

[20

], T

411

=

[22

],

find T

314

.

Solution. For simplicity, we let u1 =

121

, u2 =

133

, u3 =

411

, andv =

314

. We first need to express v as a linear combination of u1, u2, and u3.


Example

u1 =

121

, u2 =

133

, u3 =

411

, v =

314

1 1 4 3

2 3 1 11 3 1 4

R2←R2−2R1∼

1 1 4 30 1 −7 −51 3 1 4

R3←R3−R1∼

1 1 4 30 1 −7 −50 2 −3 1

R3←R3−2R2∼

1 1 4 30 1 −7 −50 0 11 11

R3← 111R3∼

1 1 4 30 1 −7 −50 0 1 1

R2←R2+7R3∼

1 1 4 30 1 0 20 0 1 1

R1←R1−4R3∼

1 1 0 −10 1 0 20 0 1 1

R1←R1−R2∼

1 0 0 −30 1 0 20 0 1 1

⇒ v = −3u1 + 2u2 + u3


Example

v = −3u1 + 2u2 + u3

T (u1) =

[01

], T (u2) =

[20

], T (u3) =

[22

]

T (v) = T (−3u1 + 2u2 + u3)

= −3T (u1) + 2T (u2) + T (u3)

= −3[01

]+ 2

[20

]+

[22

]=

[0−3

]+

[40

]+

[22

]=

[6−1

]What will happen if v is NOT a linear combination of u1, u2, and u3?[Answer]: Then, we do not know T (v).


Classwork

Let

v1 =

220

, v2 =

101

, v3 =

133

,u =

317

and a linear transformation T satisfy

T (v1) =

[11

], T (v2) =

[02

], T (v3) =

[22

].

Find T (u).

Lecture resumes at 7:45.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 4th, 2019 17 / 35

Solution

v1 =

220

, v2 =

101

, v3 =

133

, u =

317

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

First of all, we try to write u as a linear combination of v1, v2, v3. Namely, finda1, a2, a3 such that u = a1v1 + a2v2 + a3v3.31

7

= a1

220

+ a2

101

+ a3

133

=

2 1 12 0 30 1 3

a1a2a3

2 1 1 3

2 0 3 10 1 3 7

R2←R2−R1∼

2 1 1 30 −1 2 −20 1 3 7

R3←R3+R2∼

2 1 1 30 −1 2 −20 0 5 5

R3← 1

5R3∼

2 1 1 30 −1 2 −20 0 1 1

R2←R2−2R3∼

2 1 1 30 −1 0 −40 0 1 1

R2←−R2∼

2 1 1 30 1 0 40 0 1 1

R1←R1−R2∼

2 0 1 −10 1 0 40 0 1 1

R1←R1−R3∼

2 0 0 −20 1 0 40 0 1 1

R1← 12R1∼

1 0 0 −10 1 0 40 0 1 1


Solution

v1 =

220

, v2 =

101

, v3 =

133

, u =

317

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

u = a1v1 + a2v2 + a3v3 →

2 1 1 32 0 3 10 1 3 7

∼ 1 0 0 −1

0 1 0 40 0 1 1

⇒a1

a2

a3

=

−141

u = −v1 + 4v2 + v3 ⇒ T (u) = T (−v1 + 4v2 + v3)

= −T (v1) + 4T (v2) + T (v3)

= −[11

]+ 4

[02

]+

[22

]=

[19

]Next, we study several properties.


Basis

TheoremA linear transformation T is completely determined by its action on a basis.

ExampleAn unknown linear transformation T : R3 −→ R2 satisfies

T

100

=

[23

], T

010

=

[12

], T

001

=

[−11

].

Find a formula for T . (formula=matrix transformation form)Solution.

T

xyz

= T

x

100

+ y

010

+ z

001

= xT

100

+ yT

010

+ zT

001

= x

[23

]+ y

[12

]+ z

[−11

]=

[2 1 −13 2 1

] xyz

Let A =

[2 1 −13 2 1

]. Then, T (v) = Av.


Matrix transformationTheoremEvery linear transformation is a matrix transformation.

Proof.Suppose T : Rn −→ Rm is a linear transformation. Recall the standard basis ofRn , i.e, {e1, . . . , en}. Now, denote

w1 = T (e1) , w2 = T (e2) , . . . ,wn = T (en) .

Form A =[w1 w2 · · · wn

], i.e., A is the matrix with w1, . . . ,wn being its

columns and dimension m × n (wk ∈ Rm). We shall show that T (v) = Av.

For all v ∈ Rn, v =

x1...xn

⇒ T (v) = T (x1e1 + · · ·+ xnen) = x1w1 + · · ·+ xnwn

This implies

T (v) =[w1 w2 · · · wn

] x1...xn

= Av.


ExampleRevisit the class work

v1 =

220

, v2 =

101

, v3 =

133

, u =

317

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

1 Is {v1, v2, v3} a basis for R3? Check whether they are linearly independent: 2 1 12 0 30 1 3

∼∼∼ 1 0 0

0 1 00 0 1

2 1 1 32 0 3 10 1 3 7

∼

1 0 0 −10 1 0 40 0 1 1

They are linearly independent and form a basis for R3. dimR3 = 3.

2 Can we find a formula for T? To apply the same method, we need to knowT (e1) ,T (e2) ,T (e3). Since {v1, v2, v3} is a basis, we can express e1, e2, e3as linear combinations of v1, v2, v3. Find a, b, c such that

e1 = av1 + bv2 + cv3 ⇔

2 1 12 0 30 1 3

abc

=

100

⇔ 2 1 1 1

2 0 3 00 1 3 0


ExampleSimilarly, find d , e, f such that

e2 = dv1 + ev2 + f v3 ⇔

2 1 12 0 30 1 3

def

=

010

⇔ 2 1 1 0

2 0 3 10 1 3 0

and g , h, i such that

e3 = gv1 + hv2 + iv3 ⇔

2 1 12 0 30 1 3

ghi

=

001

⇔ 2 1 1 0

2 0 3 00 1 3 1

Observe that (or, trying to solve three systems together) 2 1 1

2 0 30 1 3

a d gb e hc f i

=[e1 e2 e3

]= I3

Find 2 1 12 0 30 1 3

−1


Example 2 1 1 1 0 02 0 3 0 1 00 1 3 0 0 1

R2←R2−R1∼

2 1 1 1 0 00 −1 2 −1 1 00 1 3 0 0 1

R3←R3+R2∼

2 1 1 1 0 00 −1 2 −1 1 00 0 5 −1 1 1

R3←

R35∼

2 1 1 1 0 00 −1 2 −1 1 00 0 1 − 1

515

15

R2←R2−2R3R1←R1−R3∼

2 1 1 1 0 00 −1 0 − 3

535 − 2

50 0 1 − 1

515

15

R1←R1+R2−R3∼

2 0 0 35

25 − 3

50 −1 0 − 3

535 − 2

50 0 1 − 1

515

15

R2←−R2R1← 1

2 R1∼

1 0 0 310

15 − 3

100 1 0 3

5 − 35

25

0 0 1 − 15

15

15


Example 2 1 1

2 0 3

0 1 3

−1

=

310

15 − 3

1035 − 3

525

− 15

15

15

e1 =

310

v1 +35v2 −

15v3

e2 =15v1 −

35v2 +

15v3

e3 = − 310

v1 +25v2 +

15v3

=⇒

T (e1) =310

T (v1) +35T (v2)−

15T (v3)

T (e2) =15T (v1)−

35T (v2) +

15T (v3)

T (e3) = −310

T (v1) +25T (v2) +

15T (v3)

=⇒

T (e1) =310

[11

]+

35

[02

]− 1

5

[22

]T (e2) =

15

[11

]− 3

5

[02

]+

15

[22

]T (e3) = −

310

[11

]+

25

[02

]+

15

[22

]=

[− 1

101110

]=

[ 35− 3

5

]=

[ 110910

]


Example

v1 =

220

, v2 =

101

, v3 =

133

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

T (e1) =

[− 1

101110

], T (e2) =

[ 35− 3

5

], T (e3) =

[ 110910

]

T

xyz

= T (xe1 + ye2 + ze3) = x

[− 1

101110

]+ y

[ 35− 3

5

]+ z

[ 110910

]

S =

[− 1

1035

110

1110 − 3

5910

]⇒ T (v) = Sv


Example

v1 =

220

, v2 =

101

, v3 =

133

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

T (e1) =

[− 1

101110

], T (e2) =

[ 35− 3

5

], T (e3) =

[110910

], S =

[− 1

1035

110

1110 − 3

5910

]

Recall the steps A =

2 1 1

2 0 3

0 1 3

⇒ A−1 =

310

15 − 3

1035 − 3

525

− 15

15

15

T (e1) =

310

[11

]+

35

[02

]− 1

5

[22

]T (e2) =

15

[11

]− 3

5

[02

]+

15

[22

]T (e3) = −

310

[11

]+

25

[02

]+

15

[22

] ⇒S =

[1 0 21 2 2

]A−1

by the column method.


Theorem

Theorem (Exercise 6.2.2)Assume that T : Rn → Rm is a linear transformation, and that u1, . . . ,un is abasis of Rn. For all i = 1, . . . , n, let vi = T (ui ). Let A be the matrix that hasu1, . . . ,un as its columns, and let B be the matrix that has v1, . . . , vn as itscolumns. Show that A is invertible and the matrix of T is BA−1.

Example

u1 =

220

,u2 =

101

,u3 =

133

,T (u1) =

[11

],T (u2) =

[02

],T (u3) =

[22

]

A =

2 1 12 0 30 1 3

is invertible sinceu1,u2,u3 is a basis

⇒ linearly independent ⇒ rank(A) = 3

A−1 =

310

15 − 3

1035 − 3

525

− 15

15

15

, B =

[1 0 21 2 2

], S = BA−1 =

[− 1

1035

110

1110 − 3

5910

]


Composition

For functions, we can compose two functions: f (x) = x2, g (x) = x + 1

g ◦ f (x) = g (f (x)) = g(x2) = x2 + 1

Similarly, consider two linear transformations:T : Rm → Rn S : Rk → Rm

v 7→ Av w 7→ Bw

We can compose T and S and get that

T ◦ S : Rk → Rn

w 7→ T (S (w)) = T (Bw) = A (Bw) = (AB)w

So the matrix multiplication originally was to solve the question:How can we compose two linear transformations?


Inverse

DefinitionLet T ,S : Rn → Rn be linear transformations. Suppose that for each v ∈ Rn ,

(S ◦ T )(v) = v

and(T ◦ S)(v) = v.

Then S is called the inverse of T , and we write S = T−1.

Theorem (Theorem 6.22: Matrix of the inverse transformation)Let T : Rn → Rn be a linear transformation and let A be the correspondingn × n-matrix. Then T has an inverse if and only if the matrix A is invertible. Inthis case, the matrix of T−1 is A−1.


Example

v1 =

220

, v2 =

101

, v3 =

133

,T (v1) =

[11

],T (v2) =

[02

],T (v3) =

[22

]

S =

[− 1

1035

110

1110 − 3

5910

]= BA−1 =

[1 0 21 2 2

]2 1 12 0 30 1 3

−1

We define

T1 : R3 −→ R3, T2 : R3 −→ R2

e1 7→ v1 e1 7→[11

]e2 7→ v2 e2 7→

[02

]e3 7→ v3 e3 7→

[22

]

T1 (u) = Au T2 (w) = Bw.

v1, v2, v3 is another basis forR3, A is invertible (i.e., T1has an inverse map T−1

1with matrix transformation ofA−1)

T = T2 ◦ T−11 ⇔ S = BA−1


Geometric interpretation (Read Section 6.3 *)

1. A =

[0 −11 0

]rotation by 90◦.



2. A =

[−1 00 1

]reflection about the y -axis



3. A =

[cos θ − sin θsin θ cos θ

]counterclockwise rotation by θ.


Summary of Chpt. 6

Linear Transformations:1 definition: Trick: All components are linear without the constant term.2 property: T (0) = 0 preserves addition and scalar multiplication ⇔ preserves

linear combination3 linear transformations are all matrix transformations: T (v) = Av

Given certain values, e.g., T (v1), T (v2), T (v3), find T (u) foru ∈ span {v1, v2, v3}Given T (v1), T (v2), T (v3) on a basis, find the matrix of the transformation.

4 composition↔matrix multiplication5 inverse



Lin Jiu


June 6th, 2019


Determinant

GoalTo an n × n-matrix A, we will define a scalar det(A), called the determinant of A.In particular, we will use the determinant to determine some properties of a squarematrix.

invertible?rank?

Recall: An n × n matrix is invertible iff rank(A) = n.

ClaimA is invertible iff det(A) 6= 0.

We shall begin with the case n = 2.

Definition

For a 2× 2 matrix A =

[a bc d

], the determinant is defined by det(A) := ad − bc


ExamplesExample

A =

[2 10 3

]=⇒ det(A) = 2 · 3− 1 · 0 = 6

B =

[−3 70 3

]=⇒ det(B) = (−3) · 3− 7 · 0 = −9

C =

[3 64 8

]=⇒ det(C ) = 3 · 8− 6 · 4 = 24− 24 = 0.

By the elementary row operations, we see CR1← 1

3R1∼[1 24 8

]R2←R2−4R1∼

[1 20 0

]

RemarkThese three examples above shows that the determinant can be positive, negativeand zero. Also, both A and B are in echelon forms, so rank(A) = rank(B) = 2,while rank(C ) = 1 < 2. Please recall for dot product: u · v = 0⇔ u and v areorthogonal/perpendicular. Later on we shall see det(A) = 0⇔ rank(A) < n for ann × n matrix A.


Examples

RemarkThe determinant is also often denoted by enclosing the matrix with two verticallines. Thus

det

[a bc d

]=

∣∣∣∣ a bc d

∣∣∣∣ = ad − bc.

Example

A =

[2 10 3

]=⇒ det(A) =

∣∣∣∣ 2 10 3

∣∣∣∣ = 2 · 3− 1 · 0 = 6

B =

[−3 70 3

]=⇒ det(B) =

∣∣∣∣ −3 70 3

∣∣∣∣ = (−3) · 3− 7 · 0 = −9

C =

[3 64 8

]=⇒ det(C ) =

∣∣∣∣ 3 64 8

∣∣∣∣ = 3 · 8− 6 · 4 = 24− 24 = 0.


3× 3

Definition

For a 3× 3 matrix A =

a11 a12 a13a21 a22 a23a31 a32 a33

, the determinant is defined by

det(A) := a11a22a33 + a12a23a31 + a13a21a32−(a31a22a13 + a32a23a11 + a33a21a12)

Of course, it is almost impossible to memorize.

RemarkHere is a picture for help.

1 Copy the first two columns next to the matrix (right-hand side)2 The blue lines correspond to the positive terms of the determinant:

a11a22a33, a12a23a31, and a13a21a32.3 The red lines correspond to the negative terms: a31a22a13, a32a23a11, and

a33a21a12Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 5 / 13

Examples

A =

0 1 23 1 01 1 −1

0 1 2

3 1 01 1 −1

0 13 11 1

det (A) =

∣∣∣∣∣∣0 1 23 1 01 1 −1

∣∣∣∣∣∣= 0 · 1 · (−1) + 2 · 0 · 1+ 2 · 3 · 1− (2 · 1 · 1+ 0 · 0 · 1+ 1 · 3 · (−1))= 6− (−1)= 7


Cross product

u =

u1u2u3

, v =

v1v2v3

⇒ u× v =

u1u2u3

×v1v2v3

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1

.

Recall the right-hand side system i =

100

, j =

010

, k =

001

.Theorem

u× v =

∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k

∣∣∣∣∣∣ .Namely, u× v = det (A) for A =

u1 v1 iu2 v2 ju3 v3 k


Cross product

u1u2u3

×v1v2v3

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1

A =


∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k

∣∣∣∣∣∣u1 v1u2 v2u3 v3

det(A) = u1v2k + v1ju3 + iu2v3 − iv2u3 − u1jv3 − v1u2k= (u2v3 − v2u3) i + (v1u3 − u1v3) j + (u1v2 − v1u2) k

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1


Triangular matrices

Definition

A square matrix A =

a11 · · · a1n...

. . ....

an1 · · · ann

is upper triangular if aij = 0 whenever

i > j .In other words, a matrix is upper triangular if entries below the main diagonal are0.Similarly, a square matrix is lower triangular if all entries above the main diagonalare 0.Let ∗ refers to any non-zero numbers:

∗ ∗ · · · ∗ ∗0 ∗ · · · ∗ ∗...

.... . .

......

0 0 · · · ∗ ∗0 0 · · · 0 ∗

︸︷︷︸

upper triangular

∗ 0 · · · 0 0∗ ∗ · · · ∗ 0...

.... . .

......

∗ ∗ · · · ∗ 0∗ ∗ · · · ∗ ∗

︸︷︷︸

lower triangular


Triangular matrices

TheoremFor an n × n triangular matrix A, the determinant is equal to the product of theentries on the main diagonal. Namely,

det(A) = a11a22 · · · ann.

Example

A =

1 2 3 160 2 6 −70 0 3 330 0 0 −1

=⇒ det(A) =

∣∣∣∣∣∣∣∣1 2 3 160 2 6 −70 0 3 330 0 0 −1

∣∣∣∣∣∣∣∣ = 1 · 2 · 3 · (−1) = −6.

∣∣∣∣∣∣∣∣−1 0 0 06 4 0 04 1 −2 0

10000000 8000 9 2

∣∣∣∣∣∣∣∣ = (−1) · 4 · (−2) · 2 = 16


Row operations

RemarkBe careful that matrices of the following forms are not triangular matrices:

[0 13 4

] 2 4 03 0 05 0 0

Recall the three elementary row operations:

Ri ←→ Rj , Rj ←− aRj , Rj ←− Rj + aRi (i 6= j , a 6= 0)

TheoremLet A be an n × n-matrix.

1 If B is obtained from A by switching two rows, then det (B) = − det(A);2 If B is obtained from A by multiplying one row by a non-zero scalar k , then

det (B) = k det(A);3 If B is obtained from A by adding a multiple of one row to another row, then

det (B) = det(A);


Example A

Ri←→Rj∼ B ⇒ det (B) = − det(A)

ARj←−kRj∼ B ⇒ det (B) = k det(A)

ARj←−Rj+aRi∼ B ⇒ det (B) = det(A)

Let A =

2 4 −81 8 11 4 −2

, find det (A) .

1 We see 2 4 −81 8 11 4 −2

R1↔R2∼

1 8 12 4 −81 4 −2

R2←R2−2R1∼

1 8 10 −12 −101 4 −2

R3←R3−R1∼

1 8 10 −12 −100 −4 −3

R3←R3− 13R2∼

1 8 10 −12 −100 0 1

3

det (A) = −

∣∣∣∣∣∣1 8 12 4 −81 4 −2

∣∣∣∣∣∣ = −∣∣∣∣∣∣

1 8 10 −12 −101 4 −2

∣∣∣∣∣∣ = −∣∣∣∣∣∣

1 8 10 −12 −100 −4 −3

∣∣∣∣∣∣ = −∣∣∣∣∣∣

1 8 10 −12 −100 0 1

3

∣∣∣∣∣∣Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 6th, 2019 12 / 13

ExampleA




A =

2 4 −81 8 11 4 −2

det (A) = −

∣∣∣∣∣∣1 8 10 −12 −100 0 1

3

∣∣∣∣∣∣ = −1 · (−12) · 13 = 4

2. Alternatively, 2 4 −81 8 11 4 −2

R2← 12R2∼

1 2 −41 8 11 4 −2

R2←R2−R1∼

1 2 −40 6 51 4 −2

R3←R3−R1∼

1 2 −40 6 50 2 2

R3←R3− 13R2∼

1 2 −40 6 50 0 − 1

3

∣∣∣∣∣∣

1 2 −41 8 11 4 −2

∣∣∣∣∣∣ = 12 det(A). ⇒ det(A) = 2

∣∣∣∣∣∣1 2 −41 8 11 4 −2

∣∣∣∣∣∣ = · · · = 2

∣∣∣∣∣∣1 2 −40 6 50 0 − 1

3

∣∣∣∣∣∣ = 4



Lin Jiu


June 11th, 2019


Recall

The determinant can be considered as a function: det : {square matrices} −→ R.

det

([a bc d

])=

∣∣∣∣a bc d

∣∣∣∣ = ad − bc

det

a11 a12 a13a21 a22 a23a31 a32 a33

= a11a22a33 + a12a23a31 + a13a21a32

−(a31a22a13 + a32a23a11 + a33a21a12)

Triangular: If A is upper/lower triangular, det(A) = a11a22 · · · ann.

RemarkBe careful that matrices of the following forms are not triangular matrices:

[0 13 4

] 2 4 03 0 05 0 0

MAIN DIAGONAL


Row operations

Recall the three elementary row operations:

Ri ←→ Rj , Rj ←− aRj , Rj ←− Rj + aRi (i 6= j , a 6= 0)

TheoremLet A be an n × n-matrix.

1 If B is obtained from A by switching two rows, then det (B) = − det(A);2 If B is obtained from A by multiplying one row by a non-zero scalar k , then

det (B) = k det(A);3 If B is obtained from A by adding a multiple of one row to another row, then

det (B) = det(A);


Example

A




Let A =

2 4 −81 8 11 4 −2

, find det (A) .

1 We see 2 4 −81 8 11 4 −2

A

R1↔R2∼

1 8 12 4 −81 4 −2

A1

R2←R2−2R1∼

1 8 10 −12 −101 4 −2

A2

R3←R3−R1∼

1 8 10 −12 −100 −4 −3

A3

R3←R3− 13R2∼

1 8 10 −12 −100 0 1

3

A4

det (A4) = 1 (−12) 13 = −4 By the properties:

det (A4) = det (A3) = det (A2) = det (A1) = − det (A) =⇒ det (A) = 4


ExampleA




A =

2 4 −81 8 11 4 −2

det (A) = −

∣∣∣∣∣∣1 8 10 −12 −100 0 1

3

∣∣∣∣∣∣ = −1 · (−12) · 13 = 4

2. Alternatively, 2 4 −81 8 11 4 −2

R2← 12R2∼

1 2 −41 8 11 4 −2

R2←R2−R1∼

1 2 −40 6 51 4 −2

R3←R3−R1∼

1 2 −40 6 50 2 2

R3←R3− 13R2∼

1 2 −40 6 50 0 1

3

∣∣∣∣∣∣

1 2 −41 8 11 4 −2

∣∣∣∣∣∣ = 12 det(A). ⇒ det(A) = 2

∣∣∣∣∣∣1 2 −41 8 11 4 −2

∣∣∣∣∣∣ = · · · = 2

∣∣∣∣∣∣1 2 −40 6 50 0 1

3

∣∣∣∣∣∣ = 4


Recall

det

a11 a12 a13a21 a22 a23a31 a32 a33

= a11a22a33 + a12a23a31 + a13a21a32

−(a31a22a13 + a32a23a11 + a33a21a12) a11 a12 a13a21 a22 a23a31 a32 a33

R2←R2− a21a11

R1∼

a11 a12 a130 a22 − a12a21

a11a23 − a13a21

a11a31 a32 a33

R3←R3− a31

a11R1

∼

a11 a12 a130 a22 − a12a21

a11a23 − a13a21

a110 a32 − a12a31

a11a33 − a12a31

a11

R3←R3− a11a32−a12a31

a11a22−a12a21R2

∼

a11 a12 a130 a22 − a12a21

a11a23 − a13a21

a110 0 b

,

where b = a33 − a12a31a11− a11a32−a12a31

a11a22−a12a21

(a23 − a13a21

a11

)Eventually, by simplification,∣∣∣∣∣∣

a11 a12 a13a21 a22 a23a31 a32 a33

∣∣∣∣∣∣ =∣∣∣∣∣∣a11 a12 a130 a22 − a12a21

a11a23 − a13a21

a110 0 b

∣∣∣∣∣∣ = b (a11a22 − a12a21)


Minors

GoalWe shall compute the determinant of an n× n , through the cofactor expansion.Basically, this expansion express an n × n determinant as a linear combination of(n − 1)× (n − 1) determinants.

DefinitionLet A be an n × n matrix. The ijth minor of A is the determinant of the(n − 1)× (n − 1) matrix obtained from A , by deleting the ith row and jthcolumn. We write Mij for the ijth minor.

Example

A =

1 −1 2−3 5 10 2 3

M11 =

∣∣∣∣∣∣1 −1 2−3 5 10 2 3

∣∣∣∣∣∣ =∣∣∣∣ 5 12 3

∣∣∣∣ = 5 · 3− 1 · 2 = 13


Example

A =

1 −1 2−3 5 10 2 3

=⇒ M11 =

∣∣∣∣5 12 3

∣∣∣∣ = 13

M21 =

∣∣∣∣∣∣1 −1 2−3 5 10 2 3

∣∣∣∣∣∣ =∣∣∣∣ −1 2

2 3

∣∣∣∣ = (−1) · 3− 2 · 2 = −7

M31 =

∣∣∣∣∣∣1 −1 2−3 5 10 2 3

∣∣∣∣∣∣ =∣∣∣∣ −1 2

5 1

∣∣∣∣ = (−1) · 1− 2 · 5 = −11

M12 ==

∣∣∣∣ −3 10 3

∣∣∣∣ = −9M13 =

∣∣∣∣ −3 50 2

∣∣∣∣ = −6 M22 =

∣∣∣∣ 1 20 3

∣∣∣∣ = 3 M23 =

∣∣∣∣ 1 −10 2

∣∣∣∣ = 2

M32 =

∣∣∣∣ 1 2−3 1

∣∣∣∣ = 7 M33 =

∣∣∣∣ 1 −1−3 5

∣∣∣∣ = 2


Cofactor

Definition

The ijth cofactor of A is Cij = (−1)i+jMij .

∣∣∣∣∣∣∣∣∣+ − + − · · ·− + − + · · ·+ − + − · · ·...

......

.... . .

∣∣∣∣∣∣∣∣∣Example

A =

1 −1 2−3 5 10 2 3

=⇒M11 = 13 M12 = −9 M13 = −6M21 = −7 M22 = 3 M23 = 2M31 = −11 M32 = 7 M33 = 2∣∣∣∣∣∣

+ − +− + −+ − +

∣∣∣∣∣∣ =⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2

C31 = −11 C32 = −7 C33 = 2


Determinant expansion

TheoremLet A be an n × n matrix. Then det(A) is calculated by picking a row (orcolumn) and taking the product of each entry in that row (column) with itscofactor and adding these products together. This process is known as expandingalong the ith row (or column).

det (A) =

∣∣∣∣∣∣∣a11 · · · a1n...

. . ....

an1 · · · ann

∣∣∣∣∣∣∣along the 1st row: det(A) = a11C11 +a12C12 + · · ·+ a1nC1n

along the last column: det(A) = a1nC1n +a2nC2n + · · ·+ annCnn

along the ith row: det(A) = ai1Ci1 + ai2Ci2 + · · ·+ ainCin

along the jth column: det(A) = a1jC1j + a2jC2j + · · ·+ anjCnj

RemarkRemember that: we only pick ONE row or column.


Example

A =

1 −1 2−3 5 10 2 3

=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2

C31 = −11 C32 = −7 C33 = 2

along the 1st row:

det(A) = 1 · 13 + (−1) · 9 + 2 · (−6) = 13− 9− 12 = −8

along the 2nd row:

det(A) = (−3) · 7 + 5 · 3 + 1 · (−2) = −21 + 15− 2 = −8

along the 3rd row:

det(A) = 0 · (−11) + 2 · (−7) + 3 · 2 = 0− 14 + 6 = −8

along the 1st column:

det(A) = 1 · 13 + (−3) · 7 + 0 · (−11) = 13− 21 + 0 = −8

along the 2nd column:

det(A) = (−1) · 9 + 5 · 3 + 2 · (−7) = −9 + 15− 14 = −8

along the 3rd column:

det(A) = 2 · (−6) + 1 · (−2) + 3 · 2 = −12− 2 + 6 = −8


Example

A =

1 −1 2−3 5 10 2 3

=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2

C31 = −11 C32 = −7 C33 = 2

∣∣∣∣∣∣+ − +− + −+ − +

∣∣∣∣∣∣

∣∣∣∣∣∣1 −1 2−3 5 10 2 3

∣∣∣∣∣∣ = 0 ·∣∣∣∣ −1 2

5 1

∣∣∣∣−2 · ∣∣∣∣ 1 2−3 1

∣∣∣∣+ 3 ·∣∣∣∣ 1 −1−3 5

∣∣∣∣= −2 (1 · 1− 2 · (−3)) + 3 (1 · 5− (−1) · (−3)) = −8

For a triangular matrix∣∣∣∣∣∣∣∣∣∣∣

a11 a12 a13 · · · a1n0 a22 a23 · · · a2n0 0 a33 · · · a3n...

......

. . ....

0 0 0 · · · ann

∣∣∣∣∣∣∣∣∣∣∣= a11 · (−1)1+1

∣∣∣∣∣∣∣∣∣a22 a23 · · · a2n0 a33 · · · a3n...

.... . .

...0 0 · · · ann

∣∣∣∣∣∣∣∣∣= a11 · a22

∣∣∣∣∣∣∣a33 · · · a3n...

. . ....

0 · · · ann

∣∣∣∣∣∣∣= · · · = a11a22 · · · ann.


Example

u =

u1u2u3

, v =

v1v2v3

⇒ u×v =

u1u2u3

×v1v2v3

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1

=

∣∣∣∣∣∣u1 v1 iu2 v2 ju3 v3 k

∣∣∣∣∣∣ .Expand along the last column:∣∣∣∣∣∣


∣∣∣∣∣∣ = i∣∣∣∣ u2 v2u3 v3

∣∣∣∣−j∣∣∣∣ u1 v1u3 v3

∣∣∣∣+ k∣∣∣∣ u1 v1u2 v2

∣∣∣∣= i

∣∣∣∣ u2 v2u3 v3

∣∣∣∣+j∣∣∣∣ u3 v3u1 v1

∣∣∣∣+ k∣∣∣∣ u1 v1u2 v2

∣∣∣∣= (u2v3 − u3v2) i + (u3v1 − u1v3) j + (u1v2 − u2v1) k

=

u2v3 − u3v2u3v1 − u1v3u1v2 − u2v1


Example

Compute

∣∣∣∣∣∣∣∣1 2 3 00 1 −1 21 −3 5 10 0 2 3

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣+ − + −− + − ++ − + −− + − +

∣∣∣∣∣∣∣∣

Solution. The key is to pick a row/column containing as many 0 as possible.last row:∣∣∣∣∣∣∣∣

1 2 3 00 1 −1 21 −3 5 10 0 2 3

∣∣∣∣∣∣∣∣ = −0 · ||+ 0 · || − 2

∣∣∣∣∣∣1 2 00 1 21 −3 1

∣∣∣∣∣∣+ 3

∣∣∣∣∣∣1 2 30 1 −11 −3 5

∣∣∣∣∣∣∣∣∣∣∣∣+ − +− + −+ − +

∣∣∣∣∣∣ = −2

(1∣∣∣∣ 1 2−3 1

∣∣∣∣− 2∣∣∣∣ 0 2

1 1

∣∣∣∣)︸︷︷︸1st row

+ 3(

1∣∣∣∣ 1 −1−3 5

∣∣∣∣+ 1∣∣∣∣ 2 3

1 −1

∣∣∣∣)︸︷︷︸1st column

= −2 ((1 + 6)− 2 (−2)) + 3 ((5− 3) + (−5))

= −2 (7 + 4) + 3 (2− 5) = −22− 9 = −31


Example

∣∣∣∣∣∣∣∣∣∣1 0 1 2 30 −1 0 3 01 2 1 0 00 0 0 4 01 0 0 5 2

∣∣∣∣∣∣∣∣∣∣=

∣∣∣∣∣∣∣∣∣∣1 0 1 2 30 −1 0 3 01 2 1 0 00 0 0 4 01 0 0 5 2

∣∣∣∣∣∣∣∣∣∣= 4 · (−1)4+4

∣∣∣∣∣∣∣∣1 0 1 30 −1 0 01 2 1 01 0 0 2

∣∣∣∣∣∣∣∣= 4

(−1) · (−1)2+2

∣∣∣∣∣∣1 1 31 1 01 0 2

∣∣∣∣∣∣ = −4

∣∣∣∣∣∣1 1 31 1 01 0 2

∣∣∣∣∣∣= −4

(1 · (−1)3+1

∣∣∣∣ 1 31 0

∣∣∣∣+ 2 · (−1)3+3∣∣∣∣ 1 11 1

∣∣∣∣)= −4 ((−3) + 2 · (1− 1)) = 12


Classwork

Evaluate the determinant:∣∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3

∣∣∣∣∣∣∣∣∣Lecture resumes at 7:45.


Solution

Expand along the first row seems to be (a little) easier:∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3

∣∣∣∣∣∣∣∣ = 1

∣∣∣∣∣∣3 1 24 3 −24 1 3

∣∣∣∣∣∣− 1

∣∣∣∣∣∣2 1 24 3 −23 1 3

∣∣∣∣∣∣+ 0

∣∣∣∣∣∣2 3 24 4 −23 4 3

∣∣∣∣∣∣− (−1)

∣∣∣∣∣∣2 3 14 4 33 4 1

∣∣∣∣∣∣=

∣∣∣∣∣∣3 1 24 3 −24 1 3

∣∣∣∣∣∣−∣∣∣∣∣∣2 1 24 3 −23 1 3

∣∣∣∣∣∣+∣∣∣∣∣∣2 3 14 4 33 4 1

∣∣∣∣∣∣∣∣∣∣∣∣3 1 24 3 −24 1 3

∣∣∣∣∣∣ = 3∣∣∣∣3 −21 3

∣∣∣∣− ∣∣∣∣4 −24 3

∣∣∣∣+ 2∣∣∣∣4 34 1

∣∣∣∣ = 3 (9 + 2)− (12 + 8) + 2 (4− 12) = −3

∣∣∣∣∣∣2 1 24 3 −23 1 3

∣∣∣∣∣∣ = 2∣∣∣∣3 −21 3

∣∣∣∣− ∣∣∣∣4 −23 3

∣∣∣∣+ 2∣∣∣∣4 33 1

∣∣∣∣ = 2 (9 + 2)− (12 + 6) + 2 (4− 9) = −6


Solution

∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3

∣∣∣∣∣∣∣∣ = 1

∣∣∣∣∣∣3 1 24 3 −24 1 3

∣∣∣∣∣∣− 1

∣∣∣∣∣∣2 1 24 3 −23 1 3

∣∣∣∣∣∣+ 0

∣∣∣∣∣∣2 3 24 4 −23 4 3

∣∣∣∣∣∣− (−1)

∣∣∣∣∣∣2 3 14 4 33 4 1

∣∣∣∣∣∣=

∣∣∣∣∣∣3 1 24 3 −24 1 3

∣∣∣∣∣∣︸︷︷︸−3

−

∣∣∣∣∣∣2 1 24 3 −23 1 3

∣∣∣∣∣∣︸︷︷︸−6

+

∣∣∣∣∣∣2 3 14 4 33 4 1

∣∣∣∣∣∣∣∣∣∣∣∣2 3 14 4 33 4 1

∣∣∣∣∣∣ = 2∣∣∣∣4 34 1

∣∣∣∣− 3∣∣∣∣4 33 1

∣∣∣∣+ ∣∣∣∣4 43 4

∣∣∣∣ = 2 (4− 12)− 3 (4− 9) + (16− 12) = 3

Thus, ∣∣∣∣∣∣∣∣1 1 0 −12 3 1 24 4 3 −23 4 1 3

∣∣∣∣∣∣∣∣ = (−3)− (−6) + 3 = 6.


Solution

Alternatively, we can also use the row operations:

1 1 0 −12 3 1 24 4 3 −23 4 1 3

A

R2 ← R2 − 2R1

R3 ← R3 − 4R1

R4 ← R4 − 3R1∼

1 1 0 −10 1 1 40 0 3 20 1 1 6

A1

R4←R4−R2∼

1 1 0 −10 1 1 40 0 3 20 0 0 2

A2∣∣∣∣∣∣∣∣

1 1 0 −12 3 1 24 4 3 −23 4 1 3

∣∣∣∣∣∣∣∣ = det (A) = det (A1) = det (A2) = 1 · 1 · 3 · 2 = 6.


Properties of determinants

det(I ) = 1det (AB) = det(A) det(B) This is importantA is invertible iff det(A) 6= 0 , and in this case det

(A−1

)= 1

det(A)

1 = det(I ) = det(AA−1) = det(A) det

(A−1)

det(AT)= det(A)

det(kA) = kn det(A) (Recall that ARj←−kRj∼ B ⇒ det (B) = k det(A))

If A has a row of zeros or a column of zeros, then det(A) = 0.If A has two equal rows or columns, then det(A) = 0. (rank(A) < n)


Full rank

For an n × n square matrix A, it is said to have full rank if rank(A) = n. Let A bea full rank n × n-matrix.

For x = [x1 x2 . . . xn]T and b = [b1 b2 . . . bn]

T ∈ Rn, the system Ax = b isconsistent and has a unique solution.Write the column vectors of A as A = [v1 v2 . . . vn]. The casting-outalgorithm will result in an echelon form of rank n, so that every column ispivot. v1, v2, . . . , vn are linearly independent ⇒ they form a basis of Rn.Alternatively, you can let b = 0 in the system above. The unique solution isthe trivial solution, indicating linear independence.The reduced echelon form of A is In. det (A) = C det(In) for some constantC 6= 0. Since det (In) = 1, det(A) = C 6= 0.[A|I ] ∼

[I |A−1

]so that A is invertible.

rank(AT ) = n


Example

1. How many solutions:

{5x + 2y = −97x + 3y = −13

? (Midterm I, Problem 11)

Solution.∣∣∣∣ 5 27 3

∣∣∣∣ = 5 · 3− 2 · 7 = 1 6= 0

2. Which of the following is a basis of R3 ? (Midterm II, Problem 14)

(A)

1

00

,

300

,

121

(B)

1

10

,

02−1

,

211

(C)

1

23

,

78−1

(D)

1

10

,

111

,

012

,

103

Solution. A basis of R3 should contain 3 vectors.

For (A),

∣∣∣∣∣∣1 3 10 0 20 0 1

∣∣∣∣∣∣ = 1 · 0 · 1 = 0 —not linearly independent.


Example

3. Which two vectors would extend the set

2−6−20

,

1−2−10

to a basis of R4?

(Midterm II

Problem 16

)

(A)

0100

,

0001

(B)

1000

,

1010

(C)

1000

,

0001

(D)

1000

,

0100

Solution. We can compute determinants one by one:

(A)

∣∣∣∣∣∣∣∣2 1 0 0−6 −2 1 0−2 −1 0 00 0 0 1

∣∣∣∣∣∣∣∣ = 1

∣∣∣∣∣∣2 1 0−6 −2 1−2 −1 0

∣∣∣∣∣∣ = 1 · (−1)∣∣∣∣ 2 1−2 −1

∣∣∣∣ = 0

Similarly, (B)

∣∣∣∣∣∣∣∣2 1 1 1−6 −2 0 0−2 −1 0 10 0 0 0

∣∣∣∣∣∣∣∣ = 0, (D)

∣∣∣∣∣∣∣∣2 1 1 0−6 −2 0 1−2 −1 0 00 0 0 0

∣∣∣∣∣∣∣∣ = 0


Inverse

DefinitionLet A be an n × n matrix.

The cofactor matrix of A is cof (A) =

C11 C12 · · · C1nC21 C22 · · · C2n...

.... . .

...Cn1 Cn2 · · · Cnn

.

The adjugate of A is adj (A) = (cof (A))T =

C11 C21 · · · Cn1C12 C22 · · · Cn2...

.... . .

...C1n C2n · · · Cnn

.

Theorem

A−1 =1

det(A)adj (A)


Example:(A−1 = 1

det(A)adj (A))

Recall an early example:

A =

1 −1 2−3 5 10 2 3

=⇒C11 = 13 C12 = 9 C13 = −6C21 = 7 C22 = 3 C23 = −2

C31 = −11 C32 = −7 C33 = 2=⇒ det(A) = −8

cof (A) =

13 9 −67 3 −2−11 −7 2

adj(A) =

13 7 −119 3 −7−6 −2 2

=⇒ A−1 =1−8

13 7 −119 3 −7−6 −2 2

=

− 138 − 7

8118

− 98 − 3

878

34

14 − 1

4

Check AA−1 = I as an exercise


Example 2× 2

Let A =

[1 2−3 4

]=⇒ det (A) = 1 · 4− 2 (−3) = 10 6= 0.

M11 = 4 M12 = −3M21 = 2 M22 = 1 =⇒ C11 = 4 C12 = 3

C21 = −2 C22 = 1

cof (A) =

[4 3−2 1

]=⇒ adj(A) = cof (A)T =

[4 −23 1

]Thus,

A−1 =1

det(A)adj(A) =

110

[4 −23 1

]=

[ 25 − 1

5310

110

]Theorem [

a bc d

]−1

=1

ad − bc

[d −b−c a

]Proof. [

a bc d

] [d −b−c a

]=

[ad − bc 0

0 ad − bc


Cramer’s rule (*)

TheoremSuppose A is an invertible n × n-matrix and we wish to solve the system Ax = b ,where x = [x1 x2 . . . xn]

T and b = [b1 b2 . . . bn]T ∈ Rn.

Then xi can be computed by the rule xi = det(Ai )/ det(A) where Ai is the matrixobtained by replacing the ith column of A with b.

Example{5x + 2y = −97x + 3y = −13

⇒ A =

[5 27 3

]⇒ det(A) = 1 6= 0. x =

[xy

]and b =

[−9−13

]

x =det (A1)

det (A)=

∣∣∣∣ −9 2−13 3

∣∣∣∣1

= (−9) · 3− 2 (−13) = −1

y =det (A2)

det (A)=

∣∣∣∣5 −97 −13

∣∣∣∣1

= 5 · (−13)− (−9) · 7 = −2

5 (−1) + 2 (−2) = −5− 4 = −9

7 (−1) + 3 (−2) = −7− 6 = −13


Summary of Chpt. 7

Determinant: ONLY square matrices have determinants.1 Calculation

2× 2 and triangular;

Row operations;

A




minor, cofactor, expansionYou should be able to compute a 4× 4 or 5× 5 determinant.

2 Property:det (A) 6= 0rank(A) = nrow/column vectors of A are linearly independent ⇒ form a basis of Rn

A is invertible with det(A−1) 6= 0 A−1 = 1det(A)

adj (A)

the system Ax = b has a unique solution ⇒if b = 0, the homogeneous systemhas only the trivial solution



Lin Jiu


June 13th, 2019


Eigenvalues and eigenvectors

The red arrow changes direction butthe blue arrow does not. The blue ar-row is an eigenvector of this mappingbecause it does not change direction,and since its length is unchanged, itseigenvalue is 1.

DefinitionLet A be an n × n matrix, v be a non-zero vector, and λ be a scalar such thatAv = λv. Then,

v is called an eigenvector of A,and λ is called the corresponding eigenvalue.


Example

1. A =

[2 00 1

]. Which of the following are eigenvectors of A?

u1 =

[10

]u2 =

[01

]u3 =

[11

]

Au1 =

[2 00 1

] [10

]=

[2 · 1+ 0 · 00 · 1+ 1 · 0

]=

[20

]= 2u1 YES eigenvalue 2

Au2 =

[2 00 1

] [01

]= 0

[20

]+ 1

[01

]=

[01

]= u2 YES eigenvalue 1

Au3 =

[2 00 1

] [11

]=

[2 00 1

] [10

]+

[2 00 1

] [01

]=

[21

]6= ku3 NO


Example

2. Let A =

[4 −36 −5

]and u1 =

[13

], u2 =

[12

], u3 =

[11

], and u4 =

[00

]. Which

are the eigenvalues of A? Find the corresponding eigenvalues too.Solution.

Au1 =

[4 −36 −5

] [13

]=

[4 · 1+ (−3) · 36 · 1+ (−5) · 3

]=

[−5−9

]6= λu1 NO

Au2 =

[4 −36 −5

] [12

]= 1

[46

]+ 2

[−3−5

]=

[−2−4

]= −2u2 YES

Eigenvalueλ = −2

Au3 =

[4 −36 −5

] [11

]= 1

[46

]+ 1

[−3−5

]=

[11

]= u3 YES

Eigenvalueλ = 1

Au4 =

[4 −36 −5

] [00

]=

[00

]= u4 NO An eigenvector must be non-zero.


Example

3. Let A =

−3 0 0−6 −1 23 −6 6

. The eigenvalues are 2, 3 and −3. Find the

corresponding eigenvector.Solution. (1) We need to solve v1 from the equation: Av1 = 2v1 ⇐⇒ Av1 = 2Iv1

⇐⇒ 0 = Av1 − 2Iv1 = (A− 2I ) v1 (A− 2I ) v1 = 0 –Homogeneous system

A− 2I =

−3 0 0−6 −1 23 −6 6

− 2

1 0 00 1 00 0 1

=

−5 0 0−6 −3 23 −6 4

−5 0 0 0−6 −3 2 03 −6 4 0

R1←− 15R1∼

1 0 0 0−6 −3 2 03 −6 4 0

R2←R2+6R1R3←R3−3R1∼ 1 0 0 0

0 −3 2 00 −6 4 0

R3←R3−2R2∼

1 0 0 00 −3 2 00 0 0 0

R2←− 13R2∼

1 0 0 00 1 − 2

3 00 0 0 0

xyz

=

023 tt

t=3=⇒ v1 =

023

(to avoid fractions) Exercise check: Av1 = 2v1


Example

(2) Av2 = 3v2 ⇐⇒ Av2 = 3Iv2 ⇐⇒ (A− 3I ) v2 = 0 — (A− 3I ) v2 = 0

A− 3I =

−3 0 0−6 −1 23 −6 6

− 3

1 0 00 1 00 0 1

=

−6 0 0−6 −4 23 −6 3

R1←− 16R1∼

1 0 0−6 −4 23 −6 3

R2←R2+6R1R3←R3−3R1∼

1 0 00 −4 20 −6 3

R2←−R24∼

1 0 00 1 − 1

20 −6 3

R3←R3+6R2∼

1 0 00 1 − 1

20 0 0

1 0 0 0

0 1 − 12 0

0 0 0 0

⇒xyz

=

0t2t

t=2=⇒ v2 =

012

Double-check:

Av2 =

−3 0 0−6 −1 23 −6 6

012

=

−3 · 0+ 0 · 1+ 0 · 2−6 · 0− 1 · 1+ 2 · 23 · 0− 6 · 1+ 6 · 2

=

036

= 3v2


Example(3) Av3 = −3v3 ⇐⇒ (A+ 3I ) v3 = 0 — (A+ 3I ) v3 = 0

A+ 3I =

−3 0 0−6 −1 23 −6 6

+ 3

1 0 00 1 00 0 1

=

0 0 0−6 2 23 −6 9

R1↔R2R2↔R3∼

−6 2 23 −6 90 0 0

R1←−

R66∼

1 − 13 − 1

33 −6 90 0 0

R2←R2−3R1∼

1 − 13 − 1

30 −5 100 0 0

R2←−R2

5∼

1 − 13 − 1

30 1 −20 0 0

R1←R1+

R23∼

1 0 −10 1 −20 0 0

1 0 −1 0

0 1 −2 00 0 0 0

=⇒

xyz

=

t2tt

t=1=⇒ v3 =

121

Double-check:

Av3 =

−3 0 0−6 −1 23 −6 6

121

= 1

−3−63

+ 2

0−1−6

+ 1

026

=

−3−6−3

= −3v3


Remark

From the two examples above, we see, for an square matrix A:1 Given an eigenvector v, we can compute Av = λv to find the corresponding

eigenvalue.2 Given an eigenvalue λ, since Av = λv⇐⇒ (A− λI ) v = 0, we can proceed

to find the solution space of the homogeneous system.3 It is obvious that for an eigenvalue λ, there are more than one eigenvectors:

If Av = λv, for any non-zero scalar k: A (kv) = k (Av) = kλv = λ (kv);If both v1 and v2 are eigenvectors of A with respect to the SAME eigenvalueλ: A (v1 + v2) = Av1 + Av2 = λv1 + λv2 = λ (v1 + v2)This is natural since for an eigenvalue λ, the solution space of thehomogeneous system: (A− λI ) v = 0 contains all the eigenvectors and thezero vector, which is a subspace. Naturally, it is closed under addition andscalar multiplication.

4 Just given the matrix A, can we find the eigenvalues and eigenvectors?


Finding eigenvalues

Without given any information, we can first find the eigenvalues and then thecorresponding eigenvectors.

a squre matrix −→ eigenvalue(s) −→ eigenvectors.

In general, λ is an eigenvalue if and only if there exists a non-zero vector v suchthat

Av = λv ⇐⇒ Av = λIv ⇐⇒ Av − λIv = 0 ⇐⇒ (A− λI ) v = 0⇐⇒ The homogeneous system (A− λI ) v = 0 has non-trivial solutions.⇐⇒ det (A− λI ) = 0 ←− This only involves λ not v.Therefore, we can take the following two steps:

1 Solve the equation det (A− λI ) = 0. Solutions λ1, . . . , λn are eigenvalues.2 For each λi , to find the corresponding eigenvectors, we solve the

homogeneous system (A− λi I )v = 0. In particular, we choose the basicsolutions as the representatives of the eigenvectors.


Example

Find the eigenvalues and eigenvectors of A =

[1 1−2 4

].

Solution. 1. Solve λ from the equation det (A− λI ) = 0.

A− λI =[1 1−2 4

]− λ

[1 00 1

]=

[1 1−2 4

]−[λ 00 λ

]=

[1−λ 1−2 4−λ

]A− λI is obtained by adding −λ to all the main diagonal entries.

det (A− λI ) = (1− λ) (4− λ)− 1 (−2) = 4 −λ −4λ +λ2 +2 = λ2 − 5λ+ 6

det (A− λI ) = λ2 − 5λ+ 6 = (λ− 3) (λ− 2) ax2 + bx + c =

a

(x − −b+

√b2−4ac

2a

)(x − −b−

√b2−4ac

2a

)

=⇒ det (A− λI ) = 0 has two solutions: λ1 = 2 and λ2 = 3, which are theeigenvalues of A.2. For each eigenvalue (λ1, λ2 here), find the corresponding eigenvector, bysolving the homogeneous system

(A− λI ) v = 0


Example

A =

[1 1−2 4

]=⇒ λ1 = 2, λ3 = 3

λ1 = 2: A− 2I =[1−2 1−2 4−2

]=

[−1 1−2 2

]Row operations:[

−1 1−2 2

]R1←−R1∼

[1 −1−2 2

]R2←R2+2R1∼

[1 −10 0

]⇒[

1 −1 00 0 0

]Suppose the two unknows are x and y . Then, y = t is free and x = y = t.

General solution:[tt

]and we pick the basic solution v1 =

[11

], as the

representative of the eigenvectors with respect to λ1 = 2.

Check:

Av1 =

[1 1−2 4

] [11

]=

[1 · 1+ 1 · 1

(−2) · 1+ 4 · 1

]=

[22

]= 2

[11

]= λ1v1.


Example

A =

[1 1−2 4

]=⇒ λ1 = 2, λ3 = 3

λ2 = 3: A− 3I =[1−3 1−2 4−3

]=

[−2 1−2 1

]Row operations:[

−2 1−2 1

]R2←R2−R1∼

[−2 10 0

]R1←− 1

2R1∼[1 − 1

20 0

]⇒[

1 − 12 0

0 0 0

]Suppose the two unknows are x and y . Then, y = t is free andx = y/2 = t/2.

General solution:[t/2t

]and we pick the basic solution v2 =

[1/21

], as the

representative of the eigenvectors with respect to λ2 = 3.

Check:Av2 =

[1 1−2 4

] [1/21

]= 1

2

[1−2

]+ 1

[14

]=

[ 323

]= 3

[1/21

]= λ2v2.


Example

Therefore, for matrix

A =

[1 1−2 4

]It has two eigenvalues: λ1 = 2, λ3 = 3

For λ1 = 2, the basis of corresponding eigenvectors is{[11

]}For λ2 = 3, the basis of corresponding eigenvectors is{[

1/21

]}How about a 3× 3 matrix?

Example

Find the eigenvalues and eigenvectors of A =

−2 −3 60 1 0−3 −3 7


Example

A =

−2 −3 60 1 0−3 −3 7

Solve the equation

det (A− λI ) = 0: A− λI =

−2− λ −3 60 1− λ 0−3 −3 7− λ

det (A− λI ) =

∣∣∣∣∣∣−2− λ −3 6

0 1− λ 0−3 −3 7− λ

∣∣∣∣∣∣ = (1− λ)∣∣∣∣−2− λ 6−3 7− λ

∣∣∣∣= (1− λ) [(−2− λ) (7− λ)− 6 (−3)]= (1− λ)

[−14+ 2λ− 7λ+ λ2 + 18

]= (1− λ)

(λ2 − 5λ+ 4

)= (1− λ) (λ− 1) (λ− 4)= − (λ− 1)2 (λ− 4)

The roots of det (A− λI ) = − (λ− 1)2 (λ− 4) = 0 are the eigenvalues of A:λ1 = 1 and λ2 = 4

For λ1 = 1: A− I =

−3 −3 60 0 0−3 −3 6

∼1 1 −20 0 00 0 0

y = t

z = s⇒ x = −t + 2s

General solution:

xyz

= t

−110

+ s

201

basicsolutions

:

−110

,201

.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 14 / 25

Example

A =

−2 −3 60 1 0−3 −3 7

⇒ A− 4I =

−6 −3 60 −3 0−3 −3 3

R1←R1/(−6)

R2←R2/(−3)R3←R3/(−3)∼

1 1/2 −10 1 01 1 −1

R3←R3−R1∼

1 1/2 −10 1 00 1/2 0

R1←R1−R2/2R3←R3−R2/2∼

1 0 −10 1 00 0 0

y = 0, z = t =⇒ x = z = t

General solution:

xyz

= t

101

=⇒ Basic solution:

101

.A has two eigenvalues: λ1 = 1, λ2 = 4

Eigenvalue λ = 1: Multiplicity 2 ; Basis of eigenvectors :

−110

,201

Eigenvalue λ = 4: Multiplicity 1 ; Basis of eigenvectors :

101


Classwork

A =

1 3 30 4 30 0 7

Find the eigenvalues and eigenvectors of A



Solution

A =

1 3 30 4 30 0 7

=⇒ A− λI =

1− λ 3 30 4− λ 30 0 7− λ

=⇒ det (A− λI ) = (1− λ) (4− λ) (7− λ)=⇒ det (A− λI ) = 0 has three roots: λ1 = 1, λ2 = 4, λ3 = 7

λ1 = 1: A− I =

0 3 30 3 30 0 6

∼0 1 10 1 10 0 1

∼0 1 10 0 10 0 0

∼0 1 00 0 10 0 0

xyz

=

t00

= t

100

. Basic solution: v1 =

100

.λ2 = 4: A− 4I =

−3 3 30 0 30 0 3

∼1 −1 −10 0 10 0 1

∼1 −1 00 0 10 0 0

xyz

=

tt0

= t

110


110

.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 17 / 25

Solution

A =

1 3 30 4 30 0 7

=⇒ det (A− λI ) = (1− λ) (4− λ) (7− λ)

λ3 = 7: A− 7I =

−6 3 30 −3 30 0 0

∼1 − 1

2 − 12

0 1 −10 0 0

∼1 0 −10 1 −10 0 0

xyz

=

ttt

= t

111


111

.A has three eigenvalues:

Eigenvalue λ = 1: Basis of eigenvectors :

100


110


111


Characteristic polynomial

DefinitionIf A is a square matrix , the expression p (λ) = det (A− λI ) is always apolynomial in λ. It is called the characteristic polynomial of A. Moreover, theeigenvalues of A are the roots of the characteristic polynomial.

Example

1. A =

1 3 30 4 30 0 7

The characteristic polynomial is :

det (A− λI ) =

∣∣∣∣∣∣1− λ 3 30 4− λ 30 0 7− λ

∣∣∣∣∣∣ = (1− λ) (4− λ) (7− λ) .

2. A =

[1 1−2 4

]The characteristic polynomial is

det (A− λI ) =∣∣∣∣1− λ 1−2 4− λ

∣∣∣∣ = (1− λ) (4− λ)− 1 (−2) = (λ− 3) (λ− 2)Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 13th, 2019 19 / 25

Properties

A =

1 3 30 4 30 0 7

=⇒ p1 (λ) =

∣∣∣∣∣∣1− λ 3 30 4− λ 30 0 7− λ

∣∣∣∣∣∣ = (1− λ) (4− λ) (7− λ).

If A is an n × n matrix, the degree of p (λ) is n. There are n eigenvaluescounting multiplicity.

B =

−2 −3 60 1 0−3 −3 7

=⇒ p2 (λ) = (4− λ) (λ− 1)2 ⇒

{λ1 = λ2 = 1λ3 = 4

The constant term of the characteristic polynomial = p (0) = det (A)p1 (0) = 1 · 4 · 7 = det (A) Exercise: check, p2 (0) = 4 = det (B)

(*) The n − 1 term

p (λ) = (−1)nλn + (−1)n−1 tr(A)λn−1 + · · ·+ det(A).

For triangluar matrics: p(λ) = (a11 − λ) (a22 − λ) · · · (ann − λ) Eigenvaluesare the entries on the main diagonal.


Multiplicity

Recall the example B =

−2 −3 60 1 0−3 −3 7

=⇒ p2 (λ) = (4− λ) (λ− 1)2

B has two eigenvalues: λ1 = 1, λ2 = 4

Eigenvalue λ1 = 1: Multiplicity 2 ; Basis of eigenvectors :

−110

,201

Eigenvalue λ2 = 4: Multiplicity 1 ; Basis of eigenvectors :

101

Focus on λ1 = 1, the solution space of the homogeneous system (A− I ) v = 0.

has dimension 2, since the basis

−11

0

,201

has two vectors.


Multiplicity

Consider another example A =

2 1 00 2 00 0 1

1. Find the characteristic polynomial: det (A− λI ) =

∣∣∣∣∣∣2− λ 1 00 2− λ 00 0 1− λ

∣∣∣∣∣∣= (2− λ)2 (1− λ)

2. Find the roots (eigenvalues):λ1 = 1 with multiplicity 1and λ2 = 2 with multiplicity 2.

3. Find the eigenvectors:

λ1 = 1: A− I =

1 1 00 1 00 0 0

∼ 1 0 0

0 1 00 0 0

=⇒ v1 =

00t

= t

001

λ2 = 2: A− 2I =

0 1 00 0 00 0 −1

∼ 0 1 0

0 0 10 0 0

=⇒ v2 =

t00

= t

100


Multiplicity

We see for λ1 = 2, it has multiplicity 2 in the equation p (λ) = det (A− λI ) = 0and the dimension of the solution space of (A− 2I ) v = 0 is 1.

DefinitionLet A be a square matrix.

1 If λ is a root of the characteristic polynomial of multiplicity k , we say that λis an eigenvalue of algebraic multiplicity k .

2 For an eigenvalue λ, the solution space {v|(A− λI )v = 0} = {v | Av = λv},denoted by Eλ, is called an eigenspace of A, with respect to λ.

3 If λ has an m-dimensional eigenspace , i.e., dimEλ = m, we say λ hasgeometric multiplicity m.

Theorem

1 ≤ geometric multiplicity ≤ algebraic multiplicity.


Examples

As we computed before:

Matrix

−2 −3 60 1 0−3 −3 7

has two eigenvalues:

Eigenvalue λ1 = 1: algebraic multiplicity = geometric multiplicity = 2Eigenvalue λ2 = 4: algebraic multiplicity = geometric multiplicity = 1

Matrix

2 1 00 2 00 0 1

has two eigenvalues:

Eigenvalue λ1 = 1: algebraic multiplicity = geometric multiplicity = 1Eigenvalue λ2 = 2: algebraic multiplicity = 2, while geometric multiplicity = 1


Preview

Next time (the last lecture) we shall see that:An n × n-matrix is diagonalizable iff for all its eigenvalues, the algebraicmultiplicity= geometric multiplicity.



Lin Jiu


June 18th, 2019


Recall

DefinitionLet A be an n × n matrix, v be a non-zero vector, and λ be a scalar such thatAv = λv. Then,

v is called an eigenvector of A,and λ is called the corresponding eigenvalue.

RemarkAn eigenvector cannot be a zero vector, but an eigenvalue can be 0.3 2 1

0 2 10 0 0

003

=

000

.To find eigenvalues and eigenvectors, we need to :

1 Find the characteristic polynomial p (λ) = det (A− λI )2 Find the eigenvalues, which are the roots of p(λ) = 0.3 For each λi , solve the homogeneous system (A− λI ) v = 0, by providing the

basic solutions.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 2 / 24

Recall

DefinitionLet A be a square matrix.

1 If λ is a root of the characteristic polynomial of multiplicity k , we say that λis an eigenvalue of algebraic multiplicity k .

2 For an eigenvalue λ, the solution space {v|(A− λI )v = 0} = {v | Av = λv},denoted by Eλ, is called an eigenspace of A, with respect to λ.

3 If λ has an m-dimensional eigenspace, i.e., dimEλ = m, we say λ hasgeometric multiplicity m.

Theorem

1 ≤ geometric multiplicity ≤ algebraic multiplicity.


Diagonal matrices (n × n)

Definition

A diagonal matrix is of the form

d1 0 0 · · · 00 d2 0 · · · 00 0 d3 · · · 0...

......

. . ....

0 0 0 · · · dn

.Namely, only the entries on the main diagonal can be non-zero.

Example

I2 =

[1 00 1

] 1 0 00 1 00 0 0

RemarkIt does not require d1, . . . , dn to be nonzero.


Diagonal matricesAdding, subtracting, multiplying diagonal matrices are much easier and resultsare also diagonal matrices.[

a 00 b

]+

[c 00 d

]=

[a+ c 00 b + d

][a 00 b

] [c 00 d

]=

[a · c + 0 · 0 a · 0+ 0 · c0 · c + b · 0 0 · 0+ b · d

]=

[ac 00 bd

]In particular,

[a 00 b

] [a 00 b

]=

[a2 00 b2

]=⇒

([a 00 b

])100

=

[a100 00 b100

].

In general: all operations are computed componentwise on the diagonal.c1 0 · · · 00 c2 · · · 0...

.... . .

...0 0 · · · cn

+

d1 0 · · · 00 d2 · · · 0...

.... . .

...0 0 · · · dn

=

c1 + d1 0 · · · 0

0 c2 + d2 · · · 0...

.... . .

...0 0 · · · cn + dn

c1 0 · · · 00 c2 · · · 0...

.... . .

...0 0 · · · cn

d1 0 · · · 00 d2 · · · 0...

.... . .

...0 0 · · · dn

=

c1d1 0 · · · 00 c2d2 · · · 0...

.... . .

...0 0 · · · cndn


Example

Let A =

1 0 00 4 00 0 9

. Find a matrix B such that B2 = A.

Solution. From the computation above, we see B =

1 0 00 2 00 0 3

satisfies

B2 =

1 0 00 2 00 0 3

1 0 00 2 00 0 3

=

12 0 00 22 00 0 32

=

1 0 00 4 00 0 9

= A

Of course there are other solutions−1 0 00 2 00 0 3

,1 0 00 −2 00 0 3

,−1 0 0

0 −2 00 0 3

,1 0 00 2 00 0 −3

−1 0 0

0 2 00 0 −3

,1 0 00 −2 00 0 −3

,−1 0 0

0 −2 00 0 −3


An old example

Recall an early example A =

−3 0 0−6 −1 23 −6 6

p (A) = − (λ− 1)2 (λ− 4)

λ = 1 basis

−11

0

,201

λ = 4 basis

101

=⇒

A

−110

=

−110

A

201

=

201

A

101

= 4

101

=

404

Let P =

−1 2 11 0 00 1 1

=⇒AP =

A−11

0

A

201

A

101

=

−1 2 41 0 00 1 4

P

1 0 00 1 00 0 4

︸︷︷︸

D

=

P100

P

010

P

004

=

−1 21 00 1

P

004

=

−1 2 41 0 00 1 4

.

We see AP = PD ⇐⇒ A = PDP−1.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 7 / 24

Diagonalization

TheoremLet A be an n × n matrix, with n linearly independent eigenvectors v1, . . . , vnand corresponding eigenvalues λ1, . . . , λn. Avk = λkvk k = 1, . . . , n ( allowingrepetitions of λ1, . . . , λn) Then, there exists an invertible matrix P and a diagonalmatrix D such that

P−1AP = D ⇐⇒ A = PDP−1

In the case, we say that A is diagonalizable. In particular,P is the matrix whose columns are v1, . . . , vn.D is the matrix whose main diagonal entries are λ1, . . . , λn.

RemarkThe key is that A has n linearly independent eigenvectors.


Example

Diagonalize the matrix A =

[2 −21 5

]Solution. 1. Characteristic polynomial

det (A− λI ) =∣∣∣∣ 2− λ −2

1 5− λ

∣∣∣∣ = (2− λ) (5− λ)− (−2) 1 = 10− 7λ+ λ2 + 2

= λ2 − 7λ+ 12 = (λ− 3) (λ− 4)

2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = 3 λ2 = 43. Eigenvectors

A− 3I =[−1 −21 2

]∼[1 20 0

]=⇒ x = −2y =⇒ v1 =

[−21

]A− 4I =

[−2 −21 1

]∼[1 10 0

]=⇒ x = −y =⇒ v2 =

[−11

]4. Diagonalize: D =

[3 00 4

], P =

[−2 −11 1

]=⇒ P−1 = adj(P)

det(P)

M11= 1, M12 = 1,M21 = −1, M22 = −2

⇒ cof (P) =

[1 −1

−(−1) −2

]⇒ adj(P) =

[1 1−1 −2


Example

A =

[2 −21 5

]⇒ λ1 = 3, v1 =

[−21

]λ2 = 4, v2 =

[−11

]D =

[3 00 4

], P =

[−2 −11 1

]=⇒ P−1 = adj(P)

det(P) = 1−1

[1 1−1 −2

]=

[−1 −11 2

]P−1AP = D ⇐⇒ A = PDP−1

5. Check:P−1AP =

[−1 −11 2

] [2 −21 5

] [−2 −11 1

]=

[(−1) · 2+ (−1) · 1 (−1) · (−2) + (−1) · 5

1 · 2+ 2 · 1 1 · (−2) + 2 · 5

] [−2 −11 1

]=

[−3 −34 8

] [−2 −11 1

]=

[(−3) · (−2) + (−3) · 1 (−3) · (−1) + (−3) · 1

4 · (−2) + 8 · 1 4 · (−1) + 8 · 1

]=

[3 00 4

]= D


Example

Diagonalize the matrix A =

2 0 01 4 −1−2 −4 4

Solution. 1. Characteristic polynomial

det (A− λI ) =

∣∣∣∣∣∣2− λ 0 01 4− λ −1−2 −4 4− λ

∣∣∣∣∣∣ = (2− λ)∣∣∣∣4− λ −1−4 4− λ

∣∣∣∣= (2− λ) [(4− λ) (4− λ)− (−4) (−1)]= (2− λ)

[λ2 − 8λ+ 16− 4

]= (2− λ)

(λ2 − 8λ+ 12

)= (2− λ) (λ− 2) (λ− 6) = − (λ− 2)2 (λ− 6)

2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = λ2 = 2, λ3 = 63. Eigenvectors

For λ = 2, A− 2I =

0 0 01 2 −1−2 −4 2

∼ 1 2 −1

0 0 00 0 0

⇒ x = −2y + zxyz

=

−2t + sts

= t

−210

+ s

101

=⇒ v1 =

−210

, v2 =

101


Example

A =

2 0 01 4 −1−2 −4 4

=⇒λ1 = λ2 = 2

λ3 = 6, v1 =

−210

, v2 =

101

For λ = 6, A− 6I =

−4 0 01 −2 −1−2 −4 −2

∼ 1 0 0

1 −2 −11 2 1

∼

1 0 00 −2 −10 2 1

∼ 1 0 0

0 −2 −10 0 0

∼ 1 0 0

0 1 1/20 0 0

⇒

{x = 0y = − z

2

⇒

xyz

=

0−r/2r

=

0−1/21

=⇒ v3 =

0−1/21

D =

2 0 00 2 00 0 6

P =

−2 1 01 0 −1/20 1 1


Example

A =

2 0 01 4 −1−2 −4 4

=⇒ D =

2 0 00 2 00 0 6

P =

−2 1 01 0 −1/20 1 1

P−1 =

− 14

12

14

12 1 1

2− 1

2 −1 12

=⇒ PDP−1 = A

RemarkThe order is not unique. Since both v1 and v2 are eigenvectors with respectto λ = 2, we can let

D =

2 0 00 2 00 0 6

P1 =

1 −2 00 1 −1/21 0 1

Or, we can consider

D2 =

2 0 00 6 00 0 2

P2 =

1 0 −20 −1/2 11 1 0


Remark

The key is to make sure that eigenvalues and eigenvectors are located at the samecolumns.

We do not really require the basic solution(s)

A =

2 0 01 4 −1−2 −4 4

=⇒ D =

2 0 00 2 00 0 6

P =

−2 1 01 0 −1/20 1 1

•

D =

2 0 00 2 00 0 6

P3 =

−2 1 01 0 −10 1 2

•

D =

2 0 00 2 00 0 6

P4 =

−2 1 01 0 10 1 −2


Break

During the break, please consider this question.Diagonalize

A =

8 0 10−6 −3 −6−5 0 −7

.

Namely, find a diagonal matrix D and an invertiblematrix P such that

A = PDP−1

Lecture resumes at 7:30.Lin Jiu (Dalhousie University) MATH-1030 Matrix Theory & Linear Algebra I June 18th, 2019 15 / 24

Solution

A =

8 0 10−6 −3 −6−5 0 −7

1. Characteristic polynomial: det (A− λI ) =

∣∣∣∣∣∣8− λ 0 10−6 −3− λ −6−5 0 −7− λ

∣∣∣∣∣∣= − (λ+ 3)

∣∣∣∣8− λ 10−5 −7− λ

∣∣∣∣= − (λ+ 3) [(8− λ) (−7− λ)− (−5) · 10] = − (λ+ 3)

(λ2 − λ− 6

)= − (λ+ 3) (λ+ 2) (λ− 3)

2. Eigenvalues: det (A− λI ) = 0 =⇒ λ1 = −3, λ2 = −2, λ3 = 33. Eigenvectors

For λ1 = −3, A+ 3I =

11 0 10−6 0 −6−5 0 −4

∼ 1 0 10

111 0 11 0 4

5

∼

1 0 11 0 10

111 0 4

5

∼ 1 0 1

0 0 − 111

0 0 − 15

∼ 1 0 0

0 0 10 0 0

=⇒

xyz

= t

010


Solution

A =

8 0 10−6 −3 −6−5 0 −7

⇒ λ1 = −3, λ2 = −2, λ3 = 3, v1 =

010

For λ2 = −2, A+ 2I =

10 0 10−6 −1 −6−5 0 −5

∼ 1 0 1

6 1 60 0 0

∼

1 0 10 1 00 0 0

=⇒

xyz

= t

−101

For λ3 = 3, A− 3I =

5 0 10−6 −6 −6−5 0 −10

∼ 1 0 2

1 1 10 0 0

∼

1 0 20 1 −10 0 0

=⇒

xyz

= t

−211


Solution

A =

8 0 10−6 −3 −6−5 0 −7

λ1 = −3, v1 =

010

λ2 = −2, v2 =

−101

λ3 = 3, v3 =

−211

Thus,

P =

0 −1 −21 0 10 1 1

D =

−3 0 00 −2 00 0 3

such that

A = PDP−1


Application

Example

Let A =

−3 0 0−6 −1 23 −6 6

. Find A100

Solution

P =

−1 2 11 0 00 1 1

=⇒ P−1 =

0 1 01 1 −1−1 −1 2

and D =

1 0 00 1 00 0 4

=⇒

P−1AP = D ⇔ A = PDP−1.

A100 =(PDP−1) (PDP−1) · · · (PDP−1) (PDP−1)︸︷︷︸

100 copies

= PD��P−1PD��P−1PD · · ·��P−1PDP−1 = PD100P−1. Thus,

A100 =

−1 2 11 0 00 1 1

1 0 00 1 00 0 4100

0 1 01 1 −1−1 −1 2

=

−b + 2 −b + 1 2b − 20 1 0

−b + 1 −b + 1 2b − 1

b=4100


Application

Example

Let A =

[2 −21 5

]. Find a square root of A. Namely, find B such that B2 = A.

Solution

P =

[−2 −11 1

]=⇒ P−1 =

[−1 −11 2

]and D =

[3 00 4

]=⇒ P−1AP = D ⇔ A = PDP−1. We just need to find a square root of D. Let

E =

[√3 00 2

]=⇒ E 2 =

[√3 00 2

] [√3 00 2

]=

[√3 ·√3 0

0 2 · 2

]=

[3 00 4

]= D

Now, consider B := PEP−1.

B2 =(PEP−1) (PEP−1) = PEP−1PEP−1 = PE 2P−1 = PDP−1 = A.

B := PEP−1 =

[−2+ 2

√3 −4+ 2

√3

2−√3 4−

√3


Remark

1 Since A = PDP−1,

An = PDnP−1 and A1n = PD

1n P−1

where (for m = n or m = 1/n).d1 0 0 · · · 00 d2 0 · · · 00 0 d3 · · · 0...

......

. . ....

0 0 0 · · · dn

m

=

dm

1 0 0 · · · 00 dm

2 0 · · · 00 0 dm

3 · · · 0...

......

. . ....

0 0 0 · · · dmn

However, to get the square root, we need d1, . . . , dn ≥ 0.

2 Recall another example A =

2 1 00 2 00 0 1

which has:

eigenvalue λ = 1, multiplicity 1, Basis of eigenvectors

001

;

eigenvalue λ = 2 multiplicity 2, Basis of eigenvectors

100

;

In this case,A is NOTdiagonalizable.(n linearlyindependenteigenvectors.)


Remark

TheoremEigenvectors with respect to distinct eigenvalues are linearly independent. Let Abe an n × n matrix and λ1, . . . , λk be its distinct eigenvalues. For each λj , pickANY eigenvector vj with respect to λj . Then, {v1, . . . , vk} are linearlyindependent.

Proof. (* You do not need to remember the proof).Suppose vj = a1v1 + · · ·+ aj−1vj−1, for a1, . . . , aj−1 are not all zero.λjvj = Avj = A (a1v1 + · · ·+ aj−1vj−1) = a1λ1v1 + · · ·+ aj−1λj−1vj−1=⇒ 0 = a1 (λ1 − λj) v1 + · · ·+ aj−1 (λj−1 − λj) vj−1.From here, one can prove that {v1, . . . , vk} are linearly independent by inductionon j and contradiction.

RemarkAs we know, the solution space of a homogeneous system= span {basic solutions}. Namely, the basic solutions of the eigenspace for a giveneigenvalue are linearly independent.


Remark

TheoremLet A be an n × n matrix and λ1, . . . , λk (k ≤ n) be its distinct eigenvalues. A isdiagonalizable iff for each λj , j = 1, . . . , k , its algebraic multiplicity equals itsgeometric multiplicity. In particular, if A has n distinct eigenvalues i.e., k = n, Ais diagonalizable.

Example

A =

2 1 00 2 00 0 1

. A is of ”the closest” form to a diagonal matrix.

Jordan canonical form (*)


Summary of Chpt. 8

Eigenvalues and eigenvectors for square matrix A

For nonzero vector v: Av = λvFind the eigenvalues and eigenvectors:

1 Find the characteristic polynomials p (λ) = det (A− λI )2 Find the roots of p (λ), i.e., eigenvalues p (λ) = 0 λ1, . . . , λn

3 For each λj , find the eigenvectors by solving the homogeneous system(A− λj I ) v = 0.

Diagonalize a matrix A is diagonalizable iff A has n linearly independenteigenvectors v1, . . . , vn with respect to eigenvalues λ1, . . . , λn (allowingrepetitions). Then,

P = [v1, . . . , vn] , D = diag {λ1, . . . , λn}

such thatAP = PD ⇐⇒ A = PDP−1

Compute An

Find B such that Bm = A


MATH-1030 Matrix Theory & Linear Algebra I · MATH-1030 Matrix Theory & Linear Algebra I LinJiu...

Documents

Transcript of MATH-1030 Matrix Theory & Linear Algebra I · MATH-1030 Matrix Theory & Linear Algebra I LinJiu...