1

February 24 Matrices3.2 Matrices; Row reduction

Standard form of a set of linear equations:

Chapter 3 Linear Algebra

35

12

252

zx

zyx

zyx

Matrix of coefficients:

3501

1211

2152

A

501

211

152

M

Augmented matrix:

Elementary row operations:1) Row switching.2) Row multiplication by a nonzero number.3) Adding a multiple of a row to another row.

These operations are reversible.

2

Solving a set of linear equations by row reduction:

*100

*010

*001

*100

*010

***1

*100

**10

***1

**00

***0

****

***0

***0

****

****

****

****possible as close As

:Method

Row reduced matrix

1

1

2

1100

1010

2001

1100

1010

1211

1100

0110

1211

2200

0110

1211

2310

0110

1211

2310

0330

1211

3501

2152

1211

3501

1211

2152

:Example

2)3((2)(1)

)3((2)(3)/2(2)(3)

(2)/3)1((3)2)1((2)

(2)(1)

z

y

x

3

Possible cases of the solution of a set of linear equation:Solving m equations with n unknowns:1)If Rank M < Rank A, the equations are inconsistent and there is no solution.

2)If Rank M = Rank A = n, there is one solution.

3)If Rank M = Rank A = R<n, then R unknowns can be expressed by the remainingn−R unknowns.

*000

***0

****

**00

***0

****

0000

***0

****

Rank of a matrix:The number of the nonzero rows in the row reduced matrix. It is the maximal number of linearly independent row (or column) vectors of the matrix.Rank of A = Rank of AT (where (AT)ij= (A)ji, transpose matrix).

4

Read: Chapter 3: 1-2Homework: 3.2.4,8,10,12,15;Due: March 7

5

3.3 Determinates; Cramer’s rule

Determinate of an n×n matrix:

nnn

n

nnn

n

aa

aaa

aa

aaa

**

****

*

|A|detA ,

**

****

*

A

1

11211

1

11211

Minor:

nnn

n

ij

nnn

n

aa

aaa

M

aa

aaa

**

****

*

,

**

****

*

A

1

11211

1

11211

i

j

ijji

ij MC )1(Cofactor:

Determinate of a 1×1 matrix: aa

Definition of the determinate of an n×n matrix: j

jjj

jjj MaCa 11

111 )1(A

March 3 Determinants

6

Triple product of 3 vectors:

vectors.3 by the formed ipedparallelop theof volume)signed(

)()()(

321

321

321

CCC

BBB

AAA

ACBBACCBA

Useful properties of determinants:

1)A common factor in a row (column) may be factored out.2)Interchanging two rows (columns) changes the sign of the determinant.3)A multiple of one row (column) can be added to another row (column) without changing the determinant.4)The determinate is zero if two rows (columns) are identical or proportional.

Example p91.2.

Equivalent methods:1)A determinate can be expanded by any row or any column:2)|A|=|AT|.

Example p90.1

i

ijijj

ijij CaCaA

7

Theorem:For homogeneous linear equations, the determinant of the coefficient matrix must be zero for a nontrivial solution to exist.

.0 if ,0

0

0

0

0

0

0

333231

232221

131211

333231

232221

131211

3332

2322

1312

1

333232131

323222121

313212111

aaa

aaa

aaa

aaa

aaa

aaa

aa

aa

aa

x

xaxaxa

xaxaxa

xaxaxa

Cramer’s rule in solving a set of linear equations:

. and for solutionssimilar and , 32

333231

232221

131211

33323

23222

13121

1

33323

23222

13121

3332333232131

2322323222121

1312313212111

3332131

2322121

1312111

333231

232221

131211

1

3333232131

2323222121

1313212111

xx

aaa

aaa

aaa

aac

aac

aac

x

aac

aac

aac

aaxaxaxa

aaxaxaxa

aaxaxaxa

aaxa

aaxa

aaxa

aaa

aaa

aaa

x

cxaxaxa

cxaxaxa

cxaxaxa

8

Read: Chapter 3: 3Homework: 3.3.1,10,15,17(No computer work is needed.)Due: March 21

9

March 7 Vectors

3.4 VectorsVector: A quantity that has both a magnitude and a direction.Geometrical representation of a vector: An arrow with a length and a direction .

Addition and subtraction:

Vector addition is commutative and associative.

Algebraic representation of a vector:

kjiA

kjiBA

kjiA

ˆˆˆ

ˆ)(ˆ)(ˆ)(

ˆˆˆ),,(

zyx

zzyyxx

zyxzyx

cAcAcAc

BABABA

AAAAAA

Magnitude of a vector:

222zyx AAAA A

Note: A is a vector and A is its length.They should be distinguished.

10

Vector or cross product:

rule. handright by the determined

direction its with,sinABBA

B

A

A×B

Cross product in determinant form:

(Proof)

ˆˆˆ

zyx

zyx

BBB

AAA

kji

BA

Scalar or dot product:(Proof) cos zzyyxx BABABAAB BA

B

A

.00

.0//

BABA

BABA

zzyyxx

z

z

y

y

x

x

BABABA

B

A

B

A

B

AParallel and perpendicular vectors:

Examples p102.3, p105.4.Problems 4.5,26.

Relations between the basis vectors: .ˆˆˆ ,0ˆˆ ,0ˆˆ ,1ˆˆ kjiiijiii

11

Read: Chapter 3: 4Homework: 3.4.5,7,12,18,26.Due: March 21

12

3.5 Lines and planes

Equations for a straight line:

c

zz

b

yy

a

xx

ctzz

btyy

atxx

t

000

0

0

0

0

Arr

y

z

A

r−r0

),,( 000 zyx

),,( zyx

x

Equations for a plane:

)(

0)()()(

0)(

000

000

0

czbyaxdczbyax

zzcyybxxa

rrN

N

r−r0

),,( 000 zyx

),,( zyx

Examples p109.1, 2.

March 17 Lines and planes

13

Distance from a point to a plane:

n

P

R

Q

n

N PQ

NPQPQPR cos

Distance from a point to a line:

uA

PQA

PQPQPR sin

Example p110.3.

A

P

R

Q

Example p110.4.

Distance between two skew lines:

BA

BAn

PQPQFG

A

B

P

Q

n

F

GExample p110.5,6.Problems 5.12,18,42.

Let FG be the shortest distance, then it must be perpendicular to both lines (proof).

14

Read: Chapter 3: 5Homework: 3.5.7,12,18,20,26,32,37,42.Due: March 28

15

3.6 Matrix operations

Matrix equation:

.5,4,3,254

32

dcba

dc

ba

March 19,21 Matrix operations

Multiplication by a number:

kdkc

kbka

dc

bak

Matrix addition:

hdgc

fbea

hg

fe

dc

ba

. whiledkc

bka

dc

kbka

dc

bak

16

More about matrix multiplication:

1) The product is associative: A(BC)=(AB)C2) The product is distributive: A(B+C)=AB+AC3) In general the product is not commutative: ABBA.

[A,B]=AB−BA is called the commutator.

Unit matrix:

10

01I

00

000

Zero matrix:

Product theorem: BdetAdet )ABdet(

Matrix multiplication:Note:1)The element on the ith row and jth column of AB is equal to the dot product between the ith row of A and the jth row of B.2)The number of columns in A is equal to the number of rows in B.

Example:

.AB k

kjikij BA

dhcfdgce

bhafbgae

hg

fe

dc

ba

17

Matrix inversion:1) M−1 is the inverse of M if MM−1= M−1M =I=1.2) Only square matrix can be inversed.3) det (M M−1) = det(M) det( M−1) = det(I)=1, so det (M)0 is necessary for M to be

invertible.

Calculating the inverse matrix:

.Mdet

CMMdetIMC

(p3.8) . if ,0

. if M,detCC)(MC

:Proof

T1T

TT

ji

jiMM jk

kikkj

kikij

.Mdet

CM

T1

Example p120.3.

Solving a set of linear equations by matrix operations:

k. Mr k r M

3

1

2

501

211

152

35

12

252

1

z

y

x

zx

zyx

zyx

18

.Mdet

CCC

Mdet

CCC

k Mdet

Ck Mr kr M

333231

232221

131211

33323

23222

13121

3132121113T132

T121

T11

T1

3

2

1

333231

232221

131211

aaa

aaa

aaa

aak

aak

aak

kkkkkkx

k

k

k

z

y

x

aaa

aaa

aaa

Equivalence between r=M-1k and the Cramer’s rule:

Three equivalent ways of solving a set of linear equations:1)Row reduction2)Cramer’s rule3)r=M-1k

Cramer’s rule is an actual realization of r=M-1k.

19

Gauss-Jordan method of matrix inversion:

Let (MLp MLp-1

MLp-2 … ML2

ML1 ) M = MLM = I be the result of a series of elementary

row operations on M, then (MLp MLp-1

MLp-2 … ML2

ML1 ) I = MLI = M−1.

That is,

.10

01

2/12/3

12

43

21 :Test

.2/12/3

12

10

01

13

12

20

01

13

01

20

21

10

01

43

21 :Example

)2/1()2(

)2()1(3)1()2(

Equivalence between row reduction and r=M-1k:

.)rI,()kM,(M)kM,)(MMMM(M

IM M )MMMMM(M

1LLLLL

1LLLLL

122p1pp

122p1pp

Row reduction is to decompose the M−1 in r=M−1k into many steps.

.MIIMM 11

20

Rotation matrices: Rotation of vectors

.)cos()sin(

)sin()cos(

cossin

sincos

cossin

sincosrMM

.cossin

sincosMr with MRor

,cossin

sincos

2121

2121

11

11

22

2212

y

x

y

x

y

x

Y

X

y

),( yx

x

),( YX

Functions of matrices:Expansion or power series expansion is implied.Examples:

2

3322

222

AA1A1

1!3

A

!2

AA1)Aexp(

BBAABAB)(A

kkkk

21

Read: Chapter 3: 6Homework: 3.6.9,13,15,18,21Due: March 28

22

3.7 Linear combinations, linear functions, linear operators

Linear combination: aA + bB

rrFrAr

rrrrrr

bf

faaffff

)( ,)(

:Examples

)( )( ),()()( 2121

March 24, 26 Linear operators

Linear function f (r):

Linear operator O:

fdx

d

gf

fOaafOgOfOgfO

:Example

etc. matrices, vectors,functions, numbers, becan and Here

)( )( ),()()(

Example p125.1; Problem 7.15

Linear transformation:

(Mr).r)M( ,MrMr )rM(r r. MRor 2121 kky

x

dc

ba

Y

X

The matrix M is a linear operator representing a linear transformation.

23

Orthogonal transformation:An orthogonal transformation preserves the length of a vector.Orthogonal matrix :The matrix for an orthogonal transformation is an orthogonal matrix.

Theorem: M is an orthogonal matrix if and only if MT = M−1.

.MMIMMrrMrMrrr(Mr)(Mr)

:Proof1TTTTTTT

Theorem: det M=1 if M is orthogonal.

1.det(M)1]det(M)[)Mdet(M)det(

)MMdet()MMdet(Idet

:Proof

2T

T1

24

2×2 orthogonal matrix:

.)2/sin(

)2/cos(

)2/sin(

)2/cos(

cossin

sincoscially Espe

line. 2

therespect to withreflectiona is M .)sin(

)cos(

sin

cos

cossin

sincos

.cossin

sincos 1det M or ,for

cossin

sincos )2

rotaion.a is M .)sin(

)cos(

sin

cos

cossin

sincos

1.det M or ,for cossin

sincos )1

0)sin(0cossincossin

cossin

sincos

0

1

1

1

of sign thechange

22

22

r

r

r

r

r

r

r

r

dc

ba

r

r

r

r

dc

ba

dc

ba

bdac

dc

ba

db

ca

dc

ba

Conclusion: A 2×2 orthogonal matrix corresponds to either a rotation (with det M=1)or a reflection (with det M=−1).

25

Two-dimensional rotation:

y

x

y

x

y

x

Y

X

cossin

sincos

'

'

:)basis of (Change

:axes theRotating

cossin

sincos

:)ation transform(Active

: vector theRotating

y

),( yx

x

),( YX

y),( yx

x

x'

y'

Two-dimensional reflection:

.2

tanor line, 2

therespect towith

reflection a is cossin

sincos

'

'

xy

y

x

y

x

y

),( yx

x

)','( yx

/2

Example p128.3.

3

4cos

3

4sin

3

4sin

3

4cos

2

1

2

32

3

2

1

A that Note

26

Read: Chapter 3: 7Homework: 3.7.9,15,22,26.Due: April 4

27

3.8 Linear dependence and independence

Linear dependence of vectors: A set of vectors are linearly dependent if some linear combination of them is zero, with not all the coefficients equal to zero.

CBACBAa

c

a

bacba 0,0

March 28 Linear dependence and independence

1. If a set of vectors are linearly dependent, then at least one of the vectors can be written as a linear combination of others.

2. If a set of vectors are linearly dependent, then at least one row in the row-reduced matrix of these vectors equals to zero. The rank of the matrix is then less than the number of rows.

.00,0 CBACBAa

c

a

bacba

Example: Any three vectors in the x-y plane are linearly dependent. E.g. (1,2), (3,4), (5,6).

Linear independence of vectors: A set of vectors are linearly independent if any linear combination of them is not zero, with not all the coefficients equal to zero.

28

Linear dependence of functions:A set of functions are linearly dependent if some linear combination of them is always zero, with not all the coefficients equal to zero.

Examples:

dependent.linearly are 1 and cos ,sin

t.independenlinearly are and ,1

t.independenlinearly are cos and sin

22

2

xx

xx

xx

Theorem: If the Wronskian of a set of functions

then the functions are linearly independent .

,0

)()()(

)(')(')('

)()()(

)1()1(2

)1(1

21

21

xfxfxf

xfxfxf

xfxfxf

W

nn

nn

n

n

29

.0

)()()(

)(')(')('

)()()(

00

)()()(

)(')(')('

)()()(

)()(0

)(')('0

)()(0

.0)( , ,0)('Now .0)(

and 0) be tosupposed(exit therethendependent,linearly are )(,),(),( If

:Proof

)()(2

)(1

21

21

)1()1(2

)1(1

21

21

)1()1(2

2

2

1

1

)(

11

121

xfxfxf

xfxfxf

xfxfxf

WW

xfxfxf

xfxfxf

xfxfxf

xfxf

xfxf

xfxf

k

xfkxfkxfk

kkxfxfxf

nn

nn

n

n

nn

nn

n

n

nn

n

n

n

n

j

njj

n

jjj

n

jjj

in

Examples p133.1,2.

Note: W=0 does not always imply the functions are linearly dependent. E.g., x2 and x|x| about x=0. However, when the functions are analytic (infinitely differentiable), which we meet often, W=0 implies linear dependence.

30

Homogeneous equations:

.

0000

0**0

0***

or

0*00

00*0

000*

0***

0***

0***reduction row

1. Homogeneous equations always have the trivial solution (all unknowns=0).2. If Rank M=Number of unknowns, the trivial solution is the only solution.3. If Rank M< Number of unknowns, there are infinitely many solutions.

Theorem:A set of n homogeneous equations with n unknowns has nontrivial solutions if and only if the determinant of the coefficients is zero.

Proof:1. det M ≠0 r =M-10=0. 2. Only trivial solution exists Columns of M are linearly independent det M ≠0.

Examples p135.4.

31

Read: Chapter 3: 8Homework: 3.8.7,10,13,17,24.Due: April 4

32

3.9 Special matrices and formulas

March 31 Special matrices

Transpose matrix AT of A:

Complex conjugate matrix A* of A:

Adjoint (transpose conjugate) matrix A+ of A:

Inverse matrix A-1 of A: A-1A= AA-1=1.

Symmetric matrix: A=AT (A is real)

Orthogonal matrix: A−1 =AT (A is real)

Hermitian matrix: A =A+

Unitary matrix: A-1 =A+

Normal matrix: AA+=A+A, or [A,A+]=0.

** AA ijij

**T**T AAA ,)A()A(A jijiij

jiij AAT

33

Associative law for matrix multiplication: A(BC)=(AB)C

ijl

ljillk

ljklikk

kjikij CAB CABCBABCABCA :Proof,

Transpose of a product: (AB)T=BTAT

ijk

kjikk

ikkjk

kijkjiijTTTTTTT ABABBABAABAB :Proof

Corollary: (ABC)T=CTBTAT

Index notation for matrix multiplication:

Kronecker symbol:

k

kjikij BAAB

. if ,0

. if ,1

ji

jiij

Exercises on index notations:

34

Trace of a matrix: i

iiAATr

Trace of a product: Tr(AB)=Tr(BA)

Tr(BA)ABBA )AB(Tr(AB) :Proof,,

ji

ijjijiji

iji

ii

Corollary: Tr(ABC)=Tr(BCA)=Tr(CAB)

Inverse of a product: (AB)-1=B-1A-1

.AB(AB)1AA)AA(BB)A(B (AB) :Proof -1-1-1-1-1-1-1-1

Corollary: (ABC)-1=C-1B-1A-1

35

Read: Chapter 3: 9Homework: 3.9.2,4,5,23,24.Due: April 11

36

3.10 Linear vector spaces

April 2 Linear vector spaces

n-dimensional vectors:

Linear vector space: Several vectors and the linear combinations of them form a space.

Subspace: A plane is a subspace of 3-dimensional space.

Span: A set of vectors spans the vector space if any vector in the space can be written as a linear combination of the spanning set.

Basis: A set of linearly independent vectors that spans a vector space.

Dimension of a vector space: The number of basis vectors that span the vector space.

),,,( 21 nAAA A

Examples p143.1; p144.2.

37

Inner product of two n-dimensional vectors:

n

iii BA

1

BA

Length of an n-dimensional vector:

n

iiAA

1

2AA

Two n-dimensional vectors are orthogonal if .01

n

iii BABA

n

ii

n

ii

n

iii BABAAB

1

2

1

2

1

,BASchwarz inequality:

.//only when holdssign equal The

.10)(1)()(21Then

. toothogonal ofcomponent theiswhich ,)(Construct

.1 prove toneed then we, prove To

. and along rsunit vecto are which , ,Let 0. 0, Suppose

:Proof

2122

212

212

21

212211

21

21

BA

eeeeeeeeCC

eeeeeeC

eeBA

BAB

eA

e

C

ABBA

BA

38

Orthonormal basis: A set of vectors form an orthonormal basis if they are mutually orthogonal and each vector is normalized.

Gram-Schmidt orthonormalization:

Starting from n linearly independent vectors we can construct an orthonormal basis set

,21 n,,, vv v .21 n,,, ww w

.

,

,

,

1

1

1

1

2231133

22311333

1122

11222

1

11

k

iiikk

k

iiikk

k

wwvv

wwvvw

wwvwwvv

wwvwwvvw

wwvv

wwvvw

v

vw

Example p146.4.

39

Complex Euclidean space:

Inner product:

n

iii BA

1

*BA

Length:

n

iii AAA

1

*AA

Orthogonal vectors: 01

*

n

iii BABA

n

iii

n

iii

n

iii BBAABA

1

*

1

*

1

* Schwarz inequality:

Example p146.5.

n

iii BA

AAA

A

1

*

*2

*12

1

). ( ,

BABA

AAAA

Bra-ket notation of vectors:

40

Read: Chapter 3: 10Homework: 3.10.1,10.Due: April 11

41

3.11 Eigenvalues and eigenvectors; Diagonalizing matrices

April 4, 7 Eigenvalues and eigenvectors

Eigenvalues and eigenvectors:

For a matrix M, if there is a nonzero vector r and a scalar such that then r is called an eigenvector of M, and is called the corresponding eigenvalue.

M only changes the “length” of its eigenvector r by a factor of eigenvalue , without affecting its “direction”.

,M rr

For nontrivial solutions of this homogeneous equation, we need

This is called the secular equation, or characteristic equation.

.0)Mdet(

21

22221

11211

nnnn

n

n

MMM

MMM

MMM

.0)M(M rrr

42

Example: Calculate the eigenvalues and eigenvectors of .52

34M

1

100

52

34

2

30320

52

34

.7,2052

34)Mdet(

22

2

11

1

21

r

r

yxy

x

yxy

x

Example: Calculate the eigenvalues and eigenvectors of .22

25M

1

2020

22

25

2

1020

22

25

.6,1022

25)Mdet(

22

2

11

1

21

r

r

yxy

x

yxy

x

Let operator M actively change (rotate, stretch, etc.) a vector. The matrix representation of the operator depends on the choice of basis vectors:

Let matrix C change the basis (coordinate transformation):

Question:

43

Similarity transformation:

'.r'M'R :basis new In the Mr;R :basis old In the

.CMCM'CMCM'CMrCRR'

CrM'r'M'R' 1

M′ =CMC-1 is called a similar transformation of M.M′ and M are called similar matrices. They are the same operator represented in different bases that are related by the transformation matrix C.That is: If r′ =Cr, then M′ = CMC-1.Theorem: A similarity transformation does not change the determinate or trace of a matrix:

MTr 'MTr

Mdet'Mdet

CR.'R ;Cr r'

?)CM,('M f xx'

yy' r (r')R (R')

M (M')

r r'C

R'

M'M

CR

44

Diagonalization of a matrix:

.DMCCCDMC

thens),eigenvalue of (diagnal 0

0D rs),eigenvecto of (columns CLet

0

0M

then,M and M suppose M,matrix 22 aFor

1

2

1

21

21

2

1

21

21

2211

2211

21

21

222111

yy

xx

yy

xx

yy

xx

yy

xx

rrrr

Theorem: A matrix M may be diagonalized by a similarity transformation C-1MC=D , where C consists of the column eigenvectors of M, and the diagonalized matrix D consists of the corresponding eigenvalues. That is, the diagonization equation C -1MC=D just summarizes the eigenvalues and eigenvectors of M.

.

000

000

000

D ,C

and ,M where,DMCCby diagonized becan Mmatrix An

n

2

1

21

1

n

iiinn

rrr

rr

45

. , I :examples More

.60

01

12

21

5

1

22

25

12

21

5

1DMCC

.60

01D ,

12

21

5

1C ,

12

21

5

1C

.1

2

5

1 ,6;

2

1

5

1 ,1 .

22

25M :2 Example

.70

02

12

13

52

34

32

11

5

1DMCC

.70

02D ,

32

11

5

1C ,

12

13C

.1

1 ,7;

2

3 ,2 .

52

34M :1 Example

1

1

2211

1

1

2211

EDωL

rr

rr

46

More about the diagonalization of a matrix C-1MC=D: (2×2 matrix as an example):

1.D describes in the (x', y') system the same operation as M describes in the (x, y) system.

2.The new x', y' axes are along the eigenvectors of M.

3.The operation is more clear in the new system:

1

1

21

211-

0

1

0

1CC

0

1' :Proof

y

x

yy

xx

y

x

y

xxe

'

'

'

'

0

0

'

'D

2

1

2

1

y

x

y

x

y

x

47

Diagonalization of Hermitian matrices:1.The eigenvalues of a Hermitian matrix are always real.2.The eigenvectors corresponding to different eigenvalues of a Hermitian matrix are orthogonal.

3.{A matrix has real eigenvalues and can be diagonalized by a unitary similarity transformation}if and only if {it is Hermitian}.

.0 then If

. then If0)(

HHHH

HH

:Proof*

*

* jiji

jiij

ijijij

ijijiiii

ji

j

ii

.CC1CC

.D Dand D,HCC then, , that so Dand C matricesConstruct

. , then, H,H HSuppose )2

.MUDUMDDUMU)(UMUMU)U(DMUU

.D Ddiagonal and U U withD,MU USuppose 1)

:Proof

1**

*1

**

1-1111

*11

kkijk

kkikj

kijikjk

ijiijjiij

kijikjkijiiiii

CCCCrr

DrC

rr

rrrr

Example p155.2.

48

Corollary: {A matrix has real eigenvalues and can be diagonalized by an orthogonal similarity transformation} if and only if {it is symmetric}.

.CC1CC

.D Dand D,SCC then, , that so Dand C matricesConstruct

. , then,S ,SS Suppose )2

.MODOMDDOMO)(OMOMO)(ODMOO

.D Ddiagonal and OO withD,MOO Suppose 1) :Proof

1TTT

*1

*T

1-TTT1T1TTT11

*T11

kkijk

kkikj

kijikjk

ijiijjiij

kijikjkijiiiii

CCCCrr

DrC

rr

rrrr

.CC

MM that Notice

.60

01

12

21

5

1

22

25

12

21

5

1DMCC

.60

01D ,

12

21

5

1C ,

12

21

5

1C

.1

2

5

1 ,6;

2

1

5

1 ,1 .

22

25M :Example

T1

*T

1

1

2211

ii

rr

49

Read: Chapter 3: 11Homework: 3.11.3,13,14,19,32,33,42.Due: April 18