Linear Algebra & Matrices

MfD 2004María Asunción Fernández Seara

mfseara@fil.ion.ucl.ac.ukJanuary 21st, 2004

“The beginnings of matrices and determinants go back to the second century BC although traces can be seen back to

the fourth century BC”

• Scalar: variable described by a single number (magnitude)– Temperature = 20 °C– Density = 1 g.cm-3

– Image intensity (pixel value) = 2546 a. u.

Scalars, Vectors and Matrices

2

1

1

b

e

n

v

vv

Column vector

Row vector

943d

• Vector: variable described by magnitude and direction

476

145

321

A

Square (3 x 3) Rectangular (3 x 2) d i j : ith row, jth column

83

72

41

C 3

2• Matrix: rectangular array of scalars

333231

232221

131211

ddd

ddd

ddd

D

Vector Operations

• Transpose operator

2

1

1

b 211Tb 943d

9

4

3Td

column → row row → column

476

145

321

A

413

742

651TA

332313

322212

312111

321

3

2

1

yxyxyx

yxyxyx

yxyxyx

yyy

x

x

xTxy

• Outer product = matrix

Vector Operations

• Inner product = scalar

ii

iT yxyxyxyx

y

y

y

xxx

3

1332211

3

2

1

321yx

|| x || = (x12+ x2

2 )1/2

|| x || = (x12+ x2

2 + x32 )1/2

Inner product of a vector with itself = (vector length)2

xT x =x12+ x2

2 +x32 = (|| x ||)2 x1

x2 ||x||

Right-angle triangle

Pythagoras’ theorem

• Length of a vector

i

n

ii

n

nT yx

y

y

y

xxx

1

2

1

21 ...

yx

Vector Operations

• Angle between two vectors

cos

cos

sinsincoscos)cos(cos

cossin

cossin

2211

12

12

yx

yx

yx

xyxy

xx

xx

yy

yy

T

T

yx

yx

Orthogonal vectors: xT y = 0

x

y

=/2

||x||||y||

y2

y1

• Addition (matrix of same size)

– Commutative: A+B=B+A

– Associative: (A+B)+C=A+(B+C)

Matrix Operations

33

33

11

11

22

22BA

• Multiplication (number of columns in first matrix = number of rows in second)

– Associative: (A B) C = A (B C)

– Distributive: A (B+C) = A B + A C

– Not commutative: AB BA !!!

– (A B)T = BT AT

Matrix Operations

32122

1450

362514968574

332211938271

39

28

17

654

321BAC

2 x 3 3 x 2 2 x 2

C = A B

(m x p) = (m x n) (n x p)

Cij = inner product between ith row in A and jth column in B

Some Definitions …

• Identity Matrix

• Diagonal Matrix

• Symmetric Matrix

100

010

001

I I A = A I = A

700

050

003

D

720

251

013

B B = BT bij = bji

Matrix Inverse

A-1 A = A-1 A = I

IDD

100

010

001

7

100

05

10

003

1

700

050

0031

Properties

A-1 only exists if A is square (n x n)

If A-1 exists then A is non-singular (invertible)

(A B) -1 = B-1 A-1; B-1 A-1 A B = B-1 B = I

(AT) -1 = (A-1)T; (A-1)T AT = (A A-1)T = I

Matrix Determinant

dc

baA

ihg

fed

cba

A

det (A) = ad - bc

hg

edc

ig

fdb

ih

fea detdetdet)det(A

Properties

Determinants are defined only for square matrices

If det(A) = 0, A is singular, A-1 does not exist

If det(A) 0, A is non-singular, A-1 exists

A (n x n) = [a ij ] jj

n

jj Ma 1

)1(

11 )1()det(

A

http://mathworld.wolfram.com/Determinant.html

Matrix Inverse - Calculations

dc

baA IAA

10

01

43

211

dc

ba

xx

xx

43

211

xx

xxA

ac

bd

)det(

11

AA

A general matrix can be inverted using methods such as the Gauss-Jordan elimination, Gauss elimination or LU decomposition

1

0

0

1

43

43

21

21

dxbx

cxax

dxbx

cxax

bAadbc

bxdx

a

cxb

a

cxx

)det(

1

)(0

1

1

222

21

Another Way of Looking at Matrices…

• Matrix: linear transformation between two vector spaces

A x = y

A-1 y = x

yxA

10

5

2

2

42

21

yzA

10

5

5.2

0

42

21

x

y AA-1

z

A

det(A) = 1 x 4 – 2 x 2 = 0

In this case, A is singular, A-1 does not exist

Other matrix definitions

Linearly independent Linearly dependent

• Orthonormal matrix

A = [q1 | q2 | … qj …| qn]

qjT qq = 0 (if j k) and qj

T qj = 1

AT A = I

A-1 = AT

• Matrix rank: number of linearly independent columns or rows

if rank of A (n x n) = n, then A is non-singular

• Orthogonal matrix

A = [q1 | q2 | … qj …| qn]

qjT qq = 0 (if j k) and qj

T qj = djj

AT A = D