Lin Agebra Rev

7/29/2019 Lin Agebra Rev

1/18

Outline

Review of Linear Algebra

vectors and matrices

products and norms

vector spaces and linear transformations

eigenvalues and eigenvectors

Introduction to Matlab


2/18

Why Linear Algebra?

For each data point, we will represent a setof features as feature vector

[length, weight, color,]

Linear models are simple andcomputationally feasible

Collected data will be represented as

collection of (feature) vectors [l , w , c , ] [l , w , c ,] [l , w , c ,]

1 1 1 2 2 2 3 3 3


3/18

Vectors

n-dimensional row vector [[[[ ]]]]nxxxx 21====

====

n

T

x

xxx2

1

Transpose of row vector is column vector

Vectorproduct (or inneror dotproduct)

====

====++++++++++++============ki

iinnT yxyxyxyxyxyxyx

1

2211,

[[[[ ]]]]21 x,x

1X

2X zy

y-z


4/18

More on Vectors

yxyx

T

====cos Angle between vectors xand y

Euclidian normor length ============ niixxxx

1

2

,

If |x|=1 we say xis normalizedor unitlength

Thus inner product captures direction relationship

between xand y

0cos ====

x

y

0====yxT

yxx orthogonal to y

1cos ====

0||||>>>>====

yxyx

T

x y

1cos ====

0||||


5/18

More on Vectors

Euclidian distance between vectors x and y

(((( ))))==== ==== niii yxyx

1

2

Vectors x and y are orthonormal if they areorthogonal and |x|=|y|=1

y

x x-y


6/18

Linear Dependence and Independence

Vectors are linearly dependentif there exist constants s.t.

1.

2. at least one

nxxx ,,, 21

n,,, 21

0xxx nn2211 ====++++++++++++

Vectors are linearly independentif

nxxx ,,, 21

00xxx n1nn2211 ================++++++++++++

0i


7/18

Vector Spaces and Basis

The set of all n-dimensional vectors iscalled a vector space V

A set of vectors are called abasis for vector space if any vin Vcan bewritten as

nuuu ,,, 21

nnuuuv ++++++++++++==== 2211

nuuu ,,, 21 are independent implies theyform a basis, and vice versa

nuuu ,,, 21

give an orthonormal basis if1.

2.

iui ====1

jiuuji


8/18

Matrices

====

nmnn

m

m

xxx

xxxxxx

A

21

22221

11211

====

nmm2m1

2n2212

1n1211

T

xxx

xxx xxxA

T n by m matrix A and its m by n transpose A


9/18

Matrix Product

Cc

bb

bbbbbb

aaaa

aaaaAB ij

dm1d

m331

m221

m111

nd3n2n1n

d1131211

====

====

====

BAA

# of columns of A = # of rows of B

even if defined, in general

is row iof Ais columnjof B

j

i b,acij ====

iajb


10/18

Matrices

Rank of a matrix is the number of linearlyindependent rows (or equivalently columns)

A square matrix is non-singular if its rankequal to the number of rows. If its rank isless than number of rows it is singular.

T Matrix A is symmetric if A=A

4685

634984725921

Identity matrix

====

10000

010001

IAI=IA=A


11/18

Matrices

Matrix A is positive definite if

0,

, >>>>====ji

jijiT xxAAxx

Matrix A is positive semi-definite if

0,

, ====ji

jijiT xxAAxx

Trace of a square matrix A is sum on the

elements on the diagonal[[[[ ]]]]

====

====

n

iiiaAtr

1


12/18

Matrices

Inverse of a square matrix A is matrix As.t. AA = I

-1

-1

If A is singular or not square, inverse does

not exist. Pseudo-inverse A is definedwhenever A A is not singular (it is square)

A =(A A ) A

AA =(A A) AA=I

T

T

T

-1

-1

T

T


13/18

Matrices

Determinant of n by n matrix A is

Where obtained from A by removing the ith

row and kth column

(((( )))) (((( )))) (((( ))))====++++

====

n

kikik

ik

AaA 1 det1det

ikA

Absolute value of determinant gives the volume ofparallelepiped spanned by the matrix rows

{{ }}nn2

21

1 a...aa ++++++++++++

[[[[ ]]]] i1,0i


14/18

Linear Transformations

A linear transformation fromvector space Vto vector space Uis a mapping which can be

represented by a matrix M: u= Mv

U

V

M

v

Mv

If U and V have the same

dimension, Mis a square matrix

In pattern recognition, often U

has smaller dimensionality thanV, i.e. transformation Mis usedto reduce the number of

features.

M v u=


15/18

Eigenvectors and Eigenvalues

Given n by n matrix A, and nonzero vector x.

Suppose there is which satisfies Ax= x

xis called an eigenvector of A

is called an eigenvalue of A

Note: A0=0 for any , not interesting

Linear transformation A maps an eigenvector vin a

simple way. Magnitude changes by , direction

If > 0

Av

v

Av

v

If < 0


16/18

Eigenvectors and Eigenvalues

If A is real and symmetric, then alleigenvalues are real (not complex)

If A is non singular, all eigenvalues are nonzero

If A is positive definite, all eigenvalues arepositive


17/18

MATLAB


18/18

Starting matlab

xterm -fn 12X24 matlab

Basic Navigation quit

more

help general Scalars, variables, basic arithmetic

Clear

+ - * / ^

help arith

Relational operators ==,&,|,~,xor

help relop

Lists, vectors, matrices

A=[2 3;4 5]

A

Matrix and vector operations

find(A>3), colon operator

* / ^ .* ./ .^

eye(n),norm(A),det(A),eig(A)

max,min,std

help matfun

Elementary functions

help elfun Data types

double

Char

Programming in Matlab

.m files

scripts

function y=square(x)

help lang

Flow control

if i== 1else end, if else if end

for i=1:0.5:2 end while i == 1 end

Return

help lang

Graphics

help graphics help graph3d

File I/O

load,save

fopen, fclose, fprintf, fscanf

Lin Agebra Rev

Documents

Transcript of Lin Agebra Rev