Linear Algebra, Principal Component Analysis and

their Chemometrics Applications

Linear algebra is the language of chemometrics. One can not expect to truly understand most chemometric techniques without a basic understanding of linear algebra

Linear Algebra

Vector

A vector is a mathematical quantity that is completely described by its magnitude and direction

x1

y1

x

y

P

Vector

A vector is a mathematical quantity that is completely described by its magnitude and direction

x1

y1

x

y

P

P =

x1

y1

MATLAB Notation

Length of a Vector

x1

y1

x

y

PP = x1

2 + y12

x = [x1, x2, …, xn]

x M= ( xi2 ) 0.5

i=1

n

Normal Vector

u =xx

u =1

Normalized vector

Mean Centered Vector

x1

x2

xn

…x = mx = M

xi

i=1

n

nmcx =

x1 - mx

…

x2 - mx

xn - mx

0015100

x = mx = 1

-1-1040-1-1

mcx =

0+20+21+25+21+20+20+2

y =

2237322

=mx = 3

0

0.2

0.4

0.6

0.8

1

1.2

400 450 500 550 600Wavelength (nm)

Ab

so

rba

nc

e

00.20.40.60.8

11.21.4

400 450 500 550 600Wavelength (nm)

Ab

so

rba

nc

e

-0.4

-0.2

0

0.2

0.4

0.6

400 450 500 550 600

Wavelength (nm)

Ab

so

rba

nc

e

Mean centeredMean centered

?

The length of a mean centered vector is proportional to the standard deviation of its elements

y1

y2

yn

…y = y* =

y1 - my

…

y2 - my

yn - myy* ≈ syi

A set of p vectors [x1, x2, …, xp] with same dimension n is linearly independent if the expression:

ci xi = 0Mi=1

p

holds only when all p coefficients ci are zero

Linear Independent Vectors

x1

x2

x3c1x1

c2x2

A vector space spanned by a set of p linearly independent vectors (x1, x2, …, xp) with the same dimension n is the set of all vectors that are linear combinations of the p vectors that span the space

Vector Space

Basis setA set of n vectors of dimension n which are linearly independent is called a basis of an n-dimensional vector space. There can be several bases of the same vector space

A coordinate space can be thought of as being constructed from n basis vectors of unit length which originate from a common point and which are mutually perpendicular

Coordinate Space

Q = P = x1

y1

x1

y1

x

y

P

x1

y1

Q

Vector Multiplication by a Scalar

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

400 450 500 550

Wavelength (nm)

Ab

so

rba

nc

e

x = 1.19 y = 2.38

x

y

y = 2 x

Addition of Vectorsx1

x2

xn

…

y1

y2

yn

…x + y = + =

x1 + y1

…

x2 + y2

xn + yn

x + y

x1

x2

y1

y2y

x

x1 + y1

x2 + y2

x1cx

…ax + ay = + =

x2cx

xncx

y1cy

…

y2cy

yncy

x1cx + y1cy

…

x2cx + y2cy

xncx + yncy

Component 1 Component 2 mixture

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

400 420 440 460 480 500 520 540 560

Wavelength (nm)

Ab

so

rba

nc

e

axay

ax + ay

Subtraction of Vectorsx1

x2

xn

…

y1

y2

yn

…x - y = - =

x1 - y1

…

x2 - y2

xn - yn

x - y

x1

x2

y1

y2y

x

Inner Product (Dot Product)x1

x2

xn

…x . x = xTx = [x1 x2 … xn] = x12 + x2

2 + … +xn2

= x2

x . y = xTy = x y cos

The cosine of the angle of two vectors is equal to the dot product between the normalized vectors:

x . y

x ycos =

y

xx . y = x y

y

xx . y = - x y

y

x x . y = 0

Two vectors x and y are orthogonal when their scalar product is zero

x . y = 0 and x y = 1=

Two vectors x and y are orthonormal