2. Matrix Algebra - NCU

Linear Algebra 2. Matrix Algebra CSIE NCU 1

2. Matrix Algebra2.1 Matrix operations ……………………… 22.2 The inverse of a matrix ……………….. 92.3 Characterizations of invertible matrices . 172.4 Partitioned matrices …………………….. 202.5 Matrix factorization ………………………. 302.7 Applications to computer graphics …….. 38


PurposeDiscuss the properties of matrix sum and multiplication.

2.1 Matrix operations Notation

A is an mxn matrix,

where aij is the i th row and j th column entry of A.If A is represented by a1, a2, …, an columns,then A = [a1 a2 … an].

Sums and scalar multiples

Two matrices are equal if they have the same size (the same number of rows and columns) and the same entry.

The sum A + B = [aij + bij] if A and B have the same size.


The scalar multiple r A =[raij ].

- A = (-1) A = [- aij ].

A zero matrix if its entries are all zeros.

Theorem 1 ( Addition and scalar multiplication )Let A, B, C be matrices of the same size, and let r and s be scalars.(a) A + B = B + A(b) (A + B) + C = A + (B + C)(c) A + 0 = A(d) r (A + B) = r A + r B(e) (r + s) A = r A + s A(f) r (s A) = (r s) A

加法交換律加法結合律加法單位元素乘法分配律乘法分配律乘法結合律


Matrix multiplication (Transformation)

DefinitionIf there are Amxn and Bnxp = [ b1… bp] nxp, then AB = [ Ab1… Abp] mxp.

ABx means that

Amxn, Bnxp and xpx1 Bx = x1b1+…+xpbp

A (Bx) = Ax1b1+…+ Axpbp

= x1 Ab1+…+xp Abp

= [ Ab1… Abp] mxp xpx1

The multiplication by [ Ab1… Abp] transforms x into AB (x).

x

(AB) x

BxB

A (Bx)A


Multiplication of matrices corresponds to composition of linear transformation.

Ex.3.

Each column of AB is a linear combination of the columns of A using weights from the corresponding column of B.

Row-column rule for computing AB


The i th row of AB is the i th row of A times B.

Properties of matrix multiplication

Theorem 2Let A be mxn and let B and C have sizes for which the indicated sums and products are defined.(a) A (BC) = (AB) C(b) A (B + C) = A B + A C(c) (B + C) A = B A + C A(d) r (AB) = (rA) B = A (rB), for any scalar r.(e) Im A = A = A ImProof. Omitted.

← the i th row


Warnings1) In general, the exchange law do not hold for matrix

multiplication. That is, AB ≠ BA; for example,

2) The cancellation laws do not hold for matrix multiplication.That is “AB = AC ” B = C. For example,

3) A B = 0 A = 0 or B = 0; for example,


Powers of a matrixAnxn is a square matrix and k is a nonnegative integer.Ak = A ⋅ A ⋅ A ⋅ ⋅ ⋅ A ⋅ I

A0 = I

The transpose of a matrixThe transpose of Amxn is AT, (AT)nxm. A = [aij ] AT = [aji ].

Theorem 3Let A and B denote matrices whose sizes are appropriate for the following sums and products.(a) (AT)T = A(b) (A + B)T = AT + BT

(c) (r A)T = r AT, for any scalar r.(d) (A B)T = BT AT

Exercises of Section 2.1.

k


2.2 The inverse of a matrix Only square matrix has possibly inverse matrix. Assume that A is an nxn matrix. If there is an nxn matrix C

such that AC = CA = I , where I is the nxn identity matrix, then we say that A is invertible and we call C an inverse of A.

Inverse matrix is unique.If A has two inverse matrices B and C, thenB = BI = B (AC) = (BA) C = IC = C.

The inverse matrix of A is denoted by A -1,so that A A -1 = A -1A = I.

Theorem 4 (2x2 special case for inverse matrix)

Let . If ad - bc ≠ 0, then A is invertible and

. If ad - bc = 0, then A is not invertible.


The quantity (ad - bc) is called the determinant of , and denoted by det A.

A is invertible ⇔ det A ≠ 0.

Theorem 5If A is an invertible nxn matrix, then for each b in R n, the equation Ax = b has the unique solution x = A -1b.

Invertible ⇔ exist a unique solution.

Ex.4.Use the inverse of matrix to solve linear system

3x1 + 4x2 = 3 5x1 + 6x2 = 7

Answer.


Theorem 6(a) A is invertible A-1 is invertible and (A-1)-1 = A.(b) Anxn and Bnxn are invertible AB is invertible and

(AB) -1 = B -1A -1.(c) A is invertible AT is invertible and (AT) -1 = (A -1)T.

Proof.(a) A is invertible A-1 AA-1 = A-1A = I.

AA-1 = A-1A = I can be viewed as “given A -1, A A -1

satisify the equation” and then A -1 is invertible.The inverse of A-1 = (A-1) -1 = A

(b) By associative law of multiplication(AB)(B -1A-1) = A(BB -1)A-1 = AIA-1 = AA-1 = I(B -1A-1)(AB) = B -1(A-1A)B = B -1IB = B -1B = I.

(c) (AT)(A-1)T = (A-1A)T = I T = I(A-1)T(AT) = (AA-1)T = I T = I. (by Theorem 3(d) on page 8)


Conclusion(i) The product of invertible matrices is invertible, and

the inverse is the product of their inverses in the reverse order

(ii) Invertible matrices are row equivalent to identity matrices (will be proved).

Elementary matricesAn elementary matrix is one that is obtained by performing a single elementary row operation on an identity matrix. (page 13 of Ch.1)

Ex.5.

E1 A = ? E2 A = ? E3 A = ?


Elementary row operation Elementary matrixIf an elementary row operation is performed on an mxnmatrix, the resulting matrix may be written as EA, where E is created by performing the same row operation on Im.

Each elementary matrix E is invertible. The inverse of E is the elementary matrix of the same type that transforms E back into I. That is E -1E = I.

Ex.6.


Theorem 7An nxn matrix A is invertible if and only if A is row equivalent to In, and in this case, any sequence of elementary row operation that reduces A to In also transform In into A -1.

Explain.A is invertible Ax = b has a solution for each b.

A has a pivot position in every row. EnEn-1…E1A = I and EnEn-1…E1I = A -1.

A is invertible ⇔ EnEn-1…E1A = I and EnEn-1…E1I = A -1.

Finding A -1nxn (the 1st method)

Row reduce the augmented matrix [A I ] to get [I A-1].


Ex.7.

row reduce


Another view of matrix inversion (the 2nd method)The row reduction of [A I ] to [I A -1] may be viewed as the n solution of the n linear systems.If we take A -1 = [x1 x2 …xn ],thenA A-1 = A [x1 x2 …xn ] = [Ax1 Ax2 … Axn] = I = [e1 e2 … en] Ax1 = e1, Ax2 = e2, …, Axn = en.



2.3 Characterizations of invertible matricesTheorem 8 (The invertible matrix theorem)

Let A be an nxn square matrix. Then the following statements are equivalent.(a) A is an invertible matrix.(b) A is row equivalent to the n x n identity matrix.(c) A has n pivot positions.(d) The equation Ax = 0 has only the trivial solution.(e) The columns of A form a linear independent set.(f ) The linear transformation x |→ Ax is one-to-one.(g) The equation Ax = b has at least one sol. for each b in R n.(h) The columns of A span R n.(i ) The linear transformation x |→ Ax maps R n onto R n.(j ) There is an nxn matrix C such that CA = I.(k) There is an nxn matrix D such that AD = I.(l ) AT is an invertible matrix.


Let A and B be square matrices. If either BA = I or AB = I, then A and B are invertible, with B = A -1 and A = B -1.Proof.(i) If BA = I, by Theorem 8(a) & (j), A is invertible.

B = BI = BAA -1 = IA -1 = A -1.Since A is invertible, A-1 is invertible, thus A = B -1.

(ii) If AB = I, by Theorem 8(a) & (j), then B is invertible.A = AI = ABB -1 = IB -1 = B -1.Since B is invertible, B -1 is invertible, thus B = A -1.

Inverse linear transformationx |→ Ax is linear transformation;A-1Ax |→ x is also linear transformation.

x A -1A x

A


A linear transformation T: R n → R n is said to be invertibleif there exists a function S: R n → R n such that

S (T(x)) = x for all x in R n

T (S(x)) = x for all x in R n

S is the inverse of T and denoted by T -1.

Theorem 9T: R n → R n ≡ x |→ Ax

(1) T is invertible ⇔ A is invertible.(2) The inverse transformation S of T is unique.



2.4 Partitioned matrices

For speedup the computation of sparse matrices.

Ex.1.稀疏矩陣


Addition and scalar multiplication(1) Amxn, Bmxn

(2) Partition A → [Aij ], B → [Bij ].then r A = [r Aij ] and A + B = [Aij + Bij ].

Multiplication of partition matricesPartition A → [Aij ], B → [Bjk ],then AB = [ Σ Aij Bjk ]ik.


Ex.3.2x2 2x1


Summary of special casesrow-column expansion


Theorem 10 (column-row expansion)


The relation between row-column expansion and column-row expansion.

計算次序不同而已


Ex.4.


Inverse of partitioned matricesAmxm → A2x2 to find A -1

2x2 A2x2 A -12x2 = Imxm

Ex.5. (A special case)


(1) If A22 is square, then A22 is invertible, and then B22 = A22-1.

(2) A22 is invertible, A22B21 = 0, B21 = 0.(3) B21 = 0, then A11B11 + A12B21 = A11B11 = I.

If A11 is square, then A11 is invertible and B11 = A11-1.

(4) B22 = A22-1, then A11B12 + A12B22 = A11B12 + A12A22

-1 = 0A11B12 = -A12A22

-1 B12 = -A11-1A12A22

-1.Note that B12 and B21 need not be squared.

Some terminologies

Diagonal Upper triangularmatrix matrix


Lower triangular matrix

Block diagonal matrix

Block upper triangular matrix

Block lower triangular matrix

Exercises ofSection 2.4.


2.5 Matrix factorization A |→ LU, L is a lower triangular matrix and U is a upper

triangular matrix.A is factorized into L and U.

The factorization dramatically speeds up a solution of Ax = b that uses row reduction, provided that L and U are known.

Ax = b LU x = b Ly = b then Ux = y.First solve Ly = b for y and then solve Ux = y for x. Each equation is easy to solve because L and U are triangular.


Ex.1.



Row reduction of [A|b] to [I |x] takes

Gaussian elimination= forward eliminationbackward substitution


An LU factorization algorithmA → U (through a sequence of row replacement operations)That is, Ep … E1A = U A = (Ep … E1) -1U = LU

L = (Ep…E1) -1.Assume any Ei is a lower triangular matrix. (i.e., no interchange) Products and inverses of lower triangular matrices are also lower triangular matrix. Thus L is a lower triangular matrix.

How to find L directly. (Algorithm for an LU factorization)(1) Reduce A to an echelon form U by a sequence of row

replacement operations, if possible.(2) Place entries in L such that the same sequence of row

operations reduces L to I.

A = LU L-1A = U (EpEp-1…E1) A = U L = E1-1E2

-1…Ep-1.


Ex.2.



The multiplication order must be preserved.

There is no row interchange in the example, if interchanges are desired, then the procedure above can be modified to produce an L that is permuted lower triangular matrix.

Exercises for Section 2.5.


2.7 Applications to computer graphics

We don’t present applications but just introduce the concept of homogeneous coordinates (齊次座標).

Each point (x, y) in R 2 can be identified with the point (x, y, 1) on the plane in R 3 that lies one unit above the xy-plane. We say that (x, y) has homogeneous coordinates (x, y, 1).

Each point (x, y, z) in R 3 has homogeneous coordinates (x, y, z, 1) in R 4.

In other words, (x, y, z, 1) are homogeneous coordinates for the point (x, y, z) in R 3.


If (x, y, z, 1) becomes (x’, y’, z’, h) after computation or processing, where h ≠ 1. In this situation, the values of all coordinates are needed to be normalized such that the value of the last coordinate becomes 1 through dividing all values by the last value.

For example, (x’, y’, z’, h) becomes (x′h , y′h , z′

h ,1), h ≠ 0; after

that, (x′h , y′h , z′

h) are really the coordinates of (x, y, z) after processing.

Examplesi. (5, 8, 1) is the homogeneous coordinate of (5, 8)ii. (8, 102, 36, 2) is the homogeneous coordinate of (4,51,18)

Homogeneous coordinate is used to describe both linear and non-linear transformations using matrix multiplication.


Three basic linear transformations: scaling, translation, and rotation, where translation is not a linear transformation and can’t be directly described by a matrix multiplication. Perspective projection (透視投影) is also not a linear transformation.

Here we want to describe these transformations using matrix multiplication.


2-D Translation --- with displacements (xo, yo)

2-D Scaling --- with scaling factors (sx, sy)

where (x, y) is the location before scaling and (x’, y’) is after scaling.

2-D Rotation - rotation angle is measured counterclockwise

x

y ϕ

(page76 of Ch.1)


Composite transformationsThe movement of an object on a 2-D plane often requires two or more basic transformations. The composition of such transformations corresponds to matrix multiplication when homogeneous coordinates are used.

Ex.6Find the 3×3 matrix that corresponds to the composite transformation of a scaling by 0.3, a rotation of 90o, and finally a translation that adds (-0.5, 2) to each point of a 2-D object.Solution. Since ϕ = 90o = π/2, sin ϕ = 1 and cos ϕ = 0.

translation rotation scaling


3-D Translation --- with displacements (xo, yo, zo)

3-D Scaling --- with scaling factors (sx, sy, sz)

where (x, y, z) is the location before scaling and (x’, y’, z’) is the location after scaling.


All above transformations are invertible, but not exchanged.

3-D Rotation --- using right-handed coordinate system andthe rotation angle is measured counterclockwise.

, where ? = x, y, or z;

Rx, Ry, and Rz are rotation respect to x, y, and z axis, respectively.The rotation matrices are described as follows.

y

(pitch) α

β (yaw)

θ (roll)

z

x(0, 0, 0)


Perspective transformations A perspective transformation (also called an imaging

transformation) projects 3-D points onto a plane. These transformations are nonlinear in that they involve division by coordinate values.

An imaging model (pin-hole system).

Where z-axis is called an optical axis and q is the distance from the image plane center to the camera lens center.

z

y

x

Image plane

(x, y, z)

(x’, y’)(0, 0, q)

(0, 0, 0)

Camera lens center


An image point (x’, y’) is projected from a 3-D point (x, y, z).

Top viewof the imaging model.

Front viewof the imaging model.

2D configuration

z

x

q

z − q

x

-x’z

yq

z − q

y

-y’


The projection transformation is denoted by a matrix as

The perspective projection matrix is

The perspective transformation is denoted by matrix as


The perspective projection transformation is


Exercises for Section 2.7.

The homogeneous coordinate is only a tool for unifying the operators of linear and non-linear transformations into a unique formation; however, the treatment of homogeneous coordinates is not a real linear transformation because it needs a following normalization processing.


Image-based parking guidance (IPG) Purpose: only from a rear-view image,

providing backward trajectoryto guide driver backward driving

Distortion Homographiccalibration transformation

2. Matrix Algebra - NCU

Documents

Transcript of 2. Matrix Algebra - NCU