Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra &...
Transcript of Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra &...
![Page 1: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/1.jpg)
Matrix & Vector
Basic Linear Algebra & Calculus
![Page 2: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/2.jpg)
A rectangular array of numbers (we will concentrate on
real numbers). A nxm matrix has ‘n’ rows and ‘m’
columns
34333231
24232221
14131211
MMMM
MMMM
MMMM
3x4M
What is a matrix?
First
column
First row
Second row
Third row
Second
column
Third
column Fourth
column
12MRow number
Column number
2
![Page 3: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/3.jpg)
What is a vector?
A vector is an array of ‘n’ numbers
A row vector of length ‘n’ is a 1xn matrix
A column vector of length ‘m’ is a mx1 matrix
4321 aaaa
3
2
1
a
a
a
3
![Page 4: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/4.jpg)
Special matrices
Zero matrix: A matrix all of whose entries are zero
Identity matrix: A square matrix which has ‘1’ s on the
diagonal and zeros everywhere else.
0000
0000
0000
430 x
100
010
001
33xI
4
![Page 5: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/5.jpg)
Matrix operations
.5,1,9
,7,0,3
,4,2,1
BA
B
519
703
421
A
ihg
fed
cba
ihg
fed
cba
Equality of matrices
If A and B are two matrices of the same size,
then they are “equal” if each and every entry of one
matrix equals the corresponding entry of the other.
5
![Page 6: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/6.jpg)
Matrix operations
11110
716
1450
601
013
1031
519
703
421
BAC
BA
Addition of two
matrices
If A and B are two matrices of the same size,
then the sum of the matrices is a matrix C=A+B whose
entries are the sums of the corresponding entries of A and
B
6
![Page 7: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/7.jpg)
Properties of matrix addition:
1. Matrix addition is commutative (order of
addition does not matter)
2. Matrix addition is associative
3. Addition of the zero matrix
Matrix operations
ABBA
Addition of of matrices
CBACBA
AA00A
Properties
7
![Page 8: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/8.jpg)
Matrix operations Multiplication by a
scalar
If A is a matrix and c is a scalar, then the product cA is a
matrix whose entries are obtained by multiplying each of
the entries of A by c
15327
2109
1263
3
519
703
421
cA
cA
8
![Page 9: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/9.jpg)
Matrix operations
Multiplication by a
scalar
If A is a matrix and c =-1 is a scalar, then the product
(-1)A =-A is a matrix whose entries are obtained by
multiplying each of the entries of A by -1
519
703
421
1
519
703
421
-AcA
cA
Special case
9
![Page 10: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/10.jpg)
Matrix operations Subtraction
-AA-0 and 0A- Athat Note
BAC
BA
118
710
612
601
013
1031
519
703
421
If A and B are two square matrices of the same
size, then A-B is defined as the sum A+(-1)B
10
![Page 11: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/11.jpg)
Special operations
Transpose
If A is a mxn matrix, then the transpose of A is
the nxm matrix whose first column is the first
row of A, whose second column is the second
column of A and so on.
574
102
931
A
519
703
421
A T
11
![Page 12: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/12.jpg)
Special operations
Transpose
TAA
If A is a square matrix (mxm), it is called
symmetric if
12
![Page 13: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/13.jpg)
Matrix operations Scalar (dot) product of
two vectors
3
2
1
3
2
1
b
b
b
;
a
a
a
ba
If a and b are two vectors of the same size
The scalar (dot) product of a and b is a scalar
obtained by adding the products of
corresponding entries of the two vectors
T
1 1 2 2 3 3a b a b a b a b
13
![Page 14: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/14.jpg)
Matrix operations Matrix multiplication
For a product to be defined, the number of columns
of A must be equal to the number of rows of B.
A B = AB m x r r x n m x n
inside
outside
14
![Page 15: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/15.jpg)
If A is a mxr matrix and B is a rxn matrix, then the
product C=AB is a mxn matrix whose entries are
obtained as follows. The entry corresponding to row ‘i’
and column ‘j’ of C is the dot product of the vectors
formed by the row ‘i’ of A and column ‘j’ of B
3x3 3x2
3x2
1 2 4 1 3
A 3 0 7 B 3 1
9 1 5 1 0
3 5 1 1
C AB 10 9 notice 2 3 3
7 28 4 1
T
Matrix operations Matrix multiplication
15
![Page 16: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/16.jpg)
Properties of matrix multiplication:
1. Matrix multiplication is noncommutative
(order of addition does matter)
It may be that the product AB exists but BA
does not (e.g. in the previous example
C=AB is a 3x2 matrix, but BA does not
exist)
Even if the product exists, the products AB
and BA are not generally the same
Matrix operations
ge ne ra l inBA AB
Multiplication of
matrices
Properties
16
![Page 17: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/17.jpg)
2. Matrix multiplication is associative
3. Distributive law
4. Multiplication by identity matrix
5. Multiplication by zero matrix
6.
Matrix operations
CABBCA
Multiplication of
matrices
Properties
AIA A;AI
00 A 0 ;A0
CABAACB
ACABCBA
TTTABBA
17
![Page 18: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/18.jpg)
1. If A , B and C are square matrices of the
same size, and then
does not necessarily mean that
2. does not necessarily imply that
either A or B is zero
Matrix operations Miscellaneous
properties
0A ACAB CB
0A B
18
![Page 19: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/19.jpg)
Inverse of a matrix
Definition
IABBA
If A is any square matrix and B is another
square matrix satisfying the conditions
Then
(a)The matrix A is called invertible, and
(b) the matrix B is the inverse of A and is
denoted as A-1.
The inverse of a matrix is unique
19
![Page 20: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/20.jpg)
Inverse of a matrix
Uniqueness
The inverse of a matrix is unique
Assume that B and C both are inverses of A
CB
BBIB(AC)
CIC(BA)C
ICAAC
IBAAB
Hence a matrix cannot have two or more
inverses. 20
![Page 21: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/21.jpg)
Inverse of a matrix
Some properties
AA1-1
Property 1: If A is any invertible square
matrix the inverse of its inverse is the matrix A
itself
Property 2: If A is any invertible square
matrix and k is any scalar then
1-1A
k
1Ak
21
![Page 22: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/22.jpg)
Inverse of a matrix
Properties
-111ABBA
Property 3: If A and B are invertible square
matrices then
111
11
11
11
1
ABAB
B by sides both yingPremultipl
AABB
AABBAA
AAB(AB)A
Aby sides both yingPremultipl
IAB(AB)
1-
1-
1-
1-
22
![Page 23: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/23.jpg)
The determinant of a square matrix is a number
obtained in a specific manner from the matrix.
For a 1x1 matrix:
For a 2x2 matrix:
What is a determinant?
1 11 1 aA aA )det(;
211222112221
1211aaaaA
aa
aaA
)det(;
Product along red arrow minus product along blue arrow 23
![Page 24: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/24.jpg)
Example 1
75
31A
8537175
31)A det(
Consider the matrix
Notice (1) A matrix is an array of numbers
(2) A matrix is enclosed by square brackets
Notice (1) The determinant of a matrix is a number
(2) The symbol for the determinant of a matrix is
a pair of parallel lines
Computation of larger matrices is more difficult 24
![Page 25: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/25.jpg)
For ONLY a 3x3 matrix write down the first two
columns after the third column
Duplicate column method for 3x3 matrix
3231
2221
1211
333231
232221
131211
aa
aa
aa
aaa
aaa
aaa
Sum of products along red arrow
minus sum of products along blue arrow
This technique works only for 3x3 matrices
332112322311312213
322113312312332211
aaaaaaaaa
aaaaaaaaa)A
det(
aaa
aaa
aaa
A
333231
232221
131211
25
![Page 26: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/26.jpg)
Example 2
2
01A
21-
4
3-42
12
01
42
212
401
342
0 32 3 0 -8 8
Sum of red terms = 0 + 32 + 3 = 35
Sum of blue terms = 0 – 8 + 8 = 0
Determinant of matrix A= det(A) = 35 – 0 = 35 26
![Page 27: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/27.jpg)
Finding determinant using inspection
Special case. If two rows or two columns are proportional
(i.e. multiples of each other), then the determinant of the
matrix is zero
0
872
423
872
because rows 1 and 3 are proportional to each other
If the determinant of a matrix is zero, it is called a
singular matrix
27
![Page 28: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/28.jpg)
If A is a square matrix
Cofactor method
The minor, Mij, of entry aij is the determinant of the submatrix
that remains after the ith row and jth column are deleted from A.
The cofactor of entry aij is Cij=(-1)(i+j) Mij
31233321
3331
2321
12 aaaaaa
aaM
3331
2321
1212aa
aa MC
aaa
aaa
aaa
A
333231
232221
131211
What is a cofactor?
28
![Page 29: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/29.jpg)
Sign of cofactor
What is a cofactor?
-
--
-
Find the minor and cofactor of a33
4140201
42M 33
Minor
4MM)1(C 3 33 3)33(
3 3 Cofactor
2
01A
21-
4
3-42
29
![Page 30: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/30.jpg)
Cofactor method of obtaining the determinant of a matrix
The determinant of a n x n matrix A can be computed by
multiplying ALL the entries in ANY row (or column) by
their cofactors and adding the resulting products. That is,
for each and
ni1 nj1
n jn j2 j2 j1 j1 j CaCaCaA )de t (
Cofactor expansion along the ith row
inini2i2i1i1 CaCaCaA )det(
Cofactor expansion along the jth column
30
![Page 31: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/31.jpg)
Example 3: evaluate det(A) for:
1
3
-1
0
0
4
5
1
2
0
2
1
-3
1
-2
3
A=
det(A)=(1)
4 0 1
5 2 -2
1 1 3
- (0)
3 0 1
-1 2 -2
0 1 3
+ 2
3 4 1
-1 5 -2
0 1 3
- (-3)
3 4 0
-1 5 2
0 1 1
= (1)(35)-0+(2)(62)-(-3)(13)=198
det(A) = a11C11 +a12C12 + a13C13 +a14C14
31
![Page 32: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/32.jpg)
By a cofactor along the third column
Example 4: evaluate
det(A)=a13C13 +a23C23+a33C33
det(A)=
1 5
1 0
3 -1
-3
2
2
= det(A)= -3(-1-0)+2(-1)5(-1-15)+2(0-5)=25
det(A)= 1 0
3 -1
-3* (-1)4 1 5
3 -1
+2*(-1)5 1 5
1 0
+2*(-1)6
32
![Page 33: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/33.jpg)
Quadratic form
U dkdT
The scalar
Is known as a quadratic form
If U>0: Matrix k is known as positive definite
If U≥0: Matrix k is known as positive semidefinite
matrixsquarek
vectord
33
![Page 34: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/34.jpg)
Quadratic form
2221
1211
2
1
kk
kkk
d
ddLet
Then
2
2222112
2
111
22211222121111
222112
212111
21
2
1
2212
1211
21
2
)()(
U
dkddkdk
dkdkddkdkd
dkdk
dkdkdd
d
d
kk
kkdddkd
T
Symmetric
matrix
34
![Page 35: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/35.jpg)
Differentiation of quadratic form
Differentiate U wrt d2
212111
1
22U
dkdkd
Differentiate U wrt d1
222112
2
22U
dkdkd
35
![Page 36: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/36.jpg)
dk
d
d
kk
kk
d
d
d
2
2U
U
U
2
1
2212
1211
2
1
Hence
Differentiation of quadratic form
36
![Page 37: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/37.jpg)
Review:
• The Derivatives of Vector Functions
• The Chain Rule for Vector Functions
37
![Page 38: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/38.jpg)
38
1. The Derivatives of Vector Functions
![Page 39: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/39.jpg)
39
1.1 Derivative of Vector with Respect to Vector
![Page 40: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/40.jpg)
40
If y is a scalar
It is also called the gradient of y with respect to a vector
variable x, denoted by . y
1.2 Derivative of a Scalar with Respect to Vector
1.3 Derivative of a Vector with Respect to Scalar
![Page 41: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/41.jpg)
41
Given
and
Example 5
![Page 42: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/42.jpg)
42
>> syms x1 x2 x3 real;
>> y1=x1^2-x2;
>> y2=x3^2+3*x2;
>> J = jacobian([y1;y2], [x1 x2 x3])
J =
[ 2*x1, -1, 0]
[ 0, 3, 2*x3]
Note: Matlab defines the derivatives as the transposes of those given in this lecture.
>> J'
ans =
[ 2*x1, 0]
[ -1, 3]
[ 0, 2*x3]
In Matlab
![Page 43: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/43.jpg)
43
2
T
T
T
CxC
x CC
x xx
x
x
x
n
tntt
n
ttt
n
ttt
nnnnn
n
n
Cx
Cx
Cx
x
x
x
CCC
CCC
CCC
1
12
11
2
1
21
22221
11211
T
nnnn
n
n
C
ccc
ccc
ccc
21
22212
12111
x
Cx
Exercise:
Some useful vector derivative formulas
![Page 44: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/44.jpg)
44
T
1 1
T1 1 1 1
1 1
TT T
( )
( )
n n
i j ij
i j
n n n n
i j ij k j kj i k iki j j i
k k k k
n n
j kj i ik
j i
x x C
x x C x x C x x C
x x x x
x C x C
x
x Cx
x Cx
x CxCx C x C C x
Proof:
T
T( )
x CxC C x
x
n
tntt
n
ttt
n
ttt
nnnnn
n
n
Cx
Cx
Cx
x
x
x
CCC
CCC
CCC
1
12
11
2
1
21
22221
11211
If C is symmetric:
T( )2C
x Cxx
x
Important Property of Quadratic Form xTCx
![Page 45: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/45.jpg)
Fin500J Topic
2
Fall 2010 Olin
Business School
45
Let
where z is a function of y, which is in turn a function of x, we can write
Each entry of this matrix may be expanded as
2. The Chain Rule for Vector Functions
![Page 46: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/46.jpg)
46
Then
On transposing both sides, we finally obtain
This is the chain rule for vectors (different from the conventional chain rule of calculus, the chain of matrices builds toward the left)
The Chain Rule for Vector Functions (Cont.)
![Page 47: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/47.jpg)
47
x, y are as in Example 1 and z is a function of y defined as
2
1 1 21
22 2 2 1
2 23 3 1 2
4 4 1 2
31 2 4
1 1 1 1 1 1
2 231 2 4
2 2 2 2
2
, , we have
2
2 1 2 2.
2 2 2 1
Therefore,
2
z y yz
z z y yz and
z z y y
z z y y
zz z z
y y y y y yz
y yzy z z z
y y y y
z y z
x x y
1 1 1 1 1 11
1 1
1 2 2 2
2 2
3 3 3 2 3 2 3
4 2 4 402 1 2 2
2 6 1 6 2 6 11 32 2 2 1
0 2 4 4 4 2
x y x x y xxy y
y y y yy y
x x x y x y x
Example 6
![Page 48: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/48.jpg)
48
>> z1=y1^2-2*y2;
>> z2=y2^2-y1;
>> z3=y1^2+y2^2;
>> z4=2*y1+y2;
>> Jzx=jacobian([z1; z2; z3; z4],[x1 x2 x3])
Jzx =
[ 4*(x1^2-x2)*x1, -2*x1^2+2*x2-6, -4*x3]
[ -2*x1, 6*x3^2+18*x2+1, 4*(x3^2+3*x2)*x3]
[ 4*(x1^2-x2)*x1, -2*x1^2+20*x2+6*x3^2, 4*(x3^2+3*x2)*x3]
[ 4*x1, 1, 2*x3]
>> Jzx’
ans =
[ 4*(x1^2-x2)*x1, -2*x1, 4*(x1^2-x2)*x1, 4*x1]
[ -2*x1^2+2*x2-6, 6*x3^2+18*x2+1, -2*x1^2+20*x2+6*x3^2, 1]
[ -4*x3, 4*(x3^2+3*x2)*x3, 4*(x3^2+3*x2)*x3, 2*x3]
In Matlab
![Page 49: Matrix & Vector - Stony Brookzhu/ams571/matrixvector.pdfMatrix & Vector Basic Linear Algebra & Calculus . A rectangular array of numbers (we will concentrate on real numbers). A nxm](https://reader030.fdocuments.net/reader030/viewer/2022040201/5e4dca4422be286a9e36158d/html5/thumbnails/49.jpg)