California State University Fullerton Shahin Ghazanshahi

EGEE 581

Theory of Linear Systems

Introduction

Previous Courses In System Theory Involved:

1) Linear Systems (Superposition holds)

2) Time Invariant Systems

3) Single Input Single Output (SISO)

4) Ordinary Differential Equations

5) Transfer Functions

6) Poles and Zeros

7) Hand Calculations

Previous Courses Are Inadequate For Modern, Complex Systems:

State Space Theory Addresses:

1) Linear or Non-Linear Systems

2) Time Invariant Or Time Variant Systems

3) Multiple Input Multiple Output

4) Vector Matrix Differential Equations

5) Time Domain Solutions

6) Transfer Functions Must Be Converted To Vector Matrix Differential

Equations

7) Digital Computation Can Be Exploited

3

Limitations:

Limitations Of Previous Methods:

1) 3rd

Order Systems; Single Input Single Output

Limitations of New Methods:

1) 50 -200th

Order Systems

2) Limitation Is Based On Adequacy Of Model And Power Of Digital

Computer

Paradox N0. 1:

Major Benefit Of State Space Method Is The Ability To Use Very

Powerful Digital Computer Capacity

Yet Courses And Texts Put Little Emphasis On Use Of Digital

Computation:

Paradox N0. 2:

The Critical Element In Successful Application Of Any Theory

( Including State Space) Is Adequacy Of Mathematical Models.

Modeling Success Is Gainted Through Experience; Its An Art, Requiring Detailed Knowledge Of A Physical System.

4

Chapter 1

Physical System: Collection of physical devices in real world.

system : is a model of physical system.

Two methods to study and design physical system:

1) Imperical Method:

Apply various Signals Measure Responses

Then adjust parameters or connect to a compensator to improve

performance. This approach relies on past experience and it is carried out

by trial and error and it has succeeded in designing many physical systems.

Not good for complex systems

Expensive

Too Dangerous

2) Analytical Method:

Modeling

Developing Mathematical Descriptions

Analysis

Design

1) Modeling

A physical system may have different model based on question we are

looking and its experimental range.

1) Circuit or control systems are models.

5

2) R, L, C1 are model of physical elements

3) Electronic amplifier is modeled differently at high or low

frequencies.

Choosing model that is close to a physical system and yet simple enough

to be studied analytically is the most difficult and the most important in

system design.

In this class we shall assume that models of physical systems are

available to us.

2) Mathematical Model:

Once a system (model) is selected, physical laws (i.e, KVL, KCL.

Newtons Law) will be apply to get mathematical model of physical

system. The mathematical model can be Linear/Non-linear;

Differential/Difference equations; Integral Equations; and etc.

3) Analysis:

Quantitative Analysis Interested in response of the system

Qualitative Analysis Interested in general properties of the system

such as Stability, Controllability and Observability

1 Resistor, Inductor, Capacitor

6

Systems:

Linear ( L )

Non-Linear ( NL )

Time-Invariant ( TI )

Time-Variant ( TV )

Single Input-Single Output( SISO )

Multiple Input-Multiple Output ( MIMO )

Continues Time ( CT )

Discrete Time ( DT )

Time Domain Description:

Differential Equstion

() + () + () = () + 2 ()

Convolution

() = ( )()

= 0

() := impulse response

State Space Equation:

() = () + ()

() = () + ()

() := State and internal variable

Transfer Domain Description:

() = ()()

() = Transfer function

7

8

Chapter 3-Review Of Matrix:

I assume you have basic knowledge of matrix theory. You should know:

A = {} = {}

C = AB = {} = =1

KA = {}

(A+B)C = AC + BC

AB BA (in general)

(AB)T = BTAT

det A = || = Scalar

det (AB) = det A det B (A , B n by n)

det A 0 A is non-singular; A-1 exists

det A = 0 A is singular, A-1 doesnt exist.

(AB)-1 = B-1A-1

A-1 =

|| used in hand calculations limit 3 *3

A-1 numerical methods (See EE 403 text)

Rank A n*m = Number of linearly independent rows

= Number of linearly independent columns

(A-1)T = (AT)-1

Trace A =

() =

+

(1) = 1

1

9

() = { ()}

Math Sinology:

Common short hand symbols used

= There exist

= Such that

= For all

|| = Determinant Of A

{} = Set of s

= Implied by (Only If)

= Implies (If)

= Iff = If and Only If

= Is included in (member of a set)

= Is included in (subset of a set)

One= Exactly One = One and Only One

= One Or More (at least one)

Necessity And Sufficiency:

= A implies B

= A is sufficient condition for B but it may not be necessary.

= not A not B

= If A, then B

not B not A

10

= A implied by B

A is a necessary condition for B but it may not be sufficient.

is equivalent to NOT A NOT B

Definition:

Dimension: The dimension of a vector space V = maximum number of

the linearly independent vectors V

:

(R3, R) 3 dimensional real space

Note: Any 3 of {} are linearly independent

All 4 are Not

Basis: coordinate system {} are a basis of V iff :

1) {} are linearly independent

2) Every X V is a Unique linearly combination of {}

X = 1 1 + 2 2 + 3 3

[1 , 2 , 3 ] Representation of vector X w.r.t. the basis

[1 , 2 , 3 ]

X3 X4

X2

X1

11

:

In the example above:

1) 1 , 2 are NOT a basis

2) 1 , 2 , 3 , 4 are NOT a basis

3) 1 , 2 , 3 is a basis

4) 1 , 3 , 4 is a basis

Theorem:

If the dimension of V = n, then any set of n linearly independent {} is a

basis of V

Proof:

1) {} are linearly independent

2) Choose arbitrary X V, then ; X, 1 , 2 , , 3 are linearly

dependent, since maximum number of linearly independent is n

0 + 1 1 + + = ; with {} not all zero

0 0, otherwise 0 = 0 and some other 0 , j0

1 1 + + + + = 0 linearly independent

(contradiction)

0 0 and;

X=- 1

0 (1 1 + + )

X= (1 1 + + )

12

:

An arbitrary X V can be written as a linearly combination of {}i=1

show linearly combination is Unique

Let;

X = 1 1 + + Be another linear combination

Then;

X = 1 1 + + = X = 1 1 + +

(1 1)1 + + ( ) = ;

{} linearly independent

( ) = ; i

= linearly combination is Unique

key Result:

Let {} be a Known Basis of V and X V an arbitrary vector

Then;

X = 1 1 + + = [1, , ] [

1 .

]

And X V is completely Represented by

= [

1 .

] ; a column vector of n real numb

13

Important Notes:

1) A basis is Not Unique; i.e, different Bases for some V can be chosen

(equivalent to changing coordinate systems)

2) But for Each basis, X V has Unique representation with respect to

That basis

3) The same X V has different representations (one for each basis)

4) Some basis are better (more convenient to use). Orthonormal basis

often used.

:

(1)

(2) (, R) = n dimension real vector space of n-tuples column vectors)

1 = [

1 .0

] = [

0 .1

]

{} is a basis of (, R)

Then X , is written as:

n 1

n 2

n3

14

X = 1| | = [

1 0 00 1 0 0 0 1

] [

12

]

Changing Bases:

Let X () , X arbitrary

Let V have two bases {} and {}

Then;

(*) X = [1 . ] = [1 ] ; = [

1 .

] , = [

1 .

]

How are and related.

Note:

can be written as a linearly combination of {}

[1 ] = [1 ] [11 1

1

]

= [1 ] P

Substitute into (*)

[1 ] P = [1 ] ; but linear combination is Unique

Coefficients are unique

P =

15

Similarly each can be written as a linear combination of {}

[1 ] = [1 ] [11 1

1

]

Substitute into (*)

[1 ] = [1 ] Q = Q

Previously P = = QP (**)

But XV was arbitrary chosen

is arbitrary, i.e, (**) holds

= P = 1

Summary:

Change of basis = P

= P1

:

X = [13] 2 and consider 1 = [

31] and 2 = [

22] as new basis. Note

X = [13] is representation of X at orthonormal basis. Draw from X a line, in

parallel with 2 line which interacts the 1line at 1. Draw from X a line

in

P

1 =Q

and represent X V

16

parallel with 1 which interacts the 2 line at 22. So representation of X

respect to 1 and 2 is [12

].

Check

X= [13] = [1 2] [

12

] = [3 21 2

] [12

]

= Q

= Q1

Norm of vectors:

Norm is the length or magnitude of the vector shown as when;

1) 0

2) = || real

3) 1 + 2 1 + 2

let;

X= [1, 2, , ]t , then norm of X are any of the follow.

1) 1-norm: 1 ||=1

2) 2-norm, Euclidian : 2 ||2=1

3) Infinite norm: = max ||

:

X = [2 4]

1 = 2 + |4| = 6

2 = = 22 + (4)2 = 4.47

= 4

17

Orthogonal vectors:

Two vectors 1 and 2 are said to be orthogonal if;

12 = 21

=0

Note:

= scalar and is n

Orthonormal vectors:

; = 1, 2, , n is said to be orthonormal if;

= {

01

Orthogonal Matrix:

A square matrix is an orthogonal matrix if;

= = I

Note:

If A is not square such as 32 then;

23 32 = 2 but 32

23 3

Note:

For square orthogonal matrix A , 1 is

= A = I

A1 = I 1 = AT

i i=j

18

Schmidt Orthonormalization:

Consider a set of linearly independent vectors 1, 2, , , we can

normalize them as:

1 1 1 1

1

2 2 - (12 )1 2

22

.

(

)1=1

Note:

1 2 1 is projection of the vector 2 along 1. Its subtraction from 2

yields the vertical part 2. And 2

2 normalized 2.

Finding the rank of a matrix:

Apply row operations on A until A becomes upper triangular matrix as in

Gaussian elimination.

Rank of A will not change after pre or post- multiplication of A by a non

singular matrix.

Rank (AC) = Rank (A)

Rank (DA) = Rank (A)

19

Row Operations:

1) Interchange rows

2) Multiply a row by a non zero constant

3) Add a multiple of one row to another row

:

Find rank A

A = [0 1 1 21 2 3 42 0 2 0

] [1 2 3 42 0 2 00 1 1 2

] [1 2 3 40 4 4 80 1 1 2

]

[1 2 3 40 1 1 20 1 1 2

] [1 2 3 40 1 1 20 0 0 0

] Rank =2

1and 2 are linearly independent

Linear Algebraic Equations:

1 = 1

Note:

If AX = 0 then X null vector

Definition:

1) Nullity Maximum number of linearly independent null vectors of A

2) Nullity Number of columns of A Rank A= n- ()

3) Rank A min (m,n)

20

:

A= [

0 1 1 21 2 3 42 0 2 01 2 3 4

]

1 and 2 are linearly independent and can be used as basis for range

space of A.

Range space of A:

All possible linear combinations of all columns of A.

Back to the example above!

Nullity = 4 () = 4-2=2

1 = [

11

10

] and 2 = [

020

1

] are null vectors and are basis for null space;

1 = 2 = 0 ; 1 and 2 are linearly independent

Theorem:

1 = 1

consider:

K = n - ()

= solution of AX = Y

1) If () = ; then k=0 then = unique solution

2) If K >0; then for every real ; i=1, 2, ,k; the vector

21

X = + 11 + . +

Is the solution of AX=Y where; {1, 2. , } is null basis.

Back to the previous example:

AX = [0 1 1 21 2 3 42 0 2 0

] X = [480

] = Y

Let = [0 4 0 0] be the solution, so the general solution is;

X= + 11 + 22 = [

0400

] + 1 [

11

10

] + 2 [

020

1

] 1 and 2 are real

n- ()= number of components in X which can be arbitrarily chosen

(free parameters)

Theorem:

Consider square matrix as:

1 = 1

1) If A is a non singular or ()=n, then X = 1 Y is the unique

solution

2) Homogenous equation AX = 0 has non-zero solution iff A is a

singular matrix or () < where the number of linearly

independent solution is n-(), the nullity of A.

This theorem is important and we will use it later!

22

Similarity Transformation:

Consider = . Square matrix A maps X to Y , change

basis to new one as {1 2 } = Q

The equation become;

* =

where , are representation of X and Y with respect to the basis

{1 } where as discussed before:

X=Q

Y=Q

Substituting these in AX=Y

AQ = Q

Or:

1AQ = 1Q =

So:

1AQ = comparing with *

** = A Q or A= Q

This is called similarity transformation and A and are said to be similar.

We can write ** as:

AQ = Q

A[1 ] = [1 ] = [1 ]

So the ith column of is the representation of A with respect to the

basis [1 ]

23

:

Let A=[3 2 1

2 1 04 3 1

]

Let b=[0 0 1] then:

Ab = [101

] , 2b = A(Ab) = [42

3] , 3b = A(2) = [

510

13], Also, we can

verify that:

3 = 17b 15Ab + 52

since b, Ab and 2b are linearly independent, they can be used as new

basis. Lets find the representation of A with respect to this basis. =?

Q = [0 1 40 0 21 1 3

] = [ 2]

= 1AQ = [0 0 171 0 150 1 5

]

Note:

A(b) = [ 2] [010]

A(Ab) = [ 2] [001] As we said: AQ=Q

A(2) = [ 2] [17

155

]

24

In General:

Let A be n matrix. If there exists a n 1 vector b such that b, Ab, ,

1b are linearly independent and if

b = 1b + 2(Ab) + + (1)

then which is representation of A with respect to basis

[ . . 1] is

=

[ 0 0 . 0 11 0 . 0 20 1 . 0 3 . 0 40 0 . 0 0 0 . 1 1]

companion form

Eigenvalues and Eigenvectors:

Let A be an n matrix. Then a nonzero vector X in is called an

eigenvector of A if AX is a scalar multiple of X; that is:

AX= ; for some scalar

The scalar is called an Eigenvalue of A and X is the Eigenvector of A

corresponding to eigenvalue .

Find :

AX = X, X0 ; is a scalar

AX- X = (A- )X = 0 ; X0

i.e; (A- ) is singular

() = det (A- ) = |A | = 0 = | A|

25

Note:

1) , |A | are defined iff A is square ()

2) () = 0 is a polynomial of degree n in and is called the

characteristic polynomial of A.

() = + 11+ +1+ =0

3) () has n roots {} and can be written as:

() = |A | = ( 1) ( 2) ( )=0

where {}are not necessarily distinct

4) why do we care? A matrix A will replace the transfer function in

the role of determining system stability. Later, well show that:

= Poles of the transfer function

:

A=[1 10 1

] ; | | = |1 1

0 (1 + )| = ( 1)( + 1)

() = 2-1 = ( 1)( + 1) = 0 = 1

( ) = 0 , X0

If ; = 1 [0 10 2

] [12

] = 0 = [10

] Eigenvector

Where 1 is arbitrary

Note: | | =0 Eigenvector is not unique

26

If ; = -1 [2 10 0

] [12

] = 0 = [1/22

] Eigenvector

Where 2 is arbitrary

Note: | | =0

Theorem: {} distinct:

Let {}i=1 be distinct eigenvalues of A and be the eigenvector associated

with . Then {}i=1 is a linearly independent set { is a basis}

Corollary:

Let {}i=1 be distinct eigenvalues of A. Then A is similar to a diagonal

matrix D

D = [1 0 0

]

i.e; = 1 , where Q = [1 ]

is the eigenvector associated with

Proof:

1) Q is nonsingular since the columns are linearly independent.

2) Must merely show that QD = AQ (equivalent to D = 1AQ)

QD = [1 ] [1 0 0

] = [11 ]

27

AQ = A[1 ] = [1 ]

= [11 ]

:

A=[0 0 21 0 20 1 1

] ; () = =|A | = | 0 0

1 20 1 1

|=( 2)( + 1)

1,2,3= 2, -1, 0

(A-2I) 1 = [2 0 01 2 20 1 1

] 1 = 0 1 = [011]

Note: 1 is not unique; 1 = [0 , , ]

2 = [0

21

] 3 = [21

1]

Thus D = = [2 0 00 1 00 0 0

] which is diagonal with eigenvalues at

diagonal

= 1 where Q=[0 0 21 2 11 1 1

]

Check: Q =

This Q is said to be diagonalization matrix

1, 2, 3 are eigenvectors

28

Theorem: {} not distinct = Repeated eigenvalues:

1) If {} are not distinct, then A is similar to D Diagonal Matrix; iff

the eigenvectors are linearly independent and D = 1

2) Otherwise, A is similar to J (of Jordan Canonical Form),

i.e, = 1 =

:

A = [1 0 10 1 00 0 2

] = 1, 1, 2

If; = 1 1 = [100] 2 = [

010]

If; = 2 3 = [101

]

D = [1 0 00 1 00 0 2

] = 1

:

A = [1 1 20 1 30 0 2

] = 1, 1, 2

Three

linearly

independent

{}

But only two linearly

independent {}

29

Definition:

If is repeated times, it has multiplicity . Jordan Canonical Form:

J = [

1

] Block Diagonal

Where block i (corresponding to ) is of the form:

= [

0 00 00 0 0 0 0

] 1s on first super diagonal

Note:

A = Q 1

det A = det Q det det 1 = det .

Since det Q det 1 = det I =1

= D ; det D = det = 1 2

det A = product of all eigenvalues

This implies that A is non-singular iff it has non-zero eigenvalue

Companion form matrices:

[

0 0 0 1 0 0 1 0 0 1 1

] Or [

1 2 0 1 0 0 0 0 0 1 0

]

or their transpose such as

30

[

0 1 0 00 0 1 0 1

1 2 1

]

All have same characteristic equation:

() = + 11+ +1+ = 0

Jordan form representation:

If has repeated eigenvalues with not n independent eigenvectors then

A has Jordan-form representation;

J = [

1 1 0 00 1 1 00 0 1 10 0 0 1

]

This matrix which has its eigenvalues at diagonal and 1 on the super

diagonal is called a Jordan block of order 4. The characteristic polynomial

of J is: () = | | = ( 1)4.

We can have Jordan block of various orders. Consider 55

With repeated value 1with multiplicity of 4 and single eigenvalue 2 .

There exists a non-singular matrix Q such that the similarity matrix .

31

Assume one of the following form:

= 1

1 =

[ 1 1 0 0 00 1 1 0 00 0 1 1 00 0 0 1 00 0 0 0 2]

2 =

[ 1 1 0 0 00 1 1 0 00 0 1 1 00 0 0 1 00 0 0 0 2]

Jordan block of order 4 Jordan block of order 3 and 1

3 =

[ 1 1 0 0 00 1 0 0 00 0 1 1 00 0 0 1 00 0 0 0 2]

4 =

[ 1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 2]

Jordan block of order 2 and2 3 Jordan block of order 2 and 1 and 1

5 =

[ 1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 2]

4 Jordan block of orders 1

All these matrices are said to be Jordan form! If nullity of (A -1) is four,

then 4 linearly independent associated with 1.

And A is said to be diagonalizable. So 5 eigenvectors can be used as basis.

If nullity of (A -1) = 1, then one eigenvector associated with 1.

32

Conclusion:

Computing Jordan form matrices is complicated and will not discuss

future. But we discuss a useful property of it as for Jordan block of order

4:

(J -) = [

0 1 0 00 0 1 00 0 0 10 0 0 0

]

(J ) 2 = [

0 0 1 00 0 0 10 0 0 00 0 0 0

]

(J ) 3 = [

0 0 0 10 0 0 00 0 0 00 0 0 0

]

(J ) = 0 K 4

This is called Nilpotent

Definition:

A monic polynomial is a polynomial with 1 as its leading coefficient. The

minimal polynomial of matrix A is the polynomial () with load

coefficient = 1 and the least degree () = 0

From a practical viewpoint, finding () is NOT easy and is not used,

except in theoretical derivations.

33

Caley-Hamilton Theorem:

Every finite matrix A satisfies its own characteristic equation.

() = |A | = + 11+ +1+ = 0

Then;

() = + 11+ +1+ = 0

Applications:

1) Computing ;

pre-multiply () with 1;

1+ 12+ +1+

1 = 0

1 = -1

[1 + 1

2 + + 1]

i.e; given non-singular , then 1 is a linear combination of 1 and

lower power of A ( n- terms total)

2) Reducing the degree of Polynomial;

() = + 11+ +1+ = 0

= - [ 1 + + ]

Which implies +1 can be written as a linear combination of

[1, 2 , ] which in turn can be written as linear combination of

[, 1 , 1].

Proceeding forward, we conclude that, for any f(), no matter how large

its degree is, f() can always be expressed as:

f(A) = 0 + 1 + + 11 =

1=1

One way to get f(A) is to use long division to express f().

34

f() = q() () + h() = h() = 0 + 1 + + 11

0

() is the quotient and h() is the reminder with degree less than n. then

we have:

() = q() () + h() = h() = 0 + 1 + + 11

0