System identiﬁcation: methods and algorithms...

della Svizzera italiana TecnologieInnovative

System identification: methods and algorithms (DRAFT)

Mikael Bianchi

March 22, 2013

System identification (DRAFT)

2 Mikael Bianchi March 22, 2013

Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Some preliminary examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Recapitulation of important concepts 52.1 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1.2 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.2.1 Approximation of the derivatives . . . . . . . . . . . . . . . 82.1.2.2 Exact solution with Zero-Order-Hold sampling . . . . . . . 8

2.2 Signals and proprieties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2.1 Description of the signals . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1.1 Deterministic signals . . . . . . . . . . . . . . . . . . . . . . 102.2.1.2 Stochastic signals . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.2 Signal examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.2.1 Dirac impulse . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.2.2 Step function . . . . . . . . . . . . . . . . . . . . . . . . . . 212.2.2.3 Constant value . . . . . . . . . . . . . . . . . . . . . . . . . 212.2.2.4 Sinusoidal function . . . . . . . . . . . . . . . . . . . . . . . 222.2.2.5 White noise . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.2.2.6 Colored noise . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3 Non Parametric Identification Methods 253.1 Transient analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1.1 Step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1.2 First order system . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.1.3 Second order system . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2 Impulse response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4 Parametric Identification Methods 314.1 Least Squares Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1.1 Proprieties of the LS estimate . . . . . . . . . . . . . . . . . . . . . . 34

5 Appendix 355.1 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35


Chapter 1

Introduction

The main scope of system identification is the estimate of an unknown part of a process,given one or more sets of measures. The goal is on the one hand to characterize theprocess type or structure and on the other hand to develop a model that can reproducethe studied system. An identification of the process S is performed exciting the systemwith an input signal u(t) (figure 1.1) and observing the output response y(t) over a timeinterval. Generally, the estimation essay is made difficult by several disturbances v(t) thataffect the process and add an uncertainty in the measures. The direct consequence is that

Input Output

u(t) y(t)

Disturbance v(t)

System

S?

Figure 1.1: Generic system S generating a response y(t) to the excitation signal u(t) anddisturbance v(t).

the identified model contains an uncertainty. Therefore without appropriate mathematicalconsiderations, system identification may result imprecise and inaccurate leading incon-sistent results. For this reason statistical methods play a central role in this field. To dealwith the disturbances and in order to extract more precise and accurate information of theestimation, the system identification technique exploits the redundancy of the measuresand the a prior known system characteristics.Depending on the applications, requirements and process type, different methods can beused to perform an identification. The system identification is generally subdivided in twomain families. The first category includes the non parametric methods, which provideinformation about the process in form of graphical curves or tables. The second categoryconsists of the parametric identification methods, which provide a mathematical modeldefined for example by a differential equation, reproducing the process states and outputsin function of external excitations.System identification can be applied in many fields, like control, modeling, process moni-toring and data mining. This script will give an introduction of the system identification

March 22, 2013 Mikael Bianchi 1

System identification (DRAFT) Chapter 1. Introduction

with time-discrete dynamic linear processes, putting emphasis on the development forcontrol applications and physical modeling. The main non parametric and parametricmethods will be studied and explained with practical examples.

1.1 Motivation

The system identification is not really notorious but becomes relevant when a processrequires to be modeled mathematically or its dynamic behavior needs to be characterized.Some practical applications are the development of a state-space control and the designof a filter for the signal estimation (e.g. classical observer) or for noise reduction purposes(e.g. the Kalman filter). System identification is also used to develop simulation modelsthat reproduce most trusty the process, in order to verify the control algorithms and filtersin simulation.

Input Output

u(t) y(t)

Disturbance v(t)

System

SControl

Observer

yref (t)

Statesx(t)

eref (t)

Model structure M

and parameters θ

−

+

Figure 1.2: Example of the system identification for a controlled process with desiredoutput signal yref (t). The control and observer parametrization depend on the modelstructure M and parameters θ of the process.

The development of a controller is subdivided in different phases, as explicated forexample in figure 1.3. The first task includes the process characterization and the definitionof the system structure. This part consists in practice in estimating the model order and indetermining the required model complexity. The use of nonparametric methods is a goodchoice in this case, because it allows to become a first feeling of the process. In the secondtask a modeling of the process is performed by means of the information collected in thefirst task. More precisely, the description of the structure and the determination of theparameter number is performed. Further, the model requires to be tuned with the process,that is a parameter has to be estimated, in order to fit the real system. This part is coveredtypically by the parametric identification methods. Finally a validation is performed inorder to verify if the identification has been performed well and if the model complexity issufficient to describe the process around the working point. When the appropriate modelis found the control design can be performed. As already pointed in the beginning, one


1.2. Some preliminary examples System identification (DRAFT)

Process

characterization

Modeling

Identification

Validation

Control

development

System

SControl

ObserverObserver

Process

analysis

Model

verifi

catio

n

Figure 1.3: Possible way for the design of a controller.

of the surely most bitter part of this topic is the presence of disturbances in the processthat perturbs the estimation of the model. The cause of disturbances can be for examplemeasurement noise and quantization, modeling approximations or unmodeled effects.

1.2 Some preliminary examples

Demo in classroom, first lesson.


System identification (DRAFT) Chapter 1. Introduction


Chapter 2

Recapitulation of important

concepts

2.1 Modeling

The design of a good control requires the knowledge of the process behavior. To developa model the appropriate boundaries of the process must be defined with the input, outputsignals and internal states.Although modeling can be considered as a separate subject from system identification,there is a direct connection between the two fields. In fact the choice of the model is veryimportant for the system identification and it is important to delimit the process, usingthe model information to obtain more accurate estimation of the process. In other words,the modeling for system identification shall be a task for limiting the complexity of thefinal mathematical model, due to various considerations, such as a priori experimentalmeasures or physical descriptions.In time continuous domain the model is often described using differential equations. Let’stherefore firstly consider a linear differential equation that describes a system:

∂ny

∂tnyy(t) + ac,ny−1

∂ny−1

∂tny−1y(t) + . . .+ ac,1

∂

∂ty(t) + ac,0y(t) =

bc,nu∂nu

∂tnuu(t) + bc,nu−1

∂nu−1

∂tnu−1u(t) + . . .+ bc,1

∂

∂tu(t) + bc,0u(t),

(2.1)

with any−1, . . . , a0 and bnu, . . . , b0 being the model parameters, y(t) the output signal andu(t) the input. The corresponding transfer function (Laplace transformation) is:

Y (s) =bc,nu · snu + bc,nu−1 · snu−1 + . . .+ bc,1 · s+ bc,0

sny + ac,ny−1 · sny−1 + . . .+ ac,1 · s+ ac,0U(s), (2.2)

where Y (s) and U(s) are the Laplace-Transformed of the signals y(t) and u(t). In thefield of control application, a common way to represent the differential equations is usingthe state space visualization.

x(t) = A · x(t) +B · u(t)y(t) = C · x(t) +D · u(t)

(2.3)


System identification (DRAFT) Chapter 2. Recapitulation of important concepts

2.1.1 Linearization

A nonlinear system defined by the differential equation system:

x(t) = f(x(t), u(t)), (2.4)

can be linearized around the working points y0 and u0, using the Taylor-series:

x(t) = f(x0, u0) +

nx∑

j=1

(

∂f(x, u)

∂xj

∣

∣

∣

∣

x0,u0

· (xj − x0,j)

)

+

nu∑

j=1

(

∂f(x, u)

∂uj

∣

∣

∣

∣

x0,u0

· (uj − u0,j)

)

.

A set of first order differential equations can be similarly linearized. The nonlinear state-space equation system:

x(t) = f(x(t), u(t), t)

y(t) = g(x(t), u(t), t)

can be linearized model around a nominal trajectory x0(t), y0(t) given by u0(t) and t. Theresulting linearized model is:

x(t) = x0(t) + δx(t)

y(t) = y0(t) + δy(t),

(2.5)

with δx(t) and δy(t) representing the state and output deviation from the nominal tra-jectory. Applying the Taylor series, one can find the linearized state space system for thedeviations:

δx(t) = A(t) · δx(t) +B(t) · δu(t)δy(t) = C(t) · δx(t) +D(t) · δu(t),

with the system matrices:

A(t) =

∂f1∂x1

· · · ∂f1∂xnx

.... . .

...

∂fnx

∂x1· · · ∂fnx

∂xnx

|x0(t),y0(t)

B(t) =

∂f1∂u1

· · · ∂f1∂unu

.... . .

...

∂fnx

∂u1· · · ∂fnx

∂unu

|x0(t),y0(t)

C(t) =

∂g1∂x1

· · · ∂g1∂xnx

.... . .

...

∂gny

∂x1· · · ∂gny

∂xnx

|x0(t),y0(t)

D(t) =

∂g1∂u1

· · · ∂g1∂unu

.... . .

...

∂gny

∂u1· · · ∂gny

∂unu

|x0(t),y0(t)

.


2.1. Modeling System identification (DRAFT)

2.1.2 Discretization

With the introduction of the microprocessors and the digital signal processing, the discretesystems have become more and more important. The main disadvantage of these systems,is that the measures and actuations are performed at discrete sample times, thereforethe information between two samples is lost. To avoid an uncontrolled behavior of thesystem, the signals are generally kept constant over the sampling interval by a sampleand hold element or are increased resp. decreased with a first order hold element. Onthe other hand the main disadvantage of continuous models is that these are theoreticalsystems that cannot always be explicitly solved. A discrete model is represented by thez-Transform, which is formally the introduction of the sampling in the Laplace transform.The z operator is defined as time delay and is equivalent in the Laplace transform to:

z = e−s·ts ,

with ts being the sampling time. The transformation in time domain is given by:

Zx(t− n · ts) = z−n ·X

In digital signal processing linear discrete difference equations are generally used to ap-proximate the continuous model. The generic equation in time domain is given by:

y(t) + ad,1 · y(t− ts) . . .+ ad,ny · y(t− ny · ts) == bd,0 · u(t) + bd,1 · u(t− ts) + . . .+ bd,nu · u(t− nu · ts),

with bd,0 . . . bd,nu, ad,1 . . . ad,ny being the model parameters. The equivalent z-transform ofthe difference equation is:

Y (z) =bd,0 + bd,1 · z−1 + . . .++bd,(nu−1) · z−(nu−1) + bd,nu · z−nu

1 + ad,1 · z−1 + . . .+ ad,(ny−1) · z−(ny−1) + ad,ny · z−nyU(z), (2.6)

with Y (z) and U(z) being the z-transformed of the output and input signals, respectively.Alternatively the difference equation can be expressed in state-space form:

x(t+ ts) = Ad · x(t) +Bd · u(t) (2.7)

y(t) = Cd · x(t) +Dd · u(t), (2.8)

with Ad, Bd, Cd, Dd being the discrete system matrices. Given a model in continuoustime like equation 2.2, an important task is to find the equivalent (or approximated) modelgiven by equation 2.6. The exact discretization of a model is difficult in practice, on theone side because the transformation defined in equation 2.6 is not easy to solve, and onthe other side because the signal information between the sampling intervals is lost. Someapproximation methods or assumptions are therefore required. In system identificationthe tracking of the discretization equations may be useful, in order to make possible theback-transformation of the identified model in continuous time, and so to estimate thephysical value of the parameters.



2.1.2.1 Approximation of the derivatives

A first method to estimate the discrete model from the continuous time equations is toapproximate the derivatives in equation 2.1. In fact using the definition of the derivative:

x(t) = limδ→0

x(t)− x(t− δ)

δ(2.9)

for enough small sampling times the approximation of this equation can be performed inmany ways. The most important are the Euler (backward or forward) and the Tustinmethod.

Time domain approximation z-Transform

Backward Euler transform: x(t) ∼= x(t)− x(t− ts)

ts

1− z−1

ts

Forward Euler transform: x(t) ∼= x(t+ ts)− x(t)

ts

z − 1

ts

Tustin transform:x(t+ ts) + x(t)

2∼= x(t+ ts)− x(t)

ts

2

ts

z − 1

z + 1

The Euler methods approximate the derivative by calculating the slope of two samples,the backward transform uses the past sample and forward transform the future sample.Furthermore the Tustin method approximates the slope of two samples x(t) and x(t− ts)with the mean value of the two derivatives at the times t and (t − ts). To reproduce

t t+ tst− ts

x(t− ts)

x(t)

x(t+ ts)

x(t)

˙x(t)

t t+ tst− ts

x(t)

˙x(t)

t t+ tst− ts

x(t)

˙x(t)

x(t− ts)

Euler Backward Euler Forward Tustin

Figure 2.1: Comparation between approximations of the derivative for time-discrete sys-tems

precisely the continuous time model using the derivative approximations, the choice of anenough small sampling time is required. As simple rule the sample frequency should beat least 5 times higher as the cross-over frequency of the model. In any case these modeltypes contain a stationary error given by the approximated integration.

2.1.2.2 Exact solution with Zero-Order-Hold sampling

The best discretization approach is given solving the differential equations 2.1 or 2.3 andcalculating the recursive form of the differential equation from the solution. The generic


2.2. Signals and proprieties System identification (DRAFT)

solution of the state space representation is:

x(t) = eA·(t−t0) · x(t0) +B ·∫ t

t0

eA·(t−τ) · u(τ) dτ. (2.10)

Here is assumed that the signal u(t) is known for all times. To develop the discrete-timemodel, only the signal values at the sampling times are required. The upper equationcan be simply transformed in recursive form, substituting the times t = (k + 1) · ts andt0 = k · ts. Keeping additionally the input signal piecewise constant with a zero order holdelement (u(t) = u(k · ts)), the integral can be further simplified, leading the solution:

x((k + 1) · ts) = eA·ts · x(k · ts) +∫ ts

0eA·(t−τ) dτ ·B · u(k · ts). (2.11)

The solution of the integral can be calculated if the matrix A is invertible:

x((k + 1) · ts) = eA·ts · x(k · ts) +A−1(eA·ts − I)B · u(k · ts), (2.12)

using the propriety:

d

dteA·t = A · eA·t = eA·t ·A (2.13)

2.2 Signals and proprieties

This chapter will give a brief introduction to the common signal transformations withits proprieties and the most important signal types. While the system identification isbased on discrete and finite signals, the relation to time continuous systems will also bementioned, to highlight the connection with the control theory.Generally the signals acting in a system can be subdivided into different classes:

• Useful signal: Contain the information to characterize the process.

• Auxiliary signal: Are used for example to excite a system or are correlated withanother signal, in order to extract the information of the process.

• Disturbances: Are generally the unwanted part of the process, because add uncer-tainty in the information of the useful signals.

yeff(t)System

SActuation

uref(t)Measure

ymeas(t)ueff(t)

Disturbance

Figure 2.2: Signal effects in a system.



2.2.1 Description of the signals

The signals can be described generally in time domain or in frequency domain. Thedescription depends further upon the fact if the signal is deterministic or stochastic. Thedifference between these two signal types is that a deterministic signal can be describedmathematically at each time and is fully predictable, otherwise a stochastic signal is casualand is also not predictable.

2.2.1.1 Deterministic signals

The deterministic signal is a perfectly known and in continuous time domain can bedescribed by an arbitrary function of the time:

u(t) = f(t). (2.14)

For the case of a time discrete signal, the values can represented only for the samplingtimes. During the sampling period the signal is left generally constant (zero-order-hold,ZOH), but depends on the actuation or measuring method:

u[k] = f [k]. (2.15)

A sampled signal with a constant sampling time ∆t has the notation:

x[k] = x(k ·∆t)

In order to represent the older values the backward shift operator q−1 is introduced:

x[k − nq] = x[k] · q−nq.

The parameter nq represents the time shift. In control theory the z-Transform is usedrather, which represents a projection of the Laplace transform. A z-transform is a transferfunction that represent the dynamic behavior of the system and is not dependent on theinitial signal values. As difference the backward shift operator contains also the informa-tion of the initial states of the signal. As example the relation:

y[k] + a · y[k − 5] = u[k]− a · u[k − 5]

y[k](1 + a · q−5) = u[k](1 + a · q−5)

cannot be simplified. Contrariwise the content under the parenthesis can be simplifiedusing a z-transform. In the field of system identification q−1 will be used.The mean characteristics of the deterministic signal can be used for system identificationpurposes. It may be firstly useful to differentiate if the signal has a finite energy or a finitemean power. For the case of finite energy, following propriety holds:

0 ≤∞∑

k=−∞u[k]2 < ∞, (2.16)

this means that the signal is transitory, appears and fades away at a finite time. For thecase of a finite mean power signal, we have that:

0 < limN→∞

1

2N + 1

N∑

k=−N

u[k]2 < ∞, (2.17)



that is the sum converges to a value greater than zero. For example the most periodicor pseudo random permanent signals have a finite mean power. It can be noticed, that afinite power signal cannot be a finite energy signal and vice versa.Further, some important transformations of the deterministic signals are defined:

Mean value: µu =1

N

N−1∑

k=0

u[k]

Autocorrelation function: Ruu[h] = S ·N−1∑

k=0

u[k] · u[k − h]

Autocovariance function: Cuu[h] = S ·N−1∑

k=0

(u[k]− µu) · (u[k − h]− µu)

with S being an arbitrary weighting factor. The auto-correlation represents in generalterms the correlation of the signal with itself, while the auto covariance characterizes thevariation of the signal in respect of its mean value.For h = 0 the correlation function represents the mean value of the signal power or thesignal energy (integration of the power), in dependance of the choice of the factor S:

Mean value (e.g. power signals): S =1

N

Integration (e.g. energy signals): S = 1

Varying the time delay h the dependency of the signal on the past values can be extracted.The calculation is done practically correlating the signal u with the same signal shifted bya time delay h. It’s interesting to notice, that for periodic signals the correlation functionremains periodic and for signals with a DC content Ruu[h] contains also an offset.

Example: Correlation of a periodic and transient signal

Let define a deterministic signal u[k] with a periodic (sine) and a transient (rectangularfunction) component:

u[k] = uΠ[k] +A0 sin(ω0k),

with:

uΠ =

1, for k1 < k < k20, else



0 50 100 150−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4Signal u[k]

u[k]

Time k [normed]

The autocorrelation is calculated according the upper equations:

Ruu[h] =1

N·N−1∑

k=0

u[k] · u[k − h]

=1

N·N−1∑

k=0

(uΠ[k] +A0 sin(ω0k)) · (uΠ[k − h] +A0 sin(ω0(k − h))

=1

N·N−1∑

k=0

(uΠ[k] · uΠ[k − h] +A0 sin(ω0k) · uΠ[k − h] + uΠ[k] ·A0 sin(ω0(k − h))

+A0 sin(ω0k) ·A0 sin(ω0(k − h)).

Assuming for simplicity that k2 − k1 corresponds to a number of integer periods of thesinus function, the second and third terms equals to 0. The first term represents thecorrelation of the rectangle function.

1

N·N−1∑

k=0

uΠ[k] · uΠ[k − h] =1

N·

k2∑

k=k1

uΠ[k] · uΠ[k − h]

=

|∆k−h|N

, for −∆k < h < ∆k0, else.

having ∆k = (k2− k1). Finally, for the last term the trigonometric identities can be usedto solve the summation:

1

N·N−1∑

k=0

A0 sin(ω0k) ·A0 sin(ω0(k − h)) =A2

0

2 ·N ·N−1∑

k=0

cos(ω0h)− cos(2ω0(k − h)),

The second cosine in the summation can be neglected if N → ∞, and therefore thecorrelation function of u can be approximated with:

Ruu[h] ∼=

|∆k−h|N

+A2

0

2cos(ω0h), for −∆k < h < ∆k

A2

0

2cos(ω0h), else.



−150 −100 −50 0 50 100 150−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Autocorrelation function Ruu

[h]

Ruu

[h]

Time delay h [normed]

In summary the correlation function of the signal u[k] contains a transient componentgiven by the rectangular function and a sinusoidal component with the same frequency ofthe original signal. Extending the example with an input containing sum of finite sinu-soidal functions with different frequencies and amplitudes the correlation can be found tobe a function containing the same same frequencies.

Similarly, the cross correlation and cross covariance between two different signals uand y is defined as:

(Cross-) Covariance function: Cuy[h] = S ·N−1∑

k=0

(u[k]− µu) · (y[k − h]− µy)

(Cross-) Correlation function: Ruy[h] = S ·N−1∑

k=0

u[k] · y[k − h]

A more detailed analysis of the correlation function proprieties will be performed in thenext section.

2.2.1.2 Stochastic signals

Contrarily to deterministic signals, a stochastic signal is generated by a random processthat cannot be entirely described mathematically. The evolution of the random signal canonly be defined stochastically, with probability functions or expected values. These signalsare generally associated with noise and disturbances, but due to their useful proprietiesthese can be generated using approximation functions (pseudo random signals), and arewidely used for identification purposes.



Let’s firstly define a probabilistic sample space Ω to be the set of all possible outcomesof an experiment. A numerical measured event, which provides a function mapping fromΩ to ℜ, is defined as random variable ξ. The random variables describe the casual partof the process, for which a probability function can be associated. Furthermore, a samplespace can contain a set of n random variables ξ1, ξ2,.. ξn, that can eventually belong to asubset of Ω.

Example: Sample spaces

An electronic component manufacturer is producing a large amount of resistances. Thecomponents are manufactured by two machines, that have been calibrated differently.Thus, the resistances of the first machine differ from those produced by the second ma-chine. Additionally, both machines produce the resistances with some uncertainty thatcan be described by a probabilistic function. We want to describe this situation stochas-tically.Let’s define the sample space Ω as the ensemble of all resistances of 1 [kOhm] fabricated

Ω

ξ2ξ1

Ω1 Ω2

Machine 2Machine 1

Components

produced by

machine 1

Components

produced by

machine 2

Figure 2.3: Space set and random variables for a production chain with two machinesproducing the same components

by the manufacturer. Then the first machine producing the components belongs to asubspace Ω1 of the sample space and the second machine to another subspace Ω2. Therandom variables ξ1 and ξ2 provide the effective resistance value of the components, withthe corresponding probabilistic proprieties.

In system identification the random variables are used to describe mathematically astochastic signal:

v(t) = f(t, ξ). (2.18)

This is typically a function of the time. The random variable introduces a noise in thesignal that varies at each time value. The proprieties of the stochastic signal, such asmean value and variance may also change with the time. A set of stochastic signals canbe produced by the same or different stochastic variables with different proprieties, anexplicative example is presented in figure 2.4.



Ω

ξ1

ξ2

ξ3ξ4

t

v1(t)

Ω1

Ω2

v2(t)

v3(t)

v4(t)

t

t

t

Figure 2.4: Space set and random variables

Probabilistic density functions

In general a probabilistic density function can be of any form, under the restriction thatthe overall area equals 1 (probability of overall outcomes). In this subsection only Gaus-sian distributions and distribution generated by quantization will be presented, becausethese are of particular interest for system identification in control application.The Gaussian distribution (or normal distribution) is an exponential symmetric shape.The main advantage of this distribution is that a linear operation on normal distributedvariables delivers again a normal distributed probability shape. This propriety simplifiesthe calculations for a system identification. The Gaussian distribution is defined with theequation:

p(x) =1

σx√2π

e− (x−µx)2

2σ2x ,

with σx being the standard deviation of x (Varx = σ2x) and µx the mean value. Note that

the normal distribution can be fully represented with the standard deviation and meanvalue and can be defined with the notation N (µx, σx). The generic Gaussian densityfunction for a nx-dimensional random vector is given by the formula

N (µx,Cx): p(x) = (2π)−nx/2 · |Cx|−1/2 · exp(

−(x− µx)TΣ−1

x (x− µx))

.

The matrix Cx represents here the covariance matrix and the vector µx represents theaverages of the vector x.



−3σx −2σx −σx µx +σx +2σx 3σx

68.2%

95.4%

99.7%

p(x)

x

0.1

0.2

0.4

0.3

Figure 2.5: Gaussian distribution

Example: Bidimensional normal density function

The probability density function of two normal distributed random signals v1 and v2 canbe determined, according the generic Gaussian formula:

pv1,v2(x1, x2) =

1

2π√

c1c2 − c212

exp

(

− c1c22(c1c2 − c2

12)

(

(x1 − µ1)2

r1− 2c12

c1c2(x1 − µ1) · (c2 − µ2) +

(c2 − µ2)2

c2

))

,

with the covariance matrix: Cv1v2 =

[

c1 c12c12 c2

]

.

The variables µ1 and µ2 are the mean values of v1 and v2, respectively. If the cross co-variance of the two signals is nonzero, there are a dependency between the two signals.This means, as shown in upper equation, that with c12 6= 0 the value of the first signalinfluences the value of the second signal. The figure 2.6 shows the difference between twocorrelated and uncorrelated variables, using following numerical values:

Mean value of v1: µv1 = 1.0Mean value of v2: µv2 = 0.5

Correlated signals: Rv1v2 =

[

1.0 0.50.5 0.5

]

Covariance matrix uncorrelated example: Rv1v2 =

[

1.0 0.00.0 0.2

]



µv1

µv2

-4 -3 -2 -1 0 1 2 3 4-3

-2

-1

0

1

2

3

-4 -2 -1 0 1 2 3 4-3

-2

-1

0

1

2

3Normal distribution of correlated variables Normal distribution of uncorrelated variables

v1

v2

v1-3

v2

µv2

µv1

Figure 2.6: Gaussian distribution of two variables. Left plot: correlated variables, rightplot: uncorrelated variables

Furthermore the stochastic approximation of the signal quantization will be introduced.In fact the quantization effect is common in control applications, due to the usage of A/Dconverters.Assuming that the AD converter has a quantization interval of ∆Q and the rounding strat-egy corresponds the nearest value, the probabilistic density function of the quantizationcan be modeled as a rectangular shape, with intervals [−∆Q/2 ∆Q/2].

Ensemble averaging

The ensemble averaging is the computation of the mean proprieties of a random variable.This calculation can be done only knowing the associated probabilistic density function.The mean value or expected value (first order moment) and the correlation (second ordermoment) of a stochastic signal can be calculated as follows:

Mean value: µv = E v =

∫ ∞

−∞x · pv(x) dx

Correlation value: Rv1,v2 = E v1 · v2 =

∫ ∞

−∞

∫ ∞

−∞x · y · pv1,v2(x, y) dx dy

with pv(x) being the probability density function of the random variable associated to vat the time value k and with pv1,v2(x, y) being the two dimensional probability functionof the variables v1 and v2. From the upper definitions the covariance function can beobtained:

Covariance value: Cv1,v2 = E (v1 − µv1) · (v2 − µv2) = Rv1,v2 − µv1 · µv2

Note that the averaging functions are defined in a generic way, that is the variables v1and v2 may be two distinct stochastic signals or alternatively v1 and v2 may express two



signals in function of the time v1 = u[k] and v2 = y[k−h]. In this case we can rewrite thecorrelation functions in function of the time:

Correlation function: Ru,y(t, τ) = E u(t) · y(τ)

Covariance function: Cu,y(t, τ) = E (u(t)− µu(t)) · (y(τ)− µy(τ))

Furthermore we may be interested calculating the correlations for a vector containing nsignals v = [v1, v2, . . . , vn]

T. For this purpose the correlation matrices are used:

Correlation matrix: Rv =

Rv1,v1 Rv1,v2 · · · Rv1,vn

Rv1,v2 Rv2,v2 · · · Rv2,vn...

.... . .

...

Rv1,vn Rv2,vn · · · Rvn,vn

Time averaging

If the signal of a realization remains stationary over the time, the mean values of thesingle realization generated by a stochastic variables can be approximated calculating atime average, like in section 2.2.1.1.

Ergodicity

A system is defined ergodic if every realization is typical and sufficient to describe thesample space. This implicates that each realization is generated by the same samplespace. Moreover the sample space of an ergodic system is univocal and cannot be subdi-vided into subspaces.An important propriety is that for an ergodic and stationary signal the expectedvalues can be approximated using time averages:.

Mean value: µv = E v ∼= 1

N·N−1∑

k=0

v[k]

(Cross-) Covariance function: Cv1,v2[h] = E (v1[k]− µv1

)(v2[k − h]− µv2)

∼= 1

N·N−1∑

k=0

(v1[k]− µv1) · (v2[k − h]− µv2

)

(Cross-) Correlation function: Rv1,v2[h] = E v1[k]v2[k − h]

∼= 1

N·N−1∑

k=0

v1[k] · v2[k − h]

Stationary and ergodic systems are generally really important, because measuring for asufficient long time, the stochastic proprieties of the variables can be determined. For the



special case, where the variables v1 and v2 are independent, then the correlation functionis:

Rv1,v2 [h] = E v1[k] · E v2[k − h] , (2.19)

as direct consequence the covariance function of two independent variables is zero:

Cv1,v2 [h] = 0, with v1 and v2 independent.

In this case the two variables are said to be uncorrelated. Tf v1, v2 are independent withmean value zero:

Rv1,v2 [h] = 0,

2.2.2 Signal examples

2.2.2.1 Dirac impulse

The Dirac impulse is a theoretical signal and is defined as single impulse at the time0 with energy 1. This signal is practically not generable in continuous time domainbecause requires an infinite power, but is useful for the theoretical analysis of discreteand continuous systems in the frequency domain. The time function is defined as follows:

δ(t) = limǫ→0

1ǫ , for 0 < t < ǫ0, for t = 0 and t ≥ ǫ

. (2.20)

The integral of the Dirac impulse over the time is defined to be equal 1:∫ ∞

−∞δ(t)dt = 1 (2.21)

A useful property are the sifting propriety. Having an arbitrary integrable function f(t)

1

ǫ

tǫ

δ

Area = 1

00

ω

Fδ

1

0

Figure 2.7: Definition of a Dirac-impulse with ǫ tending to 0and its Fourier transform.

following identities are can be demonstrated from the upper definitions:∫ ∞

−∞f(t) · δ(t− T )dt = f(T ) (2.22)

∫ ∞

−∞f(τ) · δ(t− T − τ)dt = f(t− T ), (2.23)



tT

δ(t− T )

0

t0

t0

*f(t)

f(t− T )

Figure 2.8: Sifting propriety of the convolution with Dirac impulse

using the propriety δ(t) = −δ(−t). This equality provides also the multiplicative identityand commutative propriety of the convolution:

δ(t) ∗ f(t) = f(t) ∗ δ(t) :=

∫ ∞

−∞f(τ) · δ(t− τ)dt = f(t− T )

It can be demonstrated, that the Fourier Transform of the Dirac Impulse equals a constant,that is the all frequencies are excited with the same amplitude:

Fδ =

∫ ∞

−∞exp−jωt ·δ(t)dt = 1. (2.24)

Calculating the inverse Fourier Transform, following equality results:

δ(t) = F−11 =1

2π

∫ ∞

−∞expjωt dω. (2.25)

The fact that Fδ equals 1 for all frequencies is very useful in system identification, be-cause a linear system could be theoretically entirely identified (remember that the responseof a linear system is a signal containing the same frequencies, but different amplitude andphase). However, in practice the Dirac impulse cannot be reproduced because tends toinfinity for t → 0. The discrete analog of the Dirac impulse is generally used instead. Thissignal is defined as the Kronecker-Delta, for all j and k ∈ Z:

δj,k =

1, for j = k0, for j 6= k

. (2.26)

The sifting propriety is satisfied for all integer values i and k, as in the continuous case:

∞∑

i=−∞ai · δi,k = ak. (2.27)

Unfortunately, discretizing the impulse signal, the frequency excitation decreases and thecapability to estimate a system is thus reduced.



2.2.2.2 Step function

The step function is a discontinuous function, with values equal 0 for t ≤ 0 and equal 1 forpositive times. This signal is used in system identification for the analysis of the transientbehavior of a system and stationary response.

u(t) =

0, for t ≤ 01, for t > 0

(2.28)

The Fourier transform of the unit step function is given by:

Fu(t) = πδ(ω) +1

jω(2.29)

It’s interesting to notice that the derivative of the step response corresponds to the Dirac

1

t

u

0

0ω

|Fu|

0

Figure 2.9: Unit step function and Fourier transform.

impulse, defined in the previous subsection:

d

dtu(t) = δ(t) (2.30)

Analogally the discrete form of the unit step function can be defined, for all k ∈ Z:

uk =

0, for k ≤ 01, for k > 0

, (2.31)

and its difference function is equal to the discrete Dirac impulse:

uk − uk−1 = δ0,k. (2.32)

2.2.2.3 Constant value

The constant value can be used in identification only to analyze the statically response ofthe system, because excites the single frequency 0. In fact the Fourier transform can becalculated using the definition of the Dirac impulse:

Fc = 2π · c · δ(ω). (2.33)



2.2.2.4 Sinusoidal function

The generic sinusoidal function is defined as follows:

f(t) = sin(ωt+ ϕ). (2.34)

2.2.2.5 White noise

The white noise is a theoretical stochastic signal, which is fully uncorrelated in the timescale and with other signals. This means that the generated signal for a time t is fullyindependent from the previous values. White noise is modeled as normal distribution withzero mean and infinite variance (infinite power):

µe = Ee(t) = 0

Ree(t− τ) = Ee(t) · e(τ) = σ2e · δ(t− τ) (2.35)

The fact that the variance is infinite can be explicated from the fact that otherwise theenergy of the signal would be zero. In nature there is no sources of white noise, becauseit requires infinite power.The time discrete white noise can be described as normal distribution with zero mean andvariance σ2

e , using a normalized sampling time of 1. The expected values for the discretesignal are:

µe = Ee[k] = 0

Ree[k − j] = Ee[k] · e[j] = σ2e · δj,k, (2.36)

or using the notation of the Gaussian distribution e[k] ∼ N(

0, σ2e

)

. For the discrete signalthe variance becomes finite and expresses the mean signal power. As in the continuoussignal, also with the discrete withe noise there is no correlation between samples. In factthe estimated correlation function from a measure in figure 2.12 show that there is notime dependency . The figure 2.10 show the time plot and the estimated distributionfrom the measures of two white noise signals with different powers. The typical Gaussiandistribution can be identified in the right plot.

2.2.2.6 Colored noise

The noise may acquire a correlation in the time, if a filter is applied on it. There existmany types of colored noise, some example is proposed below.

Simple time correlation

If the filter calculates the output as function of past input signals (finite impulse response)the correlation is finite. For a colored noise of the type:

v[k] = e[k] + c0 · e[k − 1],

where e is a stationary white noise with variance σ2e , the expected value can be calculated

as:

Ev[k] = Ee[k] + c0 · e[k − 1] = Ee[k]+ c0 · Ee[k − 1] = 0,



0 2000 4000 6000 8000 10000−5

−4

−3

−2

−1

0

1

2

3

4

5Time plot of white noise

t [samples]

e

0 0.5 1 1.5 2

x 104

−5

−4

−3

−2

−1

0

1

2

3

4

5Distribution of the values

Distribution

Val

ue o

f e

White noise with σ

e = 1

White noise with σe = 0.35

Figure 2.10: Comparation of two discrete white noises with different variance. Left: timeplot. Right: density function.

−30 −20 −10 0 10 20 30

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time shift [#]

Cor

rela

tion

Correlation function

White noise with σ

e = 1

White noise with σe = 0.35

Figure 2.11: Comparation of the correlation function for a discrete white noises withdifferent variances.

because the mean value of e is for definition equal zero. The correlation function is:

Ev[k] · v[j] = E (e[k] + c0 · e[k − 1])((e[j] + c0 · e[j − 1])= E e[k]e[j] + c0 · (e[k − 1]e[j] + e[k]e[j − 1]) + e[k − 1]e[j − 1]



Choosing a time shift h = k − j = 0 the variance becomes:

Ev[k] · v[k] = E

e[k]e[k] + c0 · (e[k − 1]e[k] + e[k]e[k − 1]) + c20e[k − 1]e[k − 1]

= σ2e(1 + c20).

For the case, where h = k − j = 1 the covariance function is:

Ev[k] · v[k − 1] = E

e[k]e[k − 1] + c0 · (e[k − 1]e[k − 1] + e[k]e[k − 2]) + c20e[k − 1]e[k − 2]

= σ2

e · c0.

This result can be also obtained choosing the time shift h = −1. It can be simplydemonstrated that for other cases (h > 1 or h < −1) the covariance function is zero.

−30 −20 −10 0 10 20 30

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Time Shift h

Cor

rela

tion

func

tion

Correlation function of colored noise v(t) = e(t) + c0*e(t−1), with c0=0.8

Figure 2.12: Comparation of the correlation function for a discrete colored noises using afinite impulse response filter.


Chapter 3

Non Parametric Identification

Methods

Non-parametric methods describe the behavior of the system on the basis of graphicalcurves or tables. A main advantage is that the structure of the system is not requiredand has not to be defined during the identification. Nevertheless, the structure can beestimated from the nonparametric analysis and parameters can be extracted interpretingthe resulting graphical curves.The non parametric identification can be performed on the time domain (transient anal-ysis or correlation method) and in the frequency domain (frequency analysis or spectralmethod).

3.1 Transient analysis

The transient analysis extraction of model information from the system response at agiven excitation signal.

3.1.1 Step response

Analyzing the plot of a unitary step response, information about the system can be ex-tracted. For this purpose the solution of the differential equation system is required.

3.1.2 First order system

Let’s consider a delayed first order model with the transfer function:

G(s) =K

τs+ 1e−s∆T , (3.1)

with the parameter K being the static gain, τ the time constant and T the time shift.The step response is then:

y(t) =

0, for t ≤ ∆T

K · (1− e−(t−∆T )/τ ), for t > ∆T

and the parameters can be directly estimated from the curve, as showed in figure XX.In particular, the time constant τ corresponds to the initial inclination of the output after


System identification (DRAFT) Chapter 3. Non Parametric Identification Methods

Time [s]

Steprespon

se[-]

Step response of a delayed first order system

K

∆T τ

Figure 3.1: Step response for a delayed firs order system

the dead time ∆T and the gain K is the value that is reached after the transient. Thetime constant may be difficult to be estimated, therefore it’s possible to transform theoutput response in a logarithmic scale:

log(y(t)) = log(K · (1− e−(t−∆T )/τ )

and introducing the auxiliary variable z(t):

z(t) = log(K − y(t))

= log(K · e−(t−∆T )/τ ) = log(K)− t/τ.

The time constant corresponds in this case to the slope of the curve after the dead time.The value of K is required.

3.1.3 Second order system

A second order system can be described by the transfer function:

G(s) =Kω2

0

s2 + 2 · ζω0 + ω20

. (3.2)

For the underdamped case (0 < ζ < 1) the solution of the step response is:

y(t) = K ·(

1− 1√

1− ζ2e−ζω0tsin

(

ω0

√

1− ζ2 · t+ arccos(ζ))

)

. (3.3)

The local extrema (maxima and minima) can be found setting the derivative of the step


3.1. Transient analysis System identification (DRAFT)

z(t)

Time [s]

Transformed first order step response (with semi-logarithmic scale)

∆T

δ

δtz(t) = −1/τ

Figure 3.2: Logarithmic plot of the step response for the time constant determination

Time [s]

Step response of a underdamped second order system

K(1 +M)

K

K(1−M2)

2K

t1 t2 t3 t4 t5 t6

ζ = 0

ζ = 0.15

ζ = 1

Figure 3.3: Logarithmic plot of the step response for the time constant determination

response in respect of the time equal zero:

tk = kπ

ω0

√

1− ζ2, k = 1, 2, ..



and thus:

y(tk) = K · (1− (−1)kMk).

where M is the overshoot given by:

M = e−ζ·π√1−ζ2 .

Firstly, the parameter K can be determined from the steady state response (for t → ∞).The parameter ζ can be determined from the overshoot M , which is estimated using thefirst local maximum:

M =y(t1)

K− 1,

and:

ζ =− log(M)

√

π2 + (log(M))2.

From the period of the step response the frequency ω0 can be estimated:

T = tk − tk−1

=2π

ω0

√

1− ζ2, (3.4)

and so ω0 is:

ω0 =2π

T√

1− ζ2=

2

T

√

π2 + (log(M))2 (3.5)

3.2 Impulse response

Instead of analyzing the impulse response from the the output response curve as in theprevious section, a method that takes advantage of the important proprieties of the impulsefunction and the linearity proprieties of linear systems is presented.For this script only linear or linearized systems are considered. A system is linear only if:

y = F (α1 · u1 + α2 · u2) = α1 · F (u1) + α2 · F (u2),

where α1 and α2 are constant parameters, u1 and u2 are signals in function of the timeand F (·) is the linear function. In general terms the function F expresses a mapping fromu1, u2 to y and may contain a differential equation or a difference equation. For a timefunction the input signal is firstly represented as sum of the single samples. The samplingof a continuous signal is done using the sifting propriety of the impulse response:

∫ ∞

−∞u(t) · δ(t− τ)dt = u(τ),


3.2. Impulse response System identification (DRAFT)

remembering that δ(t) is a Dirac impulse with area 1:∫∞−∞ δ(t− τ)dt = 1. Replacing the

representation of the input signal as sum of infinite samples in the linear system equationthe output is:

y(t) = F (u(t)) = F

(∫ ∞

−∞u(τ) · δ(t− τ)dt

)

=

∫ ∞

−∞F (u(τ)δ(t− τ))dτ =

∫ ∞

−∞F (δ(t− τ)) · u(τ)dτ (3.6)

This means, that the output y(t) of a linear model can be represented as infinite sum ofweighted impulse responses, where the weight is given by the input signal sample at thetime τ . The impulse response can therefore be defined as:

g(t, τ) ≡ F (δ(t− τ))

If the system is stationary, the impulse response is independent from the time t and thuscan be shifted by an arbitrary parameter α

g(t, τ) = g(t− α, τ − α) = g(t− τ, 0).

Last equality has been obtained choosing α = τ . In the case, where the system is stationarythe notation g(t− τ) is used. For a linear and stationary system the output response canbe calculated as:

y(t) =

∫ ∞

−∞g(t− τ) · u(τ)dt

= g(t) ∗ u(t) = u(t) ∗ g(t), (3.7)

with g(t) being the impulse response of the system.


Chapter 4

Parametric Identification Methods

4.1 Least Squares Method

Let’s consider a process affected with an unknown disturbance whose model has to beidentified. Firstly a model structure is required to delimitate the system. A simple wayto represent the structure of a system is to use the difference equation, as introduced insection 2.1.2. Considering that the order of the process is known but the parameters areunknown, the model can be described by the equation:

y[k] + a1 · y[k − 1] . . .+ any · y[k − ny] = b0 · u[k] + b1 · u[k − 1] + . . .+ bnu · u[k − nu] + e[k],

(4.1)

where y is the measured output response, u the input stimulus and a1, a2. . .any, b1,b2,. . .,bnu are the unknown parameters. The model order nu and ny is assumed to beknown. The disturbance affecting the system is modeled by the random variable e. Forthe moment e is assumed to be a white noise with variance σ2

e . As the equation 4.1 islinear in the parameters, the signals and parameters can be separated, bringing the modelequation the in vectorial form:

y[k] =[

−y[k − 1] . . . −y[k − ny] u[k − 1] . . . u[k − nu]]

·

a1...

anyb1...

bnu

+ e[k]

The model can also be represented using a linear regression:

y[k] = ϕT[k] · θ+ e[k]

with: ϕT[k] =[

−y[k − 1] . . . −y[k − ny] u[k − 1] . . . u[k − nu]]

θ =[

a1 . . . −any b1 . . . bnu]T

and e a white noise with:

Ee[k] = 0Ee[k][j] = σ2

e · δk,j


System identification (DRAFT) Chapter 4. Parametric Identification Methods

with the variables:

Regressor: ϕT [k] ∈ ℜ1×nθ

Parameter vector: θ ∈ ℜnθ×1

Kronecker-delta: δk,j = 1 for k = j, 0 otherwise

Parameter number: nθ = ny + nu

Expected value: E.

Note that the regressor contains the input and internal signals, while θ is contains severalunknown constant parameters to be identified. In the deterministic case, where no dis-

+ϕT[k] · θ

u[k]

e[k]

y[k]

Model

Figure 4.1: Model using a linear regression. Note that the disturbance is modeled in theprocess, not in the output

turbance is affecting the model and the system structure corresponds exactly to a linearregression, nθ measures would be enough to estimate unequivocally the system parameters.The fact that the measures are disturbed by an unknown noise e[k] causes an uncertaintyin the parameter estimation. An approach to reduce the uncertainty is to average a longermeasure exploiting the measure redundancy. Let’s define a measure of N samples:

y = φT · θ+ e

y[1]y[2]......

y[N ]

=

ϕT[1]ϕT[2]

...

...ϕT[N ]

·

θ1θ2...

θnθ

+

e[1]e[2]......

e[N ]

with:

Regression matrix: ϕT ∈ ℜN×nθ

Output vector: y ∈ ℜN×1

Output vector: θ ∈ ℜnθ×1

Disturbance vector: e ∈ ℜN×1

For simplicity the parameters a1, . . ., any, b1, . . ., bnu have been substituted by θ1, θ2. . .θnθ in the parameter vector.Supposing that the real process can be entirely described by the linear regression:

y = φT · θ0 + e,


4.1. Least Squares Method System identification (DRAFT)

where the vector θ0 is the real (to be identified) parameter vector. Supposing further thatthe process structure is known, the output can be estimated by the linear regression givenby:

y(θ) = φT · θ,

where θ is the estimated parameter vector. The model error vector ǫ is then:

ǫ(θ) = y − y(θ) = y − φT · θ

ǫ[1]ǫ[2]......

ǫ[N ]

=

y[1]y[2]......

y[N ]

−

ϕT[1]ϕT[2]

...

...ϕT[N ]

·

θ1θ2...

θnθ

.

The equation errors ǫ[k] are called residuals.The task of system identification is to find an estimate of the parameter vector θ such thatthe error ǫ between model and process is minimized and such that the estimated parametersconverge to the real parameter vector θ0. To solve this problem mathematically, a convexloss function that minimizes the residuals is required. A simple way is to choose thequadratic function:

VN (θ) =1

2

N∑

k=1

ǫ2[k] =1

2ǫT(θ) · ǫ(θ). (4.2)

The minimization problem solution is obtained setting the derivative of the loss functionin respect of the parameter vector θ to zero:

d

dθVN (θ) =

1

2

(

−2φ[y − φTθ])

0 = −φy + φφTθ. (4.3)

The first equality has been found using the matrix derivation propriety:

d(b + B · x)T(d + D · x)dx

= BT(d + D · x) + DT(b + B · x),

with b and d being constant vectors, B and D being matrices and x a parameter vector.Solving the equation 4.3 in respect of θ the parameter vector minimizing equation 4.2 canbe found:

θLS =(

φ · φT)−1

φ · y

The expression(

φ · φT)−1

φ is called pseudoinverse.


System identification (DRAFT) Chapter 4. Parametric Identification Methods

4.1.1 Proprieties of the LS estimate

The convergence and accurateness of the Least Squares estimate is studied in this section.Firstly an analysis of the expected value of the parameter vector is performed. Ideally onewould expect that the estimated parameters converge to the real parameter vector θ0. Toverify this assumption the equations of the real process are replaced in the Least Squaressolution and the expected value is calculated:

EθLS = E

(

φ · φT)−1

φ · y

= E

(

φ · φT)−1

φ · (φT · θ0 + e)

= E

θ0 +(

φ · φT)−1

φ · e

= θ0. (4.4)

Last equality is valid only if e and the pseudoinverse are uncorrelated. The fact, that e isa white noise is sufficient to remove any correlation. Further the estimation uncertainty isanalyzed. To do this the covariance matrix is calculated using the solution found above:

E

(θLS − θ0)T(θLS − θ0)

= E

(

φ · φT)−1

φ · e · eT · φT(

φ · φT)−1

= σ2e ·(

φ · φT)−1

.


Chapter 5

Appendix

5.1 Fourier Transform

The Fourier transform is the representation of a signal in frequency domain and is definedas:

F (ω) =

∫ ∞

−∞f(t) · exp−jtω dt. (5.1)

The inverse can be calculated as:

f(t) =1

2π

∫ ∞

−∞F (ω) · expjωt dω, (5.2)

with ω being the frequency in [rad/s]. Further some useful propriety of the Fourier trans-form will be introduced. The convolution in time domain corresponds to the multiplicationin frequency domain, that is:

f1(t) ∗ f2(t) =∫ ∞

−∞f1(τ) · f2(t− τ)dτ F1(ω) · F2(ω)

The digital transformation corresponds to the discrete transformation

Ffs(t) = F ∞∑

k=−∞f(t)δ(t− kT )

=∞∑

k=−∞f(kT ) · e−j·ωTk

= F (ejω′

) =∞∑

k=−∞f [k]e−jω′k, (5.3)

with T being the sampling period, δ the Dirac impulse, ω the frequency in [rad/s] and ω′ =ωT a normalized frequency that allows to represent the Fourier transform independentlyfrom the sampling time. The inverse digital Fourier transform is:

f [k] =1

2π

∫ π

−πF (ejω

′

)ejω′kdω′. (5.4)

Due to the discrete nature of the signal in time domain, the Fourier transformed F (ejω′

)results to be periodic in ω′ with period 2π.


System identiﬁcation: methods and algorithms...

Documents

Transcript of System identiﬁcation: methods and algorithms...