3: Random Variables, Processes, and Noise2016/01/31 · tube or solid-state device. Flicker...

Introduction Random Variables Random Processes Noise Characterization

3: Random Variables, Processes, and Noise

Y. Yoganandam, Runa Kumari, and S. R. Zinka

Department of Electrical & Electronics EngineeringBITS Pilani, Hyderbad Campus

August 10 & 12, 2015

3: Random Variables, Processes, and Noise Communication Systems, Dept. of EEE, BITS Hyderabad

Outline

1 Introduction

2 Random Variables

3 Random Processes

4 Noise Characterization

Outline

1 Introduction

2 Random Variables

3 Random Processes

Inputsignal

Inputtransducer Transmitter

Transmittedsignal

Channel

Recivedsignal

Reciver

Outputsignal

Outputtransducer

Outputmessage

Inputmessage

Distortionand

Noise can be internal too ...

Noise is the undesired signal that gets added to the desired signal andreaches the destination.

Noise ultimately determines the threshold for the minimum signal that canbe reliably detected by a receiver.

Inputsignal

Transmittedsignal

Channel

Recivedsignal

Reciver

Outputsignal

Outputtransducer

Outputmessage

Inputmessage

Distortionand

Inputsignal

Transmittedsignal

Channel

Recivedsignal

Reciver

Outputsignal

Outputtransducer

Outputmessage

Inputmessage

Distortionand

Inputsignal

Transmittedsignal

Channel

Recivedsignal

Reciver

Outputsignal

Outputtransducer

Outputmessage

Inputmessage

Distortionand

Noise lives with the desired signal. Neither amplification nor the filtering canalleviate the effect of noise on the desired signal.

The only way to keep away from the effects of noise is to see that less amountof noise, relative to the desired signal, is present at the destination.

External Noise - Sources

• Noise from stars including the sun (20MHz – 1.5GHz)• Lightning (2MHz – 10MHz)• Thermal noise from the ground• Cosmic background noise from the sky

• Man made noise, e.g., spark plugs, engine noise, etc (1MHz – 500MHz)• Radio, TV, and cellular stations• Wireless devices• Microwave ovens

• Deliberate jamming devices

• Noise from stars including the sun (20MHz – 1.5GHz)• Lightning (2MHz – 10MHz)• Thermal noise from the ground• Cosmic background noise from the sky

• Man made noise, e.g., spark plugs, engine noise, etc (1MHz – 500MHz)• Radio, TV, and cellular stations• Wireless devices• Microwave ovens• Deliberate jamming devices

• Noise from stars including the sun (20MHz – 1.5GHz)• Lightning (2MHz – 10MHz)• Thermal noise from the ground• Cosmic background noise from the sky• Man made noise, e.g., spark plugs, engine noise, etc (1MHz – 500MHz)• Radio, TV, and cellular stations• Wireless devices• Microwave ovens• Deliberate jamming devices

Internal Noise - Sources

• Thermal noise is the most basic type of noise, being caused by thermalvibration of bound charges. It is also known as Johnson or Nyquistnoise.

• Shot noise is due to random fluctuations of charge carriers in an electrontube or solid-state device.

• Flicker noise occurs in solid-state components and vacuum tubes.Flicker noise power varies inversely with frequency, and so is oftencalled 1/f noise.

• Plasma noise is caused by random motion of charges in an ionized gas,such as a plasma, the ionosphere, or sparking electrical contacts.

• Quantum noise results from the quantized nature of charge carriers andphotons; it is often insignificant relative to other noise sources.

Desired vs Undesired Noise

In some cases, such as radiometers or radio astronomy systems, the desiredsignal is actually the noise power received by an antenna, and it is necessary

to distinguish between the received noise power and the undesired noisegenerated by the receiver system itself.

As you have seen, there are many types of noise sources. However, we willconcentrate more on thermal noise. Can you guess the reason?

Before proceeding further, let’s recap the theory of random variables andprocesses ...

Outline

1 Introduction

2 Random Variables

3 Random Processes

Random Variable

Outcome of a random experi-ment could be numerical or non-numerical.

Non-numerical example: coin toss-ing experiment – Head or Tail.

For non numerical outcomes, onecan assign certain numbers.

We may assign 1 for Head and -1for Tail.

When such numerical values areassigned to a variable, the variableis called a random variable.

For the coin tossing experiment,variable x can take either 1 or -1 de-pending on the outcome.

Probability of a random variable xtaking values xi is Px(xi).

Discrete random variable: If {xi}are distinct , then x is a discrete ran-dom variable, such that

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Random Variable

Px (xi) = 1.

Continuous Random Variables

Continuous RV can assume any value in a given interval.

Probability density function (PDF) px (x) is the appropriate definition for con-tinuous random variables.

Probability for a continuous RV is defined in terms of PDF as

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

And cumulative distribution function (CDF) is defined as

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

P (x1 ≤ x ≤ x2) =

px (x) dx. (1)

Of course, ˆ ∞

−∞px (x) dx = 1. (2)

Fx (x) = P (x ≤ x) =ˆ x

−∞px (x) dx. (3)

Statistical Averages – Means

For a continuous RV case, the mean is given by

x̄ = E [x] =ˆ ∞

−∞xpx (x) dx. (4)

Mean of a function (y = g (x)) of a random variable is

g (x) = E [g (x)] =ˆ ∞

−∞g (x) px (x) dx. (5)

The above expression can be generalized for two random variables is of arandom variable as

g (x, y) = E [g (x, y)] =ˆ ∞

−∞

ˆ ∞

−∞g (x, y) pxy (x, y) dx. (6)

x̄ = E [x] =ˆ ∞

−∞xpx (x) dx. (4)

g (x) = E [g (x)] =ˆ ∞

−∞g (x) px (x) dx. (5)

g (x, y) = E [g (x, y)] =ˆ ∞

−∞

ˆ ∞

−∞g (x, y) pxy (x, y) dx. (6)

x̄ = E [x] =ˆ ∞

−∞xpx (x) dx. (4)

g (x) = E [g (x)] =ˆ ∞

−∞g (x) px (x) dx. (5)

g (x, y) = E [g (x, y)] =ˆ ∞

−∞

ˆ ∞

−∞g (x, y) pxy (x, y) dx. (6)

x̄ = E [x] =ˆ ∞

−∞xpx (x) dx. (4)

g (x) = E [g (x)] =ˆ ∞

−∞g (x) px (x) dx. (5)

g (x, y) = E [g (x, y)] =ˆ ∞

−∞

ˆ ∞

−∞g (x, y) pxy (x, y) dx. (6)

Statistical Averages – Moments

The nth moment of a random variable x is defined as

ˆ ∞

−∞xnpx (x) dx. (7)

Similarly, the nth central moment of a random variable x is defined as

(x− x)n =

ˆ ∞

−∞(x− x)n px (x) dx. (8)

The second central moment of an RV x is called variance and denoted by σ2x ,

where σx is known as standard deviation.

σ2x = (x− x)2

= x2 − 2x2 + x2 = x2 − x2 (9)

ˆ ∞

−∞xnpx (x) dx. (7)

(x− x)n =

ˆ ∞

−∞(x− x)n px (x) dx. (8)

σ2x = (x− x)2

= x2 − 2x2 + x2 = x2 − x2 (9)

ˆ ∞

−∞xnpx (x) dx. (7)

(x− x)n =

ˆ ∞

−∞(x− x)n px (x) dx. (8)

σ2x = (x− x)2

= x2 − 2x2 + x2 = x2 − x2 (9)

ˆ ∞

−∞xnpx (x) dx. (7)

(x− x)n =

ˆ ∞

−∞(x− x)n px (x) dx. (8)

σ2x = (x− x)2

= x2 − 2x2 + x2 = x2 − x2 (9)

Statistical Averages – Corollary

If x and x are independent RVs and

z = x + y, (10)

thenz = x + y (11)

σ2z = σ2

x + σ2y . (12)

z = x + y, (10)

thenz = x + y (11)

σ2z = σ2

x + σ2y . (12)

z = x + y, (10)

thenz = x + y (11)

σ2z = σ2

x + σ2y . (12)

Central-Limit Theorem

z = x + y, (13)

pz (z) =ˆ ∞

−∞px (x) py (z− x) dx. (14)

From the above equation, it is clear that the PDF pz (z) is the convolution ofPDFs px (x) and py (y) . This result can be extended to n number of RVs.

Under certain conditions, sum of large number of independent random vari-ables tends to be a Gaussian random variable, independent of the PDFs of therandom variables involved. This is the central-limit theorem.

z = x + y, (13)

pz (z) =ˆ ∞

−∞px (x) py (z− x) dx. (14)

z = x + y, (13)

pz (z) =ˆ ∞

−∞px (x) py (z− x) dx. (14)

z = x + y, (13)

pz (z) =ˆ ∞

−∞px (x) py (z− x) dx. (14)

Central-Limit Theorem – Demonstration

px(x) px(x) * px(x)

x x1-1 2-2

px(x) px(x) * px(x) px(x) * px(x) * px(x)

x xx1-1 2-2 -3 3

The Gaussian (Normal) PDF

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ= 2 1.0,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ= 2 1.0,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ=0,μ=

2 0.2,σ =2 1.0,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ=0,μ=0,μ=

2 0.2,σ =2 1.0,σ =2 5.0,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ=0,μ=0,μ=−2,μ=

2 0.2,σ =2 1.0,σ =2 5.0,σ =2 0.5,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5−1 0 2 4−2−4

0,μ= 2 1.0,σ =

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ=0,μ=0,μ=−2,μ=

2 0.2,σ =2 1.0,σ =2 5.0,σ =2 0.5,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

−5 −3 1 3 5−1 0 2 4−2−4

0,μ=0,μ=0,μ=−2,μ=

2 0.2,σ =2 1.0,σ =2 5.0,σ =2 0.5,σ =

−5 −3 1 3 5

−1 0 2 4−2−4

0,μ=0,μ=0,μ=−2,μ=

2 0.2,σ =2 1.0,σ =2 5.0,σ =2 0.5,σ =

px (x) =1

2πe−(x−µ)2/2σ2

Fx (x) =1

−∞e−(x−µ)2/2σ2

dx = 1−Q(

x− µ

The Gaussian (Normal) PDF – Interpretation

−2σ −1σ 1σ−3σ 3σ0 2σ

34.1% 34.1%

13.6%2.1%

13.6% 0.1%0.1% 2.1%

The Gaussian (Normal) PDF – Interpretation

−2σ −1σ 1σ−3σ 3σ0 2σ

34.1% 34.1%

13.6%2.1%

13.6% 0.1%0.1% 2.1%

Outline

1 Introduction

2 Random Variables

3 Random Processes

Random Process

x(t, ζ1)

Random Process

x(t, ζ1)

Random Process

x(t, ζ1)

x(t, ζ2)

Random Process

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

Random Process

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

x(t, ζ4)

Random Process

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

x(t, ζ4)

Sample function

Random Process

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

x(t, ζ4)

Ensemble

Random Process

x(t1) = x1

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

x(t, ζ4)

Randomvariable

Random Process

x(t1) = x1 x(t2) = x2

x(t, ζ1)

x(t, ζ2)

x(t, ζ3)

x(t, ζ4)

Randomvariable

Randomvariable x isa function

of time

Random Process

A random variable that is a function of time is called a random process orstochastic process.

In other words, a random process is just a collection of an infinite number ofRVs, which are generally dependent.

So, a random process is completely described by the joint PDF

px1x2···xn (x1, x2, . . . , xn)

which can also be expressed as