© 2009 Winton 1

Stability in Queuing Systems

© 2009 Winton 2

Long Run Measures of Performance• L

– the long run average over time of the number of customers in thesystem

• LQ– the long run average over time of customers in queue Q

• w– the long run average time spent in the system

• wQ– the long run average time spent in queue Q

• ρ– the server utilization, the % of time that a server is busy.

• A system is said to be stable if its long run averages exist and are < ∞– If a system is unstable, its long run measures are meaningless– A model of an unstable system must use protocols to accommodate

the effects of the instability

© 2009 Winton 3

Measure L(long run average over time of the number of customers in the system)

• If time units are approximated by discrete intervals ΔT, then let– T0 = set of intervals ΔT that 0 customers are in the system– T1 = set of intervals ΔT that exactly 1 customer is in the system– T2 = set of intervals ΔT that exactly 2 customers are in the system

. . .– Tn = set of intervals ΔT that exactly n customers are in the system

• For the time period T covered, the Ti encompass all ΔT intervals and

so and the sum approximates the total area under the

curve of C(t), where C(t) is the number of customers in the system at time t

• For this time period, the expected value

approximates the mean number of customers in the system over the given time interval

• This leads to the form

∑=

=n

0iiT T ∑

=

⋅n

0iiTi

∫∞

=0

C(t)dt T1 L

∑=

∧

⋅=n

0i

i

TTi L % of time that i customers

are in the system

© 2009 Winton 4

Measure w(the long run average time spent in the system)

• To obtain w (since customers are discrete), approximations of w are given by

where wi = time in system for customer i and N is the customer count

– As < ∞ (unless the system is unstable) • In general, the conservation equation

L = λw where λ = the mean arrival rateholds in a stable system– This equation also provides the relationship between L and w.

• Example– If we have an average time (w) in system per customer of 10 hours

and customers arrive at the rate (λ) of 2 per hour on average, then at any time we expect to have 20 customers (L) in the system

w N1 w

N

1ii∑

=

∧=

ww , N →∞→∧

© 2009 Winton 5

Server Utilization• Recap:

– L is the long run average over time of the number of customers in the system

– w is the long run average time spent in the system• Viewing the server as a system in its own right, server busy means 1

customer in the system, server idle means 0 customers in the system– If the mean service rate (customers served per unit of time) is given by

μ and customers arrive at the server with mean arrival rate λ, then the L statistic for this system is the server utilization ρ (the % of time that a server is busy)

– To see this, note that w =1/μ• the average time a customer spends in the server

– By the conservation rule L = λw = λ/μ = arrival rate × average time in server = (customers per unit of time) × (time-units of service per customer) = (time units of service)/time-unit = % of time in use = ρ.

© 2009 Winton 6

Server Utilization ρ

• It is evident that for stability, we must have ρ < 1; i.e., λ < μ.

• This agrees with our intuition that the arrival rate (λ) must be less than the service rate (μ) and the mean interarrival time (1/λ) must be greater than the mean service time (1/μ).

© 2009 Winton 7

Influence of Arrival Patterns

• For the overall system, L, LQ, w, wQ are dependent on the arrival patterns, not just λand μ– analysis requires knowledge of the data

distribution.

© 2009 Winton 8

Arrival Pattern Example• Example

– As an extreme case, suppose 1000 customers arrive at time 0, another 1000 at time 1000, and so forth

– Although the customer arrivals are quite dispersed, the average arrival rate is λ = 1

– If the average service rate μ = 1, then for M = the number of blocks of 1000 people (so customer count N=1000M)

– Essentially the same computation results in L = 500.5.• In contrast, if the customer arrivals distribute uniformly with λ = μ = 1,

then w = L = 1. Note that the variance in this case is 0, whereas above it is

w 500.5 2

1)(100010001000

1 iM1000M

1 w1000

1i==

+⋅⋅=⋅⋅= ∑

=

∧

)2

1)n(n i that (recalln

1

+=∑

998.001 )11000(1000

1 1)-(iat(i)1000

1 21000

1i

2 =−⋅=∑=

© 2009 Winton 9

Testing Stability• A necessary and sufficient condition for model stability is

mean interarrival time > mean service time(customers arrive on average slowly enough for the server to stay ahead)

• It needs to be emphasized that this condition may be insufficient in practice – The example where 1000 customers arrive at time 0, another 1000 at

time 1000, and so forth had an uncomfortably large w value of 500.5 because of the arrival pattern

• In a stable subsystem of a model where items which arrive in thesubsystem must eventually depart, then the miat at the entrance point to the subsystem is equal to the miat at the departure point from the subsystem– The flow principle: items can’t depart faster than they arrive, and if

the subsystem is stable, they can’t arrive faster than they depart

© 2009 Winton 10

Split Rule• Suppose customers arrive at a point x in a system with

mean interarrival time (miat) μ and proceed to point A with probability p1 and to point B with probability p2– the mean iat at point A is μ/p1 – the mean iat at point B is μ/p2

μ/p2

μ (miat)

x

μ/p1

A B

p1 p2

35.71

25

x

83.33

A B

.3 .7

Example

(miat at point B)

© 2009 Winton 11

Join Rule• Suppose customers arrive at point x in a system from either point A or

from point B with mean iat at point A being μA and at point B being μB

–

• add the arrival rates and invert to get the miat

1

BA μ1

μ1 μ

−

⎟⎟⎠

⎞⎜⎜⎝

⎛+=x

x

μA

A B

μB

μx = ((1/μA) + (1/μB))-1

© 2009 Winton 12

Split then Join Example

• For the example from the split rule, if we rejoin the A and B legs, then we recover the miat as (1/83.33 + 1/35.71)-1 = 25 as expected

25

x

83.33

A B

.3 .7

x μx = ((1/83.33) + (1/35.71))-1 = 25

35.71

35.71 83.33

© 2009 Winton 13

Example Exhibiting Instability• Customers arrive at an average rate of 1 every 25 seconds

– 30% proceed to the help desk • 80% of the time the help desk receives a directional question

taking on average 20 seconds to answer• The remaining questions take on average 2 minutes to answer• 20% of help desk customers proceed to clearance and then exit

– The remainder go to the intervention station» Help desk customers proceeding to the intervention

station are normally processed in 1.5 minutes– 70% proceed to the checkout station

• There are 2 servers at the checkout station– Server 1 averages 60 seconds per customer – Server 2 averages 90 seconds per customer

• From the checkout station, customers proceed to clearance– On average, clearance takes 33 seconds

© 2009 Winton 14

Analysis• The model is diagrammed

to show the splits and joins as given by the problem statement– For analysis purposes,

the miat on the upstream side of a server is kept the same on the downstream side

– Instability is identified wherever the miat at a server is calculated to be less than its mean service time

• Marked with on the diagram

25

35.71 83.33

.3 .7

35.71

89.2859.52

90/150 = .6 .4 = 60/150

60 90

89.2859.52

checkout station

help desk.8×20 + .2×120 = 40

83.33

.2.8

104.16 416.65

intervention90

104.16

32.89

clearance33

32.89

25

35.71

Remark: if the model is implemented as is, the checkout service times will increase the miat at the service exit from 35.71 to 36 (1/(60-1 + 90-1)), with a resulting miat at clearance of 33.14, an effect that masks the instability at clearance revealed by the analysis. It also changes the exit miat to 25.14.

© 2009 Winton 15

ExtendSimImplementation

• Queues at “check out” are growing beyond possibility of reduction back to 0 over time, confirming the instability predicted by analysis

• Queues at “help desk”, “intervention”, and “clearance” are all indicative of stability (although clearance is a point of instability)

– Demonstrates masking of instability at “clearance” due to earlier instability at “check out”

© 2009 Winton 16

• A feedback loop is a section of a model where items can loop back in the item flow– Customers in a cafeteria return for dessert– A part is defective and returned for more work– A job is worked on for awhile then returned to queue

• The section with the loop can be isolated by a single point of arrival and a single point of departure– The effect of the feedback is localized to the part of the model

where the feedback occurs

• The items feeding back increment the interarrival times that occur within the part of the model affected by the loop, but have no impact on miat values outside of the loop

Feedback Loops

© 2009 Winton 17Restricted Feedback (1)

• If item feedback is limited to exactly once, miat values are determined from the beginning of the loop

• To see this, consider the example, where 10% of items going down a path in the model feed back to re-enter the path, but no more than once– If the miat at the top of the

loop is 60, then the miat for the 10% of the items that feed back in is 60/0.1 = 600

– Beyond the loop, items continue with a miat of 60

t=60

service point30

54.54

60060

600

1/(1+0.1) 0.1/(1+0.1)

Restricted case: 10% of items entering the section join back at most once; the loop back miat is 600

60/1.1=54.54

(1/60 +1/600)-1 = 60/1.1

© 2009 Winton 18Restricted Feedback (2)

• Intuitively, the items in the feedback loop are undergoing an extra delay, so the miat on arrival to this part of the model matches the miat on departure– Items feeding back

decrement the miat for the service point (items arrive faster)

– Note that the percentage at the split for feedback is slightly lower than 10%

• Reflects fact that some items in the flow at this point have already fed back

t=60

service point30

54.54

60060

600

1/(1+0.1) 0.1/(1+0.1)

Restricted case: 10% of items entering the section join back at most once; the loop back miat is 600

60/1.1=54.54

(1/60 +1/600)-1 = 60/1.1

© 2009 Winton 19Unrestricted Feedback (1)

• If items can feed back more than once, miat values are determined from the bottom of the loop

• To see this, consider the example, where 10% of items arriving at the feed back point may return even if they have already done so in a prior iteration– For items to continue beyond

the feedback point with miat t=60, then for the 90% exiting, the miat coming in to the feedback point must be 54

• This forces the miat for the 10% of the items that feed back to be 54/0.1=540

t=60

service point30

54

54060

540

0.9 0.1

Unrestricted case: 10% of items reaching the feedback point loop back whether or not they already have done so; the loop back miat is 540

54

(1/60 +1/540)-1 = 54

© 2009 Winton 20

Unrestricted Feedback (2)

• For another viewpoint, at the top of the loop, p items recycle exactly once, p2

exactly twice, p3 exactly 3 times, …, so the miat for the number recycling is given by t*∑pi = pt/(1-p)– For the example, the

recycle miat calculates as (60*.09)/0.1 = 540 which agrees with the prior calculation

t=60

service point30

54

54060

540

0.9 0.1

Unrestricted case: 10% of items reaching the feedback point loop back whether or not they already have done so; the loop back miat is 540

54

(1/60 +1/540)-1 = 54

© 2009 Winton 21

Feedback Computations• The basic computations are the miat at the re-entry point and the probabilities

associated with the split at the bottom of the feedback loop• Algebraically, if t is the miat of the main flow where it joins with feedback

items at the top of the loop and the feedback probability = p, then the miat for restricted feedback at the service point is given by

which for the prior example is 60/1.1

• For unrestricted feedback, it is given by

which for the prior example is 60*0.9

p1t

tp1

pt1

t1 1

1

+=⎟

⎠⎞

⎜⎝⎛ +

=⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

⎟⎠⎞⎜

⎝⎛

+−

−

ptpt

p1p

p-1pt

1t1

1

1

=⎟⎟⎠

⎞⎜⎜⎝

⎛ −+=

⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜

⎝

⎛

⎟⎠⎞⎜

⎝⎛

+−

−

© 2009 Winton 22

Other Computations• By the flow principle, the miat into the main

flow at split point where feedback occurs must return to t (= 60 in the prior example). – This forces the probability of rejoining the main

flow in the restricted case to be 1/(1+p), which in the example is 1/(1+0.1)

• The probability of taking the feedback side at the split point is then just 1-[1/(1+p)] = p/(1+p), which is 0.1/(1+0.1) in the example

– For the unrestricted case the probabilities are just p and 1-p

© 2009 Winton 23

Mid Stream Feedback

• If feedback enters an interior point of the model section (defined by a single point of arrival and single point of departure), the analysis is unaffected– More care must be taken to

identify the section containing the feedback

• Note the commonality of the computations in comparison to the earlier example– The unrestricted case is

equally comparable

service point

service point

t=60

service point

0.6 0.4

100 150

100 150

Restricted case: 10% of items entering the section join back at most once; the loop back miat is 600

600

120

120

54.54

600601/(1+0.1) 0.1/(1+0.1)