Queuing Theory and Stochastic Service Systems
Transcript of Queuing Theory and Stochastic Service Systems
Queuing Theory and Stochastic Service Systems
Li Xia Tsinghua University, 2014 Fall
Syllabus • Instructor
– Li Xia 夏俐, FIT 3-618, 62793029, [email protected] • Text book
– D. Gross, J.F. Shortle, J.M. Thompson, and C.M. Harris, Fundamentals of Queueing Theory, 4th Edition, Hoboken: Wiley, 2008. (copy is provided)
• Reference books: – Leonard Kleinrock, Queueing Systems, vol. 1: Theory, John Wiley, 1975. – Mor Harchol-Balter, Performance Modeling and Design of Computer
Systems—Queueing Theory in Action, Cambridge Press, 2013. – Caltech course (Prof. Adam Wierman):
http://courses.cms.caltech.edu/cs147/ – 林闯,计算机网络和计算机系统的性能评价,清华大学出版社,
2001.
2 Li Xia, Tsinghua Univ., 2014
Syllabus • Grading
– Homework: 25% (4 assignments, 1 simulation task, in English) plagiary is prohibited
– Midterm: 25% – Final Project: 40% (the 9th week) – Course Interaction: 10%
• Lecture notes and assignments are available online (in English) – http://cfins.au.tsinghua.edu.cn/personalhg/xiali
/teaching/course_queues.htm
3 Li Xia, Tsinghua Univ., 2014
What’s your purpose to take this course
• What do you expect to learn from this course? – Open discussion
• Let’s see some examples in practice
4 Li Xia, Tsinghua Univ., 2014
Beijing Subway
Throughput?
Safety?
More lines Increase buffer So what?
5 Li Xia, Tsinghua Univ., 2014
Railway ticket online booking in 2012 Chinese new year
• Crash of ticket booking system – Large number of tickets for sale (4million) – Huge visit requests 秒杀? (billion) – System architecture is not optimal
• Bandwidth of network • CPU/RAM of computer • Business logic • Or other factors…
6 Li Xia, Tsinghua Univ., 2014
Modeling and Analysis • How to solve it?
– Performance analysis and optimization – Queueing scheme, increase bandwidth…
Internet
Web server Application server
client Database server
IE browser data input interaction display…
passwd verify cookies/other application…
ticket data booking records…
7 Li Xia, Tsinghua Univ., 2014
Applications in daily life
• Supermarket – How to define the express line (# of items)? – How to determine the number of checkouts? – How long customers have to wait at checkouts? – Behavior of waiting time during peak-hours
• Line at bank counters – Multiple lines v.s. one line – Specialist purpose v.s. generalist purpose – Number of counters?
8 Li Xia, Tsinghua Univ., 2014
Applications in engineering
• Computer/circuit architecture design – 1 fast disk v.s. 2 slow disks? – Invest on large buffer v.s. fast CPU? – Scheduling policy to improve performance
• Communication network design – Buffer size design of switch/router – Data packet scheduling policy in sensor or mobile
network • …
9 Li Xia, Tsinghua Univ., 2014
List of applications areas
• Production system (machine, different products) • Computer system (cpu, disk, RAM design) • Communication network (buffer design, link capacity) • Transportation system (traffic lights control) • Bank branches operation (counter/type design) • Airlines scheduling (takeoff/landing arrangement) • Data center (optimal control, energy saving) • Call center (optimize the operators, hotlines,…) • Post office (multi-class, specialization) • …
10 Li Xia, Tsinghua Univ., 2014
What’s queue? • A general queuing system
Customer arrival
waiting room
Service facility
Customer departure
r22
r11
r12
r23
r31 r21 r32
r13 r33 1µ
2µ
3µ
r20
r10
r30 γ1
γ2
γ3 Queuing network:
A single server queue:
11 Li Xia, Tsinghua Univ., 2014
Terminology in queuing theory
• Basic element in queue – Arrival pattern, service pattern, number of servers, service
discipline, system capacity, customer type,..
• Performance metrics – Average number of customers – Queue length, average number of queuing customers – Throughput – Response time, sojourn time, system time – Waiting time, queuing time
12 Li Xia, Tsinghua Univ., 2014
Why we need queuing theory?
• Resource constraints – Why queues appear? How to make them go away?
• Goal of queuing theory – Predict the performance – Design the architecture – Optimize the parameter/policy
• Counter-intuitive – Randomness is complicated – Some examples
13 Li Xia, Tsinghua Univ., 2014
Example 1: CPU design
• A simple model of CPU – Job arrival at rate λ=3/s, Poisson process – Job mean size is 1/μ, exponential
• i.e., service rate is μ=5/s
– FCFS(first come first serve), buffer is infinite – assume λ < μ, [question]why?
CPU
buffer
Model of a cpu
λ μ
NOTE: modern CPU may have other features, multi-core/PS, etc. 14 Li Xia, Tsinghua Univ., 2014
CPU design, cont.
• If the arrival rate λ doubles, how to upgrade? – If want to maintain the same delay of jobs,
[question] what you choose? • A. double μ • B. less than double μ • C. more than double μ
– Why? Double μ will cut the delay in half • prove with M/M/1 queuing theory • Physical intuition, time speeds up with scale 2
15 Li Xia, Tsinghua Univ., 2014
Example 2: Lines in bank • Customer arrival in Poisson with rate λ • Counter service rate is μ, exponential • FCFS, infinite waiting capacity
3λµ
µ
µ
µ
µ
µ
λ
λ
λ
3 lines 1 line 16 Li Xia, Tsinghua Univ., 2014
L1=1.5, L2=0.237
W1=0.5min, W2=0.079min
T1=1min, T2=0.579min
Lines in bank, cont.
• Assume μ = 2/min, λ = 1/min – queue length, – waiting time, – response time,
• [question] how is the following queue?
3λ 3µL3 = 0.5 W3 = 0.1667 T3 = 0.3333
17 Li Xia, Tsinghua Univ., 2014
Example 3: many slow v.s. one fast
• CPU selection: – 1 core CPU with 3GHz freq. – 3 core CPU with 1GHz freq.
• Which one has a better mean response time?
3λ 3µ3λ
µ
µ
µ18 Li Xia, Tsinghua Univ., 2014
• Depend on the variability of jobs – Job size variability is high, choose many slow CPU – Job size variability is low, choose one fast CPU
• Exponential distr.: coefficient of variation(cv) = 1; – For the case of M/M/c and M/M/1, the latter is better
• Uniform distr. or deterministic: cv < 1; • Hyper-exponential distr. or other distr. (PH, MAP):
cv > 1. (self-similarity of Internet traffic)
– If workload is low, one fast is preferred – If jobs are preemptible (priority, stop, resume)
• One fast is preferred
many slow v.s. one fast, cont.
19 Li Xia, Tsinghua Univ., 2014
many slow v.s. one fast, cont. • many slow v.s. a few fast, widely exist in
engineering problems – Power allocation in data center with server farm
• Fast freq., more power consumption, green data center
– Bandwidth partition in communication systems • Small chunks of bandwidth, TDMA/FDMA/CDMA …
– Road network in transportation • Big trunk road v.s. many small roads
– etc. Consider economic factors… – Service rate control in Jackson network
• (Xia and Shihada, IEEE-TAC 2013) 20 Li Xia, Tsinghua Univ., 2014
Example 4: Closed queueing network
• Model the intensive traffic with N capacity of network – Batch system, intensive queue with limited
capacity, etc.
113
µ =0.5
0.5
N=6 jobs
113
µ =
21 Li Xia, Tsinghua Univ., 2014
Closed queueing network, cont.
• If we double the speed of server 1 – How it effects the response time of job? – How it effects the throughput?
• [Answer] only change by a small amount
• Suppose N is very large, how is above question? – Change 0, if N ∞
• What if N is very small – If N=1, changed amount is large
22 Li Xia, Tsinghua Univ., 2014
Closed queueing network, cont.
• What if the queueing network is open? – remarkable improvement of throughput and
average response time
0.5
0.5
λ 113
µ =
113
µ =
23 Li Xia, Tsinghua Univ., 2014
Example 5: Task assignment in a server farm
• Front-end dispatcher, web server farm, assign task among back-end servers – used in engineering, Cisco/IBM network device
λ 1µ
2µ
Arrivals Dispatcher (Load Balancer)
24 Li Xia, Tsinghua Univ., 2014
• Task assignment policy – Determine the task should go to which server – Based on the system state, policy in MDP
• Different policy – Random – Shortest-Queue (SQ) – Size-Interval-Task Assignment (SITA) – Least-Work-Left (LWL) – Central-Queue (CQ)
• Question: which one has best mean response time?
Task assignment, cont.
25 Li Xia, Tsinghua Univ., 2014
• Answer – Depend on the property of job size
• If job size is known, LWL is usually the best • LWL = CQ ?
– If server discipline is processor-sharing (PS) • SQ is the near optimal
• Task assignment problem – FCFS/PS, modeled as an MDP optimization problem – Minimize variance of response time, rather than the mean
response time – Variance (fairness, risk) v.s. Mean (social welfare)
Task assignment, cont.
Chinese Proverb: 不患寡而患不均 26 Li Xia, Tsinghua Univ., 2014
Example 6: Scheduling
• How service disciplines affect response time? – FCFS, first come first serve – LCFS, last come first serve – Random – [answer] all the same
λ µ
27 Li Xia, Tsinghua Univ., 2014
Scheduling, cont.
• What if PR-LCFS, preemptive-resumed LCFS? – Depends on the randomness of job size
• High randomness, big improvement • No randomness, twice worse
λ µ
28 Li Xia, Tsinghua Univ., 2014
Summary of examples
• Why counter-intuition? – Randomness of queuing – Interactions among customers and servers
• Toy example, but many insights – Models – Analysis – Design – Optimization – …
29 Li Xia, Tsinghua Univ., 2014