Operating Systems - Concurrency
44
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science Operating Systems CMPSCI 377 Concurrency Patterns Emery Berger University of Massachusetts Amherst
-
Upload
emery-berger -
Category
Technology
-
view
3.278 -
download
3
description
Transcript of Operating Systems - Concurrency
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Operating SystemsCMPSCI 377
Concurrency PatternsEmery Berger
University of Massachusetts Amherst
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Finishing Up From Last Time
Avoiding deadlock: is this ok?
2
lock (a);
lock (b);
unlock (b);
unlock (a);
lock (b);
lock (a);
unlock (a);
unlock (b);
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Finishing Up From Last Time
Not ok – may deadlock.
Solution: impose canonical order (acyclic)
3
lock (a);
lock (b);
unlock (b);
unlock (a);
lock (b);
lock (a);
unlock (a);
unlock (b);
lock (a);
lock (b);
unlock (b);
unlock (a);
lock (a);
lock (b);
unlock (b);
unlock (a);
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Motivating Example: Web Server
Client (browser)
Requests HTML, images
Server
Caches requests
Sends to client http://server/Easter-bunny/200x100/75.jpg
not found
client
webserver
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Possible Implementation
while (true) {
wait for connection;
read from socket & parse URL;
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
5
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Possible Implementation
while (true) {
wait for connection; // net
read from socket & parse URL; // cpu
look up URL contents in cache; // cpu
if (!in cache) {
fetch from disk / execute CGI;//disk
put in cache; // cpu
}
send data to client; // net
}
6
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Problem: Concurrency
Sequential fine until:
More clients
Bigger server Multicores, multiprocessors
Goals:
Hide latency of net & disk I/O Don’t keep clients waiting
Improve throughput Serve up more pages
clients
webserver
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Building Concurrent Apps
Patterns / Architectures
Thread pools
Producer-consumer
“Bag of tasks”
Worker threads (work stealing)
Goals:
Minimize latency
Maximize parallelism
Keep progs. simple to program & maintain
8
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 9
Thread Pools
Thread invocation & destruction relatively expensive
Instead: use pool of threads
When new task arrives, get thread from pool to work on it; block if pool empty
Faster with many tasks
Limits max threads
( ThreadPoolExecutor class in Java)
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer
Can get pipeline parallelism:
One thread (producer) does work
E.g., I/O
and hands it off to other thread (consumer)
10
producer consumer
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer
Can get pipeline parallelism:
One thread (producer) does work
E.g., I/O
and hands it off to other thread (consumer)
11
producer consumer
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer
Can get pipeline parallelism:
One thread (producer) does work
E.g., I/O
and hands it off to other thread (consumer)
12
producer consumer
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer
Can get pipeline parallelism:
One thread (producer) does work
E.g., I/O
and hands it off to other thread (consumer)
13
producer consumer
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer
Can get pipeline parallelism:
One thread (producer) does work
E.g., I/O
and hands it off to other thread (consumer)
14
producer consumer
LinkedBlockingQueueBlocks on put() if full, poll() if empty
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer Web Server
Use 2 threads: producer & consumer
queue.put(x) and x = queue.poll();
15
while (true) {
do something…
queue.put (x);
}
while (true) {
x = queue.poll();
do something…
}
while (true) {
wait for connection;
read from socket & parse URL;
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer Web Server
Pair of threads – one reads, one writes
16
while (true) {
wait for connection;
read from socket & parse URL;
queue.put (URL);
}
while (true) {
URL = queue.poll();
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer Web Server
More parallelism –optimizes common case (cache hit)
17
while (true) {
wait for connection;
read from socket & parse URL;
queue1.put (URL);
}
while (true) {
URL = queue1.poll();
look up URL contents in cache;
if (!in cache) {
queue2.put (URL); return;
}
send data to client;
}
while (true) {
URL = queue2.poll();
fetch from disk / execute CGI;
put in cache;
send data to client;
}
1
2
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
When to Use Producer-Consumer
Works well for pairs of threads
Best if producer & consumer are symmetric
Proceed roughly at same rate
Order of operations matters
Not as good for
Many threads
Order doesn’t matter
Different rates of progress
18
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Producer-Consumer Web Server
Have to be careful to balance load across threads
19
while (true) {
wait for connection;
read from socket & parse URL;
queue1.put (URL);
}
while (true) {
URL = queue1.poll();
look up URL contents in cache;
if (!in cache) {
queue2.put (URL);
}
send data to client;
}
while (true) {
URL = queue2.poll();
fetch from disk / execute CGI;
put in cache;
send data to client;
}
1
2
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
20
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
21
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
22
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
23
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
24
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
25
worker worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks
Collection of mostly independent tasks
Bag could also be LinkedBlockingQueue(put, poll)
26
worker worker worker worker
addWork
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks Web Server
Re-structure this into bag of tasks:
addWork & worker threads
t = bag.poll() or bag.put(t)
27
while (true) {
wait for connection;
read from socket & parse URL;
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks Web Server
Re-structure this into bag of tasks:
addWork & worker
t = bag.poll() or bag.put(t)
28
addWork:
while (true) {
wait for connection;
bag.put (URL);
}
Worker:
while (true) {
URL = bag.poll();
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks Web Server
Re-structure this into bag of tasks:
t = bag.poll() or bag.put(t)
29
worker worker
addWork: while (true){
wait for connection;
bag.put (URL);
}
worker: while (true) {
URL = bag.poll();
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
worker
addWork
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Bag of Tasks vs. Prod/Consumer
Exploits more parallelism
Even with coarse-grained threads
Don’t have to break up tasks too finely
Easy to change or add new functionality
But: one major performance problem…
30
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
What’s the Problem?
31
worker worker worker worker
addWork
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
What’s the Problem?
Contention – single lock on structure
Bottleneck to scalability
32
worker worker worker worker
addWork
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Queues
33
executor
Each thread has own work queue (deque)
No single point of contention
Threads now generic “executors”
Tasks (balls): blue = parse, yellow = connect…
executor executor executor
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
executor executor executor executor
Work Queues
34
Each thread has own work queue (deque)
No single point of contention
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Queues
35
Each thread has own work queue (deque)
No single point of contention
executor executor executor executor
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
executor executor executor executor
Work Queues
36
Each thread has own work queue (deque)
No single point of contention
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Queues
37
Each thread has own work queue
No single point of contention
Now what?
executor executor executor executor
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing
38
worker
When thread runs out of work,steal work from random other thread
worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing
39
worker
When thread runs out of work,steal work from top of random deque
Optimal load balancing algorithm
worker worker worker
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing Web Server
Re-structure:readURL, lookUp, addToCache, output
myQueue.put(new readURL (url))
40
while (true) {
wait for connection;
read from socket & parse URL;
look up URL contents in cache;
if (!in cache) {
fetch from disk / execute CGI;
put in cache;
}
send data to client;
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing Web Server
Re-structure:readURL, lookUp, addToCache, output
myQueue.put(new readURL (url))
41
readURL(url) {
wait for connection;
read from socket & parse URL;
myQueue.put (new lookUp (URL));
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing Web Server
Re-structure:readURL, lookUp, addToCache, output
myQueue.put(new readURL (url))
42
readURL(url) {
wait for connection;
read from socket & parse URL;
myQueue.put (new lookUp (URL));
}
lookUp(url) {
look up URL contents in cache;
if (!in cache) {
myQueue.put (new addToCache (URL));
} else {
myQueue.put (new output(contents));
}
}addToCache(URL) {
fetch from disk / execute CGI;
put in cache;
myQueue.put (new output(contents));
}
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
Work Stealing
Works great for heterogeneous tasks
Convert addWork and worker into units of work (different colors)
Flexible: can easily re-define tasks
Coarse, fine-grained, anything in-between
Automatic load balancing
Separates thread logic from functionality
Popular model for structuring servers
43
UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
The End
44
