1

A Whirlwind Tour through Concurrency

Kedar Namjoshi

Bell Labs

Columbia University, Jan. 2008

2

Outline

Concurrency

Common Concurrency Patterns

The Plan9 Thread Model

Reasoning about Concurrency

The “Session Data Type” Model

3

Concurrency

Processes which overlap in time and exchange information

The world is concurrent. Purely sequential behavior is often artificial (“like a machine”)

However, writing concurrent programs is viewed as a hard problem. Why?

4

Concurrency in programming For a long time, concurrency has been a concern

mainly for designers and implementers of – operating systems

– supercomputers

– processor chips

– database systems

– and phone networks …

This is about to change!Faster Chips Are Leaving Programmers In Their Dust

(The New York Times, December 17, 2007)

5

Multi-Core Processors

Dual-core already common; Quad-core on its way; expect more!

(from http://www.amd.com)

6

Shared-Memory Concurrency

initially, account x holds $0

P1: x := x+50; // deposit $50 to x

P2: x := x+100; // deposit $100 to x

What is the amount in x after P1||P2?

It should be $150 --- but it might be $50 !

How could this happen?

7

Shared-Memory II

Let’s take a closer look

P1: t1 := x; t1 := t1+50; x := t1;

P2: t2 := x; t2 := t2+100; x := t2;

For the schedule P2;P1;P2;P1;P2;P1, the final value of x is $50!

Mistake: implicit assumption that the deposit action is atomic.

8

Shared-Memory IIIAtomicity must be enforced.

P1’: lock x; deposit $50; unlock xP2’: lock x; deposit $100; unlock x

Locks are difficult to get right. programmers forget to place them (“data races”) or forget to release locks, or do so in the wrong order (“deadlock”) or put too many locks (minimal concurrency) Locking could result in priority inversion (a higher priority process gets

locked out) --- the Mars Pathfinder failure was traced back to priority inversion

Locks often don’t match up well to the structure of a problem.

9

Shared Memory IVCompilers can get in the way, too! P1: P2: x := 0; y := 0; y := 1; x := 1;

On termination, we may expect anything but x=0,y=0. However, the compiler (or the processor) is free to reorder instructions!

P1’: P2’: y := 1; x := 1; x := 0; y := 0;

Now x=0,y=0 is a possibility! This leads to considerations of a weak consistency model.

10

CSP [Hoare 1978]CSP: Communicating Sequential ProcessesThree key concepts No shared variables Processes communicate through synchronized send-receive

actions A process may respond to one of a set of communication

actions

Simple. Clean. Elegant.

Hoare received the Turing Award in 1980 for this and many other contributions to computing

11

Communication Patterns

Often, the most natural description of a concurrent program is through its communication pattern.

We’ll see a number of very common patterns.

12

1. The knock on the door … … also known as a “real-time interrupt” C + Unix : signals Java: Thread.interrupt()

“As captain of the crew I had had extensive experience (dating back to 1958) in making basic software dealing with real time interrupts and I knew by bitter experience that as a result of the irreproducibility of the interrupt moments, a program error could present itself misleadingly like an occasional machine malfunctioning. As a result I was terribly afraid. Having fears regarding the possibility of debugging we decided to be as careful as possible and —prevention is better than cure!— to try to prevent bugs from entering the construction.”

-- Edsger W. Dijkstra, The Structure of the “THE”–multiprogramming System (1968) EWD 196

Dijkstra received the Turing Award in 1972 for his work on concurrency and structured programming

13

How does one prevent bugs?

The non-determinism introduced by the real-time interrupt implies that a program execution may be stopped at any point, in any intermediate state.

The key is to write down the minimal set of facts that should be invariant (i.e., unchanged) across an interrupt, for the main program to work correctly.

Then write the interrupt handling routine to respect the invariant, or disable interrupts over a short execution sequence.

14

2. Copy

process copy(channel in, channel out)

var x;

*[in?x out!x]

? is the input action

! is the output action

*[] means repeat forever

15

3. Pipeline

A chain of N (N >= 1) copy processes:

copy(c0,c1) || copy(c1,c2) || … || copy(c(N-1),c(N))

If the i’th process is altered to

[in?x out!f(i)(x)]*

the pipeline computes the function

f(N) * f(N-1) * … * f(2) * f(1)

16

4. Delegation: Taxi Rank

All taxis follow the same protocol.

process Taxi[i]:

taxi-rank! i; *[dispatch[i]?p drive(p); taxi-rank! i]

process Dispatcher:

*[queue?p (taxi-rank?i dispatch[i]!p)]

Also known as a “thread pool”

17

5. Delegation: Futures

process A:

toB ! (“x+2”,x:10);

…. stuff ….

fromB ? y;

Process B:

*[toB? (expr, binding) fromB! eval(expr,binding)]

Introduced in Scheme (1975?) as delay and force

18

6. Co-authoring a paper

process Kedar:

toDennis! token;

*[fromDennis? token write; toDennis! token]

process Dennis:

*[toDennis? token write; fromDennis! token]

Also known as a “co-routine” .

19

7. Callback

K:

C! “let me know if you can’t pick up the kids today”;

[work interrupt (C? “can’t do it” pick up kids)]

Best understood as an expected interrupt.

All warnings regarding interrupt processing apply.

20

8. Multiplexing Merge messages from two channels into a single channel

process mux:

*[

| channel1? x out! x

| channel2? y out! y

]

The operator “|” defines alternatives. If both alternatives are available, the choice is made non-deterministically.

The result may not be a fair merge!

21

9. De-Multiplexing

Copy input from one channel into two

process demux:

*[ input? x out1! x; out2! x]

Or, perhaps:

process demux:

*[input? X if (x>0) out1!x; else out2!x]

22

10. On Friday … K:

C! “let me know if you can’t pick up the kids today”;

[work interrupt

(

| C? “can’t do it” pick up kids

| Al? “would you like to teach a class?” chat

| Dennis? “lunch?” go to lunch

| …

)

]

All warnings regarding interrupt processing apply.

23

The Plan9 Thread Model Plan9 is an operating system developed at Bell Labs from about 1990 onwards. Lots of interesting features. In particular, the OS thread model influenced by CSP.

CSP also influenced two languages at Bell Labs newsqueak (Pike, Cardelli) Rob Pike’s talk Alef (Winterbottom) Russ Cox’s CSP-Plan9 page

and, in the wider world … The design of the Ada programming language The Occam programming language The Transputer chip The BPEL web services language

The CSP book is available.

24

What is the Plan9 thread model?Two kinds of entities: processes and threads.

Threads are co-routines within a process (i.e., scheduling is cooperative)

Processes are pre-emptively scheduled

Channels are used for communication and synchronization. They can be synchronous (as in CSP) or buffered (with finite capacity, essentially a pipeline)

Further reading: these lecture notes (Sape Mullender) and a description of the structure of the Plan9 window system (Pike)

25

How is it used? Typically, shared data is held in one process, and

accessed through commands sent through a channel (e.g., read, write)

Hence, no locks. Synchronization is enforced through channels. Hence, no race conditions

Clean fit to the design

Communication deadlock is possible, but it can often be detected early by analyzing the communication “skeleton”

26

Programming Languages: Shared-variable concurrency High-Performance Fortran (HPF), OpenMP, X10,

C*: designed for high-performance, scientific computing. “forall” construct, data layout specification

Java: Threads, synchronized methods, external locks. Also, lock-free implementations of basic data structures in java.util.concurrent

C/C++: concurrency provided by external thread libraries (e.g., pthreads). C++ will include language constructs for concurrency in its next version

OCaml/F#: OCaml and F# use external thread libraries. F# has a concurrent garbage collector

27

Programming Languages: Message-passing concurrency Ada: processes (called tasks) communicating

through “rendezvous” operations (like CSP ! and ?)

Erlang: processes communicating though unbounded message queues. Used by Ericsson for programming telephony switches

MPI: Message-passing interface. Used for wide-area and supercomputer applications

28

Session Data Types: SimpleManagement of the Life-cycle of Session-oriented Services

Al Aho*, Dennis Dams, Rick Hull, John Letourneau, Kedar Namjoshi, Frans Panken, Mans van Tellingen

Alcatel-Lucent and *Columbia University

Bell Labs Research

29

Session Data Types

A new project at Bell Labs

a simple model and a (fairly simple) API for programming real-time communications

Targets applications that mix web-based setup and control with real-time communications via telephony, video, and IM

Model based around the key concept of a session.

30

The “impedance mismatch”

Programming of converged services: programmers still think in multiple paradigms– Web: HTTP, SOAP, WSDL, XML

– All-IP Telecom: SIP, IMS, Parlay/ParlayX, AIN, PacketTV, XMPP, …

Web Services

App Server

SIP App Server

(including SB)

Other?

Eclipse-based SCE

Devices

Service Developer/

Blender

Device OS’s and

platforms

SIP, SIP

Servlets, SB

Steplets, …

Web Services,SOAP,

WSDL, …

. .

.

31

SDT Model

Three basic building blocks

Party: e.g., kedar@cellphone, dennis@IMserver, aho@laptop

Bubble: a group of parties in conversation. The group could be empty, or have a single person in it (usually a temporary situation)

Session: a group of bubbles, related in some way

And three key concepts

Application: a control program

State Machine: Each party is limited to behavior specified by a state machine over events such as “pickup”, “drop”, “mute”

Notify-Respond: the application is notified for events it registers for. It can trigger changes to a state machine or to a session in response

32

SDT Model

Session 1

Bob

Alice

Session Manager

Global Address Book /

Social Network

DeborahCharles

Bubble 1 Bubble 2

[ people, devices, friends ]

33

“Hello World” in SDT// register as a client for a session mangerISessionManager sm = SessionManager.register(“config.properties”, this);

// start a session with a predefined audio bubbleAudioSDT s = new AudioSDT(sm);AudioBubble b = s.getBubble();

// add yourself and a friendb.addParty(new PartyID(“Kedar”, “19085821891”), Role.ReadWrite);b.addParty(new PartyID(“Dennis”, “19085828859”), Role.ReadWrite);

// …… talk …… System.out.println(“Hello World!”);Thread.sleep(whatever);

// exit: sm shuts down sessions and bubbles

34

A bit more …// set up a handler for pickup actions

public class pickupHandler implements IHandler {

public Response run(TriggerBinding[] b) {

num_pickup++;

return Response.Continue;

}

}

// link the handler to the pickup action

PartyVar who = new PartyVar(“who”);

Expr e = b.mkEvent(AudioBubble.SM.Events.pickup(),who);

b.addTrigger(e, new pickupHandler());

35

And that’s it!The session/bubble model is simple, yet very flexible, thanks to

the notify-respond mechanism.

Typically, the application program sets up sessions and bubbles and waits for events to happen

The response to an event may create (or delete) sessions and bubbles, add (or drop) parties, or create new event triggers, all at run-time

This results in an incredible amount of flexibility, out of a very few basic primitives.

36

What we have done

We have an implementation of this model, running on an externally available server

We have written sample applications, including:

A customizable, network-hosted, speed-conference Through a web interface, set your phone buttons to perform SDT actions. As the service is network-hosted, these bindings work from anywhere. [subject to some fine print]

A conference system with the ability to create “whisper” sub-sessions

37

What you might do Design a scripting language that hides some of the

Java intricacies

Design a graphical, web-based language to implement your own view of speed dial

Create a conferencing application which you can distribute through Facebook

… or something far more interesting!

We are available throughout the semester for help with the API and the system.