Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel...
Transcript of Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel...
![Page 1: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/1.jpg)
Parallel Languages
We now have a brief look at some languages that weredesigned specifically with parallelism in mind
• Occam (channels)• Erlang (explicit parallelism)• Go (explicit parallelism)• Rust (explicit parallelism)• SISAL (implicit parallelism)• Strand (declarative)
Picked pretty much at random: by no means an exhaustive oreven comprehensive list, many other languages exist
1
![Page 2: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/2.jpg)
Parallel Languages
We now have a brief look at some languages that weredesigned specifically with parallelism in mind
• Occam (channels)• Erlang (explicit parallelism)• Go (explicit parallelism)• Rust (explicit parallelism)• SISAL (implicit parallelism)• Strand (declarative)
Picked pretty much at random: by no means an exhaustive oreven comprehensive list, many other languages exist
2
![Page 3: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/3.jpg)
Occam
Occam was a language that was based on CommunicatingSequential Processes (CSP) a theoretical model of parallelcomputation (c.f., Lambda Calculus)
CSP models processes that communicate by passingmessages between themselves along channels
There are various rules on combining processes anddescriptions on how these combined objects behave
Then theoreticians get busy on proving that behaviours ofvarious systems are equivalent (or not)
3
![Page 4: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/4.jpg)
Occam
Occam was a language that was based on CommunicatingSequential Processes (CSP) a theoretical model of parallelcomputation (c.f., Lambda Calculus)
CSP models processes that communicate by passingmessages between themselves along channels
There are various rules on combining processes anddescriptions on how these combined objects behave
Then theoreticians get busy on proving that behaviours ofvarious systems are equivalent (or not)
4
![Page 5: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/5.jpg)
Occam
Occam was a language that was based on CommunicatingSequential Processes (CSP) a theoretical model of parallelcomputation (c.f., Lambda Calculus)
CSP models processes that communicate by passingmessages between themselves along channels
There are various rules on combining processes anddescriptions on how these combined objects behave
Then theoreticians get busy on proving that behaviours ofvarious systems are equivalent (or not)
5
![Page 6: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/6.jpg)
Occam
Occam was a language that was based on CommunicatingSequential Processes (CSP) a theoretical model of parallelcomputation (c.f., Lambda Calculus)
CSP models processes that communicate by passingmessages between themselves along channels
There are various rules on combining processes anddescriptions on how these combined objects behave
Then theoreticians get busy on proving that behaviours ofvarious systems are equivalent (or not)
6
![Page 7: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/7.jpg)
Occam
Occam was a realisation of CSP, designed hand-in-hand withthe hardware it would run on: the transputer
The transputer (early 1980s) was going to be the future ofparallel processing: a new hardware architecture explicitlysupporting message passing between cores
Unfortunately, the level of technology of the time was not reallyup to the task: they had problems with clock speeds and heatmanagement
There was no real advantage to using a transputer overexisting, classical processors (like Intel), so it never managed tosell in numbers large enough to be successful
But the transputer was designed primarily to run Occam
7
![Page 8: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/8.jpg)
Occam
Occam was a realisation of CSP, designed hand-in-hand withthe hardware it would run on: the transputer
The transputer (early 1980s) was going to be the future ofparallel processing: a new hardware architecture explicitlysupporting message passing between cores
Unfortunately, the level of technology of the time was not reallyup to the task: they had problems with clock speeds and heatmanagement
There was no real advantage to using a transputer overexisting, classical processors (like Intel), so it never managed tosell in numbers large enough to be successful
But the transputer was designed primarily to run Occam
8
![Page 9: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/9.jpg)
Occam
Occam was a realisation of CSP, designed hand-in-hand withthe hardware it would run on: the transputer
The transputer (early 1980s) was going to be the future ofparallel processing: a new hardware architecture explicitlysupporting message passing between cores
Unfortunately, the level of technology of the time was not reallyup to the task: they had problems with clock speeds and heatmanagement
There was no real advantage to using a transputer overexisting, classical processors (like Intel), so it never managed tosell in numbers large enough to be successful
But the transputer was designed primarily to run Occam
9
![Page 10: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/10.jpg)
Occam
Occam was a realisation of CSP, designed hand-in-hand withthe hardware it would run on: the transputer
The transputer (early 1980s) was going to be the future ofparallel processing: a new hardware architecture explicitlysupporting message passing between cores
Unfortunately, the level of technology of the time was not reallyup to the task: they had problems with clock speeds and heatmanagement
There was no real advantage to using a transputer overexisting, classical processors (like Intel), so it never managed tosell in numbers large enough to be successful
But the transputer was designed primarily to run Occam
10
![Page 11: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/11.jpg)
Occam
Occam was a realisation of CSP, designed hand-in-hand withthe hardware it would run on: the transputer
The transputer (early 1980s) was going to be the future ofparallel processing: a new hardware architecture explicitlysupporting message passing between cores
Unfortunately, the level of technology of the time was not reallyup to the task: they had problems with clock speeds and heatmanagement
There was no real advantage to using a transputer overexisting, classical processors (like Intel), so it never managed tosell in numbers large enough to be successful
But the transputer was designed primarily to run Occam
11
![Page 12: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/12.jpg)
OccamOccam has explicit parallelism of tasks:
PAR
f(x)
g(y)
runs f and g concurrently
More unusually, Occam has explicit sequentiality:
SEQ
f(x)
g(y)
runs f, then g
This is because in CSP sequential composition of code is ofequal note to parallel composition of code
12
![Page 13: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/13.jpg)
OccamOccam has explicit parallelism of tasks:
PAR
f(x)
g(y)
runs f and g concurrently
More unusually, Occam has explicit sequentiality:
SEQ
f(x)
g(y)
runs f, then g
This is because in CSP sequential composition of code is ofequal note to parallel composition of code
13
![Page 14: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/14.jpg)
OccamOccam has explicit parallelism of tasks:
PAR
f(x)
g(y)
runs f and g concurrently
More unusually, Occam has explicit sequentiality:
SEQ
f(x)
g(y)
runs f, then g
This is because in CSP sequential composition of code is ofequal note to parallel composition of code
14
![Page 15: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/15.jpg)
Occam
Communication between processes is via channels
ch ! x
writes the value of x down the channel named ch
ch ? y
reads the value of x from the channel named ch
Both are blocking: the write will wait for the corresponding read;the read will wait for the corresponding write
15
![Page 16: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/16.jpg)
Occam
Communication between processes is via channels
ch ! x
writes the value of x down the channel named ch
ch ? y
reads the value of x from the channel named ch
Both are blocking: the write will wait for the corresponding read;the read will wait for the corresponding write
16
![Page 17: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/17.jpg)
Occam
Thus we get communication and synchronisation betweenthreads
INT x:
CHAN INT ch:
PAR
SEQ
print("hello")
ch ! 42
SEQ
ch ? x
print(" world")
will print "hello world"
17
![Page 18: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/18.jpg)
Occam
There is also non-deterministic choice
ALT
in1 ? x
SEQ
x := x+1
out1 ! x
in2 ? x
SEQ
x := x-1
out2 ! x
will wait until data arrives on channel in1 or in2 and will thenexecute the relevant section of code
If data arrives on both simultaneously, one branch will be takennon-deterministically
18
![Page 19: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/19.jpg)
Occam
There is also non-deterministic choice
ALT
in1 ? x
SEQ
x := x+1
out1 ! x
in2 ? x
SEQ
x := x-1
out2 ! x
will wait until data arrives on channel in1 or in2 and will thenexecute the relevant section of code
If data arrives on both simultaneously, one branch will be takennon-deterministically
19
![Page 20: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/20.jpg)
Occam
The only way for tasks to communicate is via channels
There is no concept of shared or distributed, so a programshould work equally on shared or distributed memory
This is a bit like MPI messaging: it provides independence fromthe hardware
20
![Page 21: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/21.jpg)
Occam
The only way for tasks to communicate is via channels
There is no concept of shared or distributed, so a programshould work equally on shared or distributed memory
This is a bit like MPI messaging: it provides independence fromthe hardware
21
![Page 22: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/22.jpg)
Occam
The only way for tasks to communicate is via channels
There is no concept of shared or distributed, so a programshould work equally on shared or distributed memory
This is a bit like MPI messaging: it provides independence fromthe hardware
22
![Page 23: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/23.jpg)
Occam
Plus loads more features: boolean guards (on ALT); timeoutson guards; priority ordered ALTs; functions; procedures; arrays;while loops; etc.
A program is a bunch of processes (threads in modern terms),joined by PARs, that send data along channels to each other
By being closely related to CSP, there were opportunities to doproofs on Occam programs
23
![Page 24: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/24.jpg)
Occam
Plus loads more features: boolean guards (on ALT); timeoutson guards; priority ordered ALTs; functions; procedures; arrays;while loops; etc.
A program is a bunch of processes (threads in modern terms),joined by PARs, that send data along channels to each other
By being closely related to CSP, there were opportunities to doproofs on Occam programs
24
![Page 25: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/25.jpg)
Occam
Plus loads more features: boolean guards (on ALT); timeoutson guards; priority ordered ALTs; functions; procedures; arrays;while loops; etc.
A program is a bunch of processes (threads in modern terms),joined by PARs, that send data along channels to each other
By being closely related to CSP, there were opportunities to doproofs on Occam programs
25
![Page 26: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/26.jpg)
Occam
Occam never took off as the hardware of the time (transputer)was not really up to it
Programmers never got the hang of it, either
It might be better used as a scripting language to control otherparallel programs
There was an extension: Occam-π. This was a realisation ofthe π-calculus, which is itself a generalisation of CSP, wherechannels and processes are first class objects, e.g., pass achannel down a channel
A good model to revisit in light of the current obsession withmobile processes
26
![Page 27: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/27.jpg)
Occam
Occam never took off as the hardware of the time (transputer)was not really up to it
Programmers never got the hang of it, either
It might be better used as a scripting language to control otherparallel programs
There was an extension: Occam-π. This was a realisation ofthe π-calculus, which is itself a generalisation of CSP, wherechannels and processes are first class objects, e.g., pass achannel down a channel
A good model to revisit in light of the current obsession withmobile processes
27
![Page 28: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/28.jpg)
Occam
Occam never took off as the hardware of the time (transputer)was not really up to it
Programmers never got the hang of it, either
It might be better used as a scripting language to control otherparallel programs
There was an extension: Occam-π. This was a realisation ofthe π-calculus, which is itself a generalisation of CSP, wherechannels and processes are first class objects, e.g., pass achannel down a channel
A good model to revisit in light of the current obsession withmobile processes
28
![Page 29: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/29.jpg)
Occam
Occam never took off as the hardware of the time (transputer)was not really up to it
Programmers never got the hang of it, either
It might be better used as a scripting language to control otherparallel programs
There was an extension: Occam-π. This was a realisation ofthe π-calculus, which is itself a generalisation of CSP, wherechannels and processes are first class objects, e.g., pass achannel down a channel
A good model to revisit in light of the current obsession withmobile processes
29
![Page 30: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/30.jpg)
Occam
Occam never took off as the hardware of the time (transputer)was not really up to it
Programmers never got the hang of it, either
It might be better used as a scripting language to control otherparallel programs
There was an extension: Occam-π. This was a realisation ofthe π-calculus, which is itself a generalisation of CSP, wherechannels and processes are first class objects, e.g., pass achannel down a channel
A good model to revisit in light of the current obsession withmobile processes
30
![Page 31: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/31.jpg)
Erlang
Erlang is a single assignment functional language, with explicitsupport for MIMD parallelism
A program can contain a large number of very lightweightthreads: 20 million is possible they claim
Thus these threads do not correspond directly to OS threads,but are managed by the Erlang runtime (a VM)
Having no shared state, the threads act more like OSprocesses than normal threads
31
![Page 32: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/32.jpg)
Erlang
Erlang is a single assignment functional language, with explicitsupport for MIMD parallelism
A program can contain a large number of very lightweightthreads: 20 million is possible they claim
Thus these threads do not correspond directly to OS threads,but are managed by the Erlang runtime (a VM)
Having no shared state, the threads act more like OSprocesses than normal threads
32
![Page 33: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/33.jpg)
Erlang
Erlang is a single assignment functional language, with explicitsupport for MIMD parallelism
A program can contain a large number of very lightweightthreads: 20 million is possible they claim
Thus these threads do not correspond directly to OS threads,but are managed by the Erlang runtime (a VM)
Having no shared state, the threads act more like OSprocesses than normal threads
33
![Page 34: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/34.jpg)
Erlang
Erlang is a single assignment functional language, with explicitsupport for MIMD parallelism
A program can contain a large number of very lightweightthreads: 20 million is possible they claim
Thus these threads do not correspond directly to OS threads,but are managed by the Erlang runtime (a VM)
Having no shared state, the threads act more like OSprocesses than normal threads
34
![Page 35: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/35.jpg)
Erlang
They do not share state because the processes (they call theirthreads “processes”) may be on distributed memory
Two processes might be on the same local shared memory, butyou cannot assume that
Also, this fits in nicely with the functional style: everything islocal to the process and everything is referentially transparent
An important consideration is that the overheads of creation,destruction and context switching are very small: thusencouraging many small, single-use processes
35
![Page 36: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/36.jpg)
Erlang
They do not share state because the processes (they call theirthreads “processes”) may be on distributed memory
Two processes might be on the same local shared memory, butyou cannot assume that
Also, this fits in nicely with the functional style: everything islocal to the process and everything is referentially transparent
An important consideration is that the overheads of creation,destruction and context switching are very small: thusencouraging many small, single-use processes
36
![Page 37: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/37.jpg)
Erlang
They do not share state because the processes (they call theirthreads “processes”) may be on distributed memory
Two processes might be on the same local shared memory, butyou cannot assume that
Also, this fits in nicely with the functional style: everything islocal to the process and everything is referentially transparent
An important consideration is that the overheads of creation,destruction and context switching are very small: thusencouraging many small, single-use processes
37
![Page 38: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/38.jpg)
Erlang
They do not share state because the processes (they call theirthreads “processes”) may be on distributed memory
Two processes might be on the same local shared memory, butyou cannot assume that
Also, this fits in nicely with the functional style: everything islocal to the process and everything is referentially transparent
An important consideration is that the overheads of creation,destruction and context switching are very small: thusencouraging many small, single-use processes
38
![Page 39: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/39.jpg)
Erlang
An Erlang runtime will typically run one OS-style thread percore; each running an Erlang scheduler
These schedulers will choose and run the Erlang-style threadsin a manager/worker fashion
Thus it avoids the overhead of OS thread creation/deletion
An Erlang thread would typically require approximately 600bytes of state
39
![Page 40: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/40.jpg)
Erlang
An Erlang runtime will typically run one OS-style thread percore; each running an Erlang scheduler
These schedulers will choose and run the Erlang-style threadsin a manager/worker fashion
Thus it avoids the overhead of OS thread creation/deletion
An Erlang thread would typically require approximately 600bytes of state
40
![Page 41: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/41.jpg)
Erlang
An Erlang runtime will typically run one OS-style thread percore; each running an Erlang scheduler
These schedulers will choose and run the Erlang-style threadsin a manager/worker fashion
Thus it avoids the overhead of OS thread creation/deletion
An Erlang thread would typically require approximately 600bytes of state
41
![Page 42: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/42.jpg)
Erlang
An Erlang runtime will typically run one OS-style thread percore; each running an Erlang scheduler
These schedulers will choose and run the Erlang-style threadsin a manager/worker fashion
Thus it avoids the overhead of OS thread creation/deletion
An Erlang thread would typically require approximately 600bytes of state
42
![Page 43: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/43.jpg)
Erlang
Erlang threads communicate via messages like Occam/CSP,but they are asynchronous, unlike Occam/CSP
Again, messages works equally over shared and distributedmemory
Also, Erlang does not have named channels, but each processhas a “mailbox” where it receives all its messages
The messages can be values, tuples of values, or any otherdatatype, including closures (functions)
And there is pattern matching to fetch messages from themailbox (a bit like MPI tags, but more general matching)
43
![Page 44: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/44.jpg)
Erlang
Erlang threads communicate via messages like Occam/CSP,but they are asynchronous, unlike Occam/CSP
Again, messages works equally over shared and distributedmemory
Also, Erlang does not have named channels, but each processhas a “mailbox” where it receives all its messages
The messages can be values, tuples of values, or any otherdatatype, including closures (functions)
And there is pattern matching to fetch messages from themailbox (a bit like MPI tags, but more general matching)
44
![Page 45: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/45.jpg)
Erlang
Erlang threads communicate via messages like Occam/CSP,but they are asynchronous, unlike Occam/CSP
Again, messages works equally over shared and distributedmemory
Also, Erlang does not have named channels, but each processhas a “mailbox” where it receives all its messages
The messages can be values, tuples of values, or any otherdatatype, including closures (functions)
And there is pattern matching to fetch messages from themailbox (a bit like MPI tags, but more general matching)
45
![Page 46: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/46.jpg)
Erlang
Erlang threads communicate via messages like Occam/CSP,but they are asynchronous, unlike Occam/CSP
Again, messages works equally over shared and distributedmemory
Also, Erlang does not have named channels, but each processhas a “mailbox” where it receives all its messages
The messages can be values, tuples of values, or any otherdatatype, including closures (functions)
And there is pattern matching to fetch messages from themailbox (a bit like MPI tags, but more general matching)
46
![Page 47: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/47.jpg)
Erlang
Erlang threads communicate via messages like Occam/CSP,but they are asynchronous, unlike Occam/CSP
Again, messages works equally over shared and distributedmemory
Also, Erlang does not have named channels, but each processhas a “mailbox” where it receives all its messages
The messages can be values, tuples of values, or any otherdatatype, including closures (functions)
And there is pattern matching to fetch messages from themailbox (a bit like MPI tags, but more general matching)
47
![Page 48: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/48.jpg)
Erlang
Otherproc ! { hello, 99 }
sends a tuple with atom (like a Lisp symbol) hello and theinteger 99 to the process named by Otherproc (variables startwith capital letters)
receive
{ hello, X } -> io:format("x was ~B~n", [X]);
{ bye, X } -> io:format("time to go~n", []);
_ -> io:format("eh?~n", [])
end.
an underscore matches any message
48
![Page 49: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/49.jpg)
Erlang
Otherproc ! { hello, 99 }
sends a tuple with atom (like a Lisp symbol) hello and theinteger 99 to the process named by Otherproc (variables startwith capital letters)
receive
{ hello, X } -> io:format("x was ~B~n", [X]);
{ bye, X } -> io:format("time to go~n", []);
_ -> io:format("eh?~n", [])
end.
an underscore matches any message
49
![Page 50: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/50.jpg)
Erlang
Creation of processes is via spawn
factrec(0) -> 1;
factrec(N) when N > 0 -> N*factrec(N-1).
fact(N, Ans) -> Ans ! factrec(N).
FactPid = spawn(fact, [5, self()]).
receive
F -> io:format("factorial is ~B~n", [F])
end.
is clumsy code to make a new process running fact witharguments 5 and the PID of the current process
The receive causes the current process (self()) to wait for amessage (from anyone), and stores it in F
50
![Page 51: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/51.jpg)
Erlang
Creation of processes is via spawn
factrec(0) -> 1;
factrec(N) when N > 0 -> N*factrec(N-1).
fact(N, Ans) -> Ans ! factrec(N).
FactPid = spawn(fact, [5, self()]).
receive
F -> io:format("factorial is ~B~n", [F])
end.
is clumsy code to make a new process running fact witharguments 5 and the PID of the current process
The receive causes the current process (self()) to wait for amessage (from anyone), and stores it in F
51
![Page 52: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/52.jpg)
Erlang
A PID is the way you refer to a process, in particular for sendinga message to it
N.B. some liberties taken with Erlang modules here
52
![Page 53: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/53.jpg)
Erlang
Erlang is quite popular in real systems as it has lots of usefulfeatures
For example, Process Restart , where a process is immediatelyrestarted by the runtime if it crashes for any reason
This allows Erlang to cope with hardware failure and buggycode
In fact, Erlang has hot swap of code: code can be changed(fixed or updated) while the main program is running
Load balancing of processes is done by the runtime VM
53
![Page 54: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/54.jpg)
Erlang
Erlang is quite popular in real systems as it has lots of usefulfeatures
For example, Process Restart , where a process is immediatelyrestarted by the runtime if it crashes for any reason
This allows Erlang to cope with hardware failure and buggycode
In fact, Erlang has hot swap of code: code can be changed(fixed or updated) while the main program is running
Load balancing of processes is done by the runtime VM
54
![Page 55: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/55.jpg)
Erlang
Erlang is quite popular in real systems as it has lots of usefulfeatures
For example, Process Restart , where a process is immediatelyrestarted by the runtime if it crashes for any reason
This allows Erlang to cope with hardware failure and buggycode
In fact, Erlang has hot swap of code: code can be changed(fixed or updated) while the main program is running
Load balancing of processes is done by the runtime VM
55
![Page 56: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/56.jpg)
Erlang
Erlang is quite popular in real systems as it has lots of usefulfeatures
For example, Process Restart , where a process is immediatelyrestarted by the runtime if it crashes for any reason
This allows Erlang to cope with hardware failure and buggycode
In fact, Erlang has hot swap of code: code can be changed(fixed or updated) while the main program is running
Load balancing of processes is done by the runtime VM
56
![Page 57: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/57.jpg)
Erlang
Erlang is quite popular in real systems as it has lots of usefulfeatures
For example, Process Restart , where a process is immediatelyrestarted by the runtime if it crashes for any reason
This allows Erlang to cope with hardware failure and buggycode
In fact, Erlang has hot swap of code: code can be changed(fixed or updated) while the main program is running
Load balancing of processes is done by the runtime VM
57
![Page 58: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/58.jpg)
Erlang
Originally designed by Ericsson to support (soft) realtimesystems that can’t be taken down for maintenance (liketelephone exchanges), it has found use in other areas
Companies like Yahoo, Facebook, WhatsApp, Bet365, etc. useit for some element of their products
Somewhat an under-appreciated language
58
![Page 59: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/59.jpg)
Erlang
Originally designed by Ericsson to support (soft) realtimesystems that can’t be taken down for maintenance (liketelephone exchanges), it has found use in other areas
Companies like Yahoo, Facebook, WhatsApp, Bet365, etc. useit for some element of their products
Somewhat an under-appreciated language
59
![Page 60: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/60.jpg)
Erlang
Originally designed by Ericsson to support (soft) realtimesystems that can’t be taken down for maintenance (liketelephone exchanges), it has found use in other areas
Companies like Yahoo, Facebook, WhatsApp, Bet365, etc. useit for some element of their products
Somewhat an under-appreciated language
60
![Page 61: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/61.jpg)
Go
Go (also called “Golang”) is a language devised by the peoplewho gave us C
It tries to have the simplicity of C, but with modern languagefeatures too, such as packages, type safety, type inference,closures and garbage collection
61
![Page 62: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/62.jpg)
Go
Go (also called “Golang”) is a language devised by the peoplewho gave us C
It tries to have the simplicity of C, but with modern languagefeatures too, such as packages, type safety, type inference,closures and garbage collection
62
![Page 63: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/63.jpg)
Go
Go implements concurrency by threads (called goroutines)communicating via channels, similar to Occam
Channels are type safe (“channel of int”) and blocking
There is a select that acts like Occam’s ALT waiting onmultiple channels
Exercise. “goroutine” is a play on “coroutine”, an early(non-parallel) concurrency technique. Read about this (and alimited form of coroutine, called a generator )
63
![Page 64: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/64.jpg)
Go
Go implements concurrency by threads (called goroutines)communicating via channels, similar to Occam
Channels are type safe (“channel of int”) and blocking
There is a select that acts like Occam’s ALT waiting onmultiple channels
Exercise. “goroutine” is a play on “coroutine”, an early(non-parallel) concurrency technique. Read about this (and alimited form of coroutine, called a generator )
64
![Page 65: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/65.jpg)
Go
Go implements concurrency by threads (called goroutines)communicating via channels, similar to Occam
Channels are type safe (“channel of int”) and blocking
There is a select that acts like Occam’s ALT waiting onmultiple channels
Exercise. “goroutine” is a play on “coroutine”, an early(non-parallel) concurrency technique. Read about this (and alimited form of coroutine, called a generator )
65
![Page 66: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/66.jpg)
Go
Go implements concurrency by threads (called goroutines)communicating via channels, similar to Occam
Channels are type safe (“channel of int”) and blocking
There is a select that acts like Occam’s ALT waiting onmultiple channels
Exercise. “goroutine” is a play on “coroutine”, an early(non-parallel) concurrency technique. Read about this (and alimited form of coroutine, called a generator )
66
![Page 67: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/67.jpg)
Go
Creating new threads is simple
go fun(x+y, x-y)
creates a new asynchronous thread running fun with thevalues of those arguments
Similar to Erlang, goroutines are lightweight and scheduled bythe Go runtime across OS threads
You can create “1000s” of goroutines
But this means the Go runtime is very complex (like Erlang)
67
![Page 68: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/68.jpg)
Go
Creating new threads is simple
go fun(x+y, x-y)
creates a new asynchronous thread running fun with thevalues of those arguments
Similar to Erlang, goroutines are lightweight and scheduled bythe Go runtime across OS threads
You can create “1000s” of goroutines
But this means the Go runtime is very complex (like Erlang)
68
![Page 69: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/69.jpg)
Go
Creating new threads is simple
go fun(x+y, x-y)
creates a new asynchronous thread running fun with thevalues of those arguments
Similar to Erlang, goroutines are lightweight and scheduled bythe Go runtime across OS threads
You can create “1000s” of goroutines
But this means the Go runtime is very complex (like Erlang)
69
![Page 70: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/70.jpg)
Go
Creating new threads is simple
go fun(x+y, x-y)
creates a new asynchronous thread running fun with thevalues of those arguments
Similar to Erlang, goroutines are lightweight and scheduled bythe Go runtime across OS threads
You can create “1000s” of goroutines
But this means the Go runtime is very complex (like Erlang)
70
![Page 71: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/71.jpg)
Go
Synchronisation and communication are provided by channels
Libraries provide condition variables, mutexes, atomics and avariety of other low-level functionality
Channels are the recommended ways of passing data betweenthreads; though you can also use shared variables
Though shared variables are not recommended as Go providesno inherent protection against the usual data races (if you don’tremember to use mutexes and the like)
From the Go website (worth repeating!):
Share memory by communicating; don’t communicateby sharing memory.
71
![Page 72: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/72.jpg)
Go
Synchronisation and communication are provided by channels
Libraries provide condition variables, mutexes, atomics and avariety of other low-level functionality
Channels are the recommended ways of passing data betweenthreads; though you can also use shared variables
Though shared variables are not recommended as Go providesno inherent protection against the usual data races (if you don’tremember to use mutexes and the like)
From the Go website (worth repeating!):
Share memory by communicating; don’t communicateby sharing memory.
72
![Page 73: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/73.jpg)
Go
Synchronisation and communication are provided by channels
Libraries provide condition variables, mutexes, atomics and avariety of other low-level functionality
Channels are the recommended ways of passing data betweenthreads; though you can also use shared variables
Though shared variables are not recommended as Go providesno inherent protection against the usual data races (if you don’tremember to use mutexes and the like)
From the Go website (worth repeating!):
Share memory by communicating; don’t communicateby sharing memory.
73
![Page 74: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/74.jpg)
Go
Synchronisation and communication are provided by channels
Libraries provide condition variables, mutexes, atomics and avariety of other low-level functionality
Channels are the recommended ways of passing data betweenthreads; though you can also use shared variables
Though shared variables are not recommended as Go providesno inherent protection against the usual data races (if you don’tremember to use mutexes and the like)
From the Go website (worth repeating!):
Share memory by communicating; don’t communicateby sharing memory.
74
![Page 75: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/75.jpg)
Go
Synchronisation and communication are provided by channels
Libraries provide condition variables, mutexes, atomics and avariety of other low-level functionality
Channels are the recommended ways of passing data betweenthreads; though you can also use shared variables
Though shared variables are not recommended as Go providesno inherent protection against the usual data races (if you don’tremember to use mutexes and the like)
From the Go website (worth repeating!):
Share memory by communicating; don’t communicateby sharing memory.
75
![Page 76: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/76.jpg)
Go
Still relatively new as a language, Go was designed by peoplewith a considerable amount of expertise
It has a lot of promise, with some vocal proponents
. . . in fact, there isn’t much to Go other than channelsand goroutines!
Stjepan Glavina
76
![Page 77: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/77.jpg)
Go
Still relatively new as a language, Go was designed by peoplewith a considerable amount of expertise
It has a lot of promise, with some vocal proponents
. . . in fact, there isn’t much to Go other than channelsand goroutines!
Stjepan Glavina
77
![Page 78: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/78.jpg)
Go
Still relatively new as a language, Go was designed by peoplewith a considerable amount of expertise
It has a lot of promise, with some vocal proponents
. . . in fact, there isn’t much to Go other than channelsand goroutines!
Stjepan Glavina
78
![Page 79: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/79.jpg)
Rust
A language originally designed and developed by the Mozillateam, with the eventual aim of reimplementing the Firefoxbrowser in Rust, but now a language in its own right
As a lot of the problems in browsers (and many other largeprograms) are to do with bad memory management it is amemory safe language, meaning it can not have problems likedangling pointers, uninitialised variables or buffer overflows
It does this by having a concept of the owner of a memorylocation and tracking that ownership in the compiler
79
![Page 80: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/80.jpg)
Rust
A language originally designed and developed by the Mozillateam, with the eventual aim of reimplementing the Firefoxbrowser in Rust, but now a language in its own right
As a lot of the problems in browsers (and many other largeprograms) are to do with bad memory management it is amemory safe language, meaning it can not have problems likedangling pointers, uninitialised variables or buffer overflows
It does this by having a concept of the owner of a memorylocation and tracking that ownership in the compiler
80
![Page 81: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/81.jpg)
Rust
A language originally designed and developed by the Mozillateam, with the eventual aim of reimplementing the Firefoxbrowser in Rust, but now a language in its own right
As a lot of the problems in browsers (and many other largeprograms) are to do with bad memory management it is amemory safe language, meaning it can not have problems likedangling pointers, uninitialised variables or buffer overflows
It does this by having a concept of the owner of a memorylocation and tracking that ownership in the compiler
81
![Page 82: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/82.jpg)
Rust
In an assignment y = x; the ownership of the memoryreferred to by x is transferred to y. It is now illegal/impossible touse the variable x in subsequent code
The compiler would flag any later reference to x as an errorand refuse to compile
This helps with memory management, as the compiler canprecisely determine the lifetime of a value and so its memorycan be deallocated automatically when the compiler can proveit is not longer accessible (without the need for a garbagecollector)
Thus avoiding the programming errors common to C-likelanguages
82
![Page 83: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/83.jpg)
Rust
In an assignment y = x; the ownership of the memoryreferred to by x is transferred to y. It is now illegal/impossible touse the variable x in subsequent code
The compiler would flag any later reference to x as an errorand refuse to compile
This helps with memory management, as the compiler canprecisely determine the lifetime of a value and so its memorycan be deallocated automatically when the compiler can proveit is not longer accessible (without the need for a garbagecollector)
Thus avoiding the programming errors common to C-likelanguages
83
![Page 84: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/84.jpg)
Rust
In an assignment y = x; the ownership of the memoryreferred to by x is transferred to y. It is now illegal/impossible touse the variable x in subsequent code
The compiler would flag any later reference to x as an errorand refuse to compile
This helps with memory management, as the compiler canprecisely determine the lifetime of a value and so its memorycan be deallocated automatically when the compiler can proveit is not longer accessible (without the need for a garbagecollector)
Thus avoiding the programming errors common to C-likelanguages
84
![Page 85: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/85.jpg)
Rust
In an assignment y = x; the ownership of the memoryreferred to by x is transferred to y. It is now illegal/impossible touse the variable x in subsequent code
The compiler would flag any later reference to x as an errorand refuse to compile
This helps with memory management, as the compiler canprecisely determine the lifetime of a value and so its memorycan be deallocated automatically when the compiler can proveit is not longer accessible (without the need for a garbagecollector)
Thus avoiding the programming errors common to C-likelanguages
85
![Page 86: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/86.jpg)
Rust
The good thing is that this also helps with data races
A data race can happen when one memory location isaccessed by two threads, at least one doing a write
A read-only value can be shared; a read-write (mutable) valuecan’t be shared (c.f., RW locks)
The compiler will spot an attempt to modify a shared value andrefuse to compile
86
![Page 87: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/87.jpg)
Rust
The good thing is that this also helps with data races
A data race can happen when one memory location isaccessed by two threads, at least one doing a write
A read-only value can be shared; a read-write (mutable) valuecan’t be shared (c.f., RW locks)
The compiler will spot an attempt to modify a shared value andrefuse to compile
87
![Page 88: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/88.jpg)
Rust
The good thing is that this also helps with data races
A data race can happen when one memory location isaccessed by two threads, at least one doing a write
A read-only value can be shared; a read-write (mutable) valuecan’t be shared (c.f., RW locks)
The compiler will spot an attempt to modify a shared value andrefuse to compile
88
![Page 89: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/89.jpg)
Rust
The good thing is that this also helps with data races
A data race can happen when one memory location isaccessed by two threads, at least one doing a write
A read-only value can be shared; a read-write (mutable) valuecan’t be shared (c.f., RW locks)
The compiler will spot an attempt to modify a shared value andrefuse to compile
89
![Page 90: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/90.jpg)
Rust
Thus making the programmer face up to the problem and fix itbefore the code will even compile
Rust developers call this “Fearless Concurrency” as thelanguage itself prevents these kinds of data-race
This fixes data races: unfortunately the Rust compiler is not(yet?) able to spot non-data-race race conditions, like deadlock
90
![Page 91: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/91.jpg)
Rust
Thus making the programmer face up to the problem and fix itbefore the code will even compile
Rust developers call this “Fearless Concurrency” as thelanguage itself prevents these kinds of data-race
This fixes data races: unfortunately the Rust compiler is not(yet?) able to spot non-data-race race conditions, like deadlock
91
![Page 92: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/92.jpg)
Rust
Thus making the programmer face up to the problem and fix itbefore the code will even compile
Rust developers call this “Fearless Concurrency” as thelanguage itself prevents these kinds of data-race
This fixes data races: unfortunately the Rust compiler is not(yet?) able to spot non-data-race race conditions, like deadlock
92
![Page 93: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/93.jpg)
Rust
Rust has a classical threaded approach, with a thread modulethat contains functions like thread::spawn()
This takes a closure as argument as the thing to execute
let threadid = thread::spawn(|| foo(x+1,y-1));
...let val = threadid.join().unwrap();
93
![Page 94: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/94.jpg)
Rust
Rust has a classical threaded approach, with a thread modulethat contains functions like thread::spawn()
This takes a closure as argument as the thing to execute
let threadid = thread::spawn(|| foo(x+1,y-1));
...let val = threadid.join().unwrap();
94
![Page 95: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/95.jpg)
Rust
Rust has a classical threaded approach, with a thread modulethat contains functions like thread::spawn()
This takes a closure as argument as the thing to execute
let threadid = thread::spawn(|| foo(x+1,y-1));
...let val = threadid.join().unwrap();
95
![Page 96: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/96.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
96
![Page 97: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/97.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
97
![Page 98: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/98.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
98
![Page 99: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/99.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
99
![Page 100: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/100.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
100
![Page 101: Parallel Computing CM30225 · Sequential Processes (CSP) a theoretical model of parallel computation (c.f., Lambda Calculus) CSP models processes that communicate by passing messages](https://reader030.fdocuments.net/reader030/viewer/2022040914/5e8a9ed240ecaf52b01d4254/html5/thumbnails/101.jpg)
RustA Mutex can be used to wrap any data:let mtx = Mutex::new(v);
And now the only way of accessing the data that used to be inv is via the mutex: let mut data = mtx.lock().unwrap();
The ownership of the value has passed to within the mutex mtx,and so is no longer available from v
This prevents accidental direct access to the data, and this ischecked and enforced by the compiler
There is no unlock method: the mutex automatically unlockswhen the holder goes out of scope
Thus the programmer can’t forget to unlock a mutex, or unlock itin the wrong place, or access the data without using the mutex
101