Static Analysis of Communication Structures in Parallel Programs
description
Transcript of Static Analysis of Communication Structures in Parallel Programs
![Page 1: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/1.jpg)
Static Analysis of Static Analysis of Communication Structures in Communication Structures in
Parallel ProgramsParallel Programs
So-Yan Ho and Nai-Wei Lin
Department of Computer Science and Information Engineering
National Chung Cheng University
![Page 2: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/2.jpg)
2
MotivationsMotivations
• Communication structures in message passing programs is the key to understand the behaviors of the programs
• Static analysis of communication structures allows programmers to identify potential communication bugs at compile time
![Page 3: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/3.jpg)
3
Message Passing SchemesMessage Passing Schemes
• Direct-Addressing: CSP, Occam, Ada, MPI
send(task-id, message);message = receive(task-id);
• Indirect-Addressing: Fortran M, CCCsend(channel-id, message);message = receive(channel-id);
![Page 4: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/4.jpg)
4
Channels in CCCChannels in CCC
• Pipes: one-to-one communication
• Spliters: one-to-many communication
• Mergers: many-to-one communication
• Multiplexers: many-to-many communication
![Page 5: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/5.jpg)
5
Merits of Abundant ChannelsMerits of Abundant Channels
• Communication structures are more comprehensive
• The specification of communication structures is easier
• The implementation of communication structures is more efficient
• The static analysis of communication structures is more effective
![Page 6: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/6.jpg)
6
Higher-Order ChannelsHigher-Order Channels
• A channel is first-order if it is used to transfer data
• A channel is higher-order if it is used to transfer channels
• The static analysis of higher-order channels is much more difficult than the static analysis of first-order channels
![Page 7: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/7.jpg)
7
A Simple CCC ProgramA Simple CCC Program
task::main(){ spliter int ch; /* one-to-many channel */ int i;
ch = channel(); par { producer(ch, many); /* one sender */ parfor (i = 0; i < many; i++) consumer(ch); /* many receivers */ }}
![Page 8: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/8.jpg)
8
A Simple CCC ProgramA Simple CCC Program
task::producer(ch, many)spliter int ch;int many;{ int i; for (i = 0; i < num_data; i++) send(ch, i); for (i = 0; i < many; i++) send(ch, end_data); /* signal end of data */}
![Page 9: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/9.jpg)
9
A Simple CCC ProgramA Simple CCC Program
task::consumer(ch)spliter int ch;{ int data; while (1) { data = receive(ch); if (data == end_data) break; process(data); }}
![Page 10: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/10.jpg)
10
Overview of the AnalysisOverview of the Analysis
• For each channel ch, infer the number of senders of ch, ch.s and the number of receivers of ch, ch.r
• Intraprocedural analysis: within a function– infer them once and for all, and use them as the
initial values of the interprocedural analysis
• Interprocedural analysis: among functions– propagate the values among functions until they
reach a fixed point
![Page 11: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/11.jpg)
11
First-Order ChannelsFirst-Order Channels
• Use aliasing induced by parameter passing
• For each channel ch, inferch.s = ch.self.s + ch.others.sch.r = ch.self.r + ch.others.r
where,ch.self.s, ch.self.r: {0, 1}ch.others.s, ch.others.r: {0, 1, 2}
• We call the four tuple {ch.self.s, ch.self.r, ch.others.s, ch.others.r} the mode of ch
![Page 12: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/12.jpg)
12
Intraprocedural AnalysisIntraprocedural Analysis
• For each channel ch,if ch is sent by current function then ch.self.s = 1else ch.self.s = 0if ch is received by current function then ch.self.r = 1else ch.self.r = 0
![Page 13: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/13.jpg)
13
Interprocedural AnalysisInterprocedural Analysis
• Compact control flow trees– call nodes, spawn nodes, block nodes, alternativ
e nodes, repetition nodes– depth-first order (once)
• The call graph and the strongly connected component graph– topological sort order (once)
• Strongly connected components– worklist (iterations)
![Page 14: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/14.jpg)
14
Compact Control Flow TreesCompact Control Flow Treestask::main(){ spliter int ch; int i;
ch = channel(); par { producer(ch, many); parfor (i = 0; i < many; i++) consumer(ch); }}
B
B
S
S
B
R
![Page 15: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/15.jpg)
15
The Call Graph and Strongly The Call Graph and Strongly Connected Component GraphConnected Component Graph
main
producer consumer
process
task :: main() { . . . par { producer(ch,…); parfor ( . . . ) consumer(ch); } . . .}
task :: consumer(ch){ . . . process(data); . . .}
task ::producer(ch, …){ . . .}
![Page 16: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/16.jpg)
16
Call NodesCall Nodes
• Let L be the set of channel parameters aliased with ch at the call node n
• Then chn.self.s = x L x.self.s,
chn.self.r = x L x.self.r,
chn.others.s = x L x.others.s,
chn.others.r = x L x.others.r
![Page 17: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/17.jpg)
17
Spawn NodesSpawn Nodes
• Let L be the set of channel parameters aliased with ch at the spawn node n
• Then chn.self.s = 0,
chn.self.r = 0,
chn.others.s = x L x.self.s + x L x.others.s,
chn.others.r = x L x.self.r + x L x.others.r
![Page 18: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/18.jpg)
18
Block NodesBlock Nodes
• If the block node n is the initial node chn.self.s = value from intraprocedural analysis, chn.self.r = value from intraprocedural analysis, chn.others.s = 0, chn.others.r = 0
• Otherwise chn.self.s = 0, chn.self.r = 0, chn.others.s = 0, chn.others.r = 0
![Page 19: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/19.jpg)
19
Bolck NodesBolck Nodes
• Let L be the sequence of nodes inside the block node n
• Then chn.self.s = chn.self.s || x L chx.self.s,
chn.self.r = chn.self.r || x L chx.self.r,
chn.others.s = chn.others.s + x L chx.others.s,
chn.others.r = chn.other.r + x L chx.others.r
![Page 20: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/20.jpg)
20
Alternative NodesAlternative Nodes
• Let L be the sequence of nodes inside the alternative node n
• Then chn.self.s = maxx L chx.self.s,
chn.self.r = maxx L chx.self.r,
chn.others.s = maxx L chx.others.s,
chn.others.r = maxx L chx.others.r
![Page 21: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/21.jpg)
21
Repetition NodesRepetition Nodes
• Let m be the node inside the repetition node n
• Then chn.self.s = chm.self.s,
chn.self.r = chm.self.r,
chn.others.s = if chm.others.s == 0 then 0 else 2,
chn.others.r = if chm.others.r == 0 then 0 else 2
![Page 22: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/22.jpg)
22
Second-Order ChannelsSecond-Order Channels
• Use aliasing induced by both parameter passing and message passing
• For each second-order channel ch, inferch.sS = the set of channels sent to chch.rS = the set of channels received from
ch
![Page 23: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/23.jpg)
23
Intraprocedural AnalysisIntraprocedural Analysis
• For each second-order channel ch, ch.sS = ch.rS = For each first-order channel x, if x is sent to ch ch.sS = ch.sS ∪ { x } if x is received from ch ch.rS = ch.rS ∪ { x }
![Page 24: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/24.jpg)
24
Parameter Passing AliasingParameter Passing Aliasing
task :: main(){ merger pipe int ch; pipe int m; . . . par { parfor ( . . . ) client(ch, m); server(ch); }}
task :: client(ch, c) { . . . send(ch, c); . . .}
task :: server(ch){ . . . s = receive(ch); . . .}
ch.sS : { c }
ch.sS : { m }ch.sS ::
![Page 25: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/25.jpg)
25
Call and Spawn NodesCall and Spawn Nodes
• Let S be the set of second-order channel parameters aliased with ch at the call node n
• Let F be the set of first-order channel parameters at n, and for each p F, let (p) represent the corresponding channel argument of p at n
• Then chn.sS = x S x.sS{pF ,p/(p)} ,
chn.rS = x S x.rS {pF ,p/(p)}
![Page 26: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/26.jpg)
26
Message Passing AliasingMessage Passing Aliasing
task :: main(){ merger pipe int ch; pipe int m, n; . . . par { parfor ( . . . ) client(ch, m); server(ch, n); }}
task :: client(ch, c) { . . . send(ch, c); . . .}
task :: server(ch, s){ . . . s = receive(ch); . . .}
ch.sS : { c }ch.rS:
ch.sS : { m }ch.rS: { n }
ch.sS :: ch.rS: { s }
![Page 27: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/27.jpg)
27
Call and Spawn NodesCall and Spawn Nodes
• For each x in ch.sS, x is aliased with every y in ch.rS
• Then, for each c in {x} ch.rS c.self.s = d {x} ch.rS d.self.s,
c.self.r = d {x} ch.rS d.self.r,
c.others.s = d {x} ch.rS d.others.s,
c.others.r = d {x} ch.rS d.others.r
![Page 28: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/28.jpg)
28
Block NodesBlock Nodes
• If the block node n is the initial node chn.sS = value from intraprocedural analysis,
chn.rS = value from intraprocedural analysis
• Otherwise chn.sS = ,
chn.rS =
![Page 29: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/29.jpg)
29
Block NodesBlock Nodes
• Let L be the sequence of nodes inside the block node n
• Then chn.sS = chn.sS x L chx.sS,
chn.rS = chn.rS x L chx.rS
![Page 30: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/30.jpg)
30
Alternative NodesAlternative Nodes
• Let L be the sequence of nodes inside the alternative node n
• Then chn.sS = x L chx.sS,
chn.rS = x L chx.rS
![Page 31: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/31.jpg)
31
Repetition NodesRepetition Nodes
• Let m be the node inside the repetition node n
• Then chn.sS = chm.sS,
chn.rS = chm.rS
![Page 32: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/32.jpg)
32
Time ComplexityTime Complexityintraprocedural analysis;for each SCC in SCCG begin while changed begin for each CCFT in the SCC begin for each node in the CCFT begin … endfor endfor endwhileendfor
The time complexity: O(V+E)
![Page 33: Static Analysis of Communication Structures in Parallel Programs](https://reader036.fdocuments.net/reader036/viewer/2022062315/56814f39550346895dbcd937/html5/thumbnails/33.jpg)
33
ConclusionsConclusions
• Design a linear algorithm to infer the number of senders and receivers for each channel
• Handle both first-order and higher-order channels
• Use the information about aliasing induced by both parameter passing and message passing