Parallel Architectures Group Grupo de Arquitecturas...

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas Paralelas

Criti al Issues in Design andImplementation of Inter onne tsfor Workstation Clusters

Jos�e DuatoDept. of Information Systems and Computer Ar hite tureUniversidad Polit�e ni a de Valen ia, SpainE-mail: jduato�gap.upv.es WWW: http://www.gap.upv.es 1

��

��

��

��


OutlineIntrodu tionCriti al issuesImproving performan eImproving availability/reliabilityImproving quality of servi e (QoS)

2

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasIntrodu tionThe performan e of most digital systems today is limited by their ommuni ation or inter onne tion, not by their logi or memoryMost of the power is used to drive wires and most of the lo k y leis spent on wire delay, not gate delayAs te hnology improves, pin density and wiring density are s alingat a slower rate than the omponents themselves.The frequen y of ommuni ation between omponents is laggingfar beyond the lo k rates of modern pro essorsThese fa tors ombine to make inter onne tion the key fa tor inthe su ess of future digital systems 3

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasExample: E�e t on exe ution time of ar hite tural improvementsin the network for a DSM with 32 4-way supers alar pro essors

1 2 40.9

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2

2.1

2.2Radix

Bristling factor

Nor

mal

ised

exe

cutio

n tim

e

1 2 40.9

1

1.1

1.2

1.3

1.4

1.5

1.6Ocean

Bristling factor

Nor

mal

ised

exe

cutio

n tim

e

1DQ1DP2DQ2DP1DQ1DP2DS1DQ1DP1AQ1AP1DQ1DP2AS

1DQ1DP2DQ2DP1DQ1DP2DS1DQ1DP1AQ1AP1DQ1DP2AS

4

��

��

��

��


Appli ationsInter onne tion networks are being and will be used for manydi�erent appli ations, ranging from internal buses in VLSI ir uitsto wide area omputer networks. These appli ations in lude:� On- hip inter onne ts (between ALUs and register �les, SMPs)� System area networks (SANs) and system I/O (SIO)� Internal networks for ATM and IP swit hes and routers� Physi al and logi al buses for SMPs� Inter onne ts for multi omputers and DSMs� Networks of workstations (NOWs)}� Lo al area networks (LANs)� Metropolitan area networks (MANs) Computer networks� Wide area networks (WANs)� Networks for industrial appli ations 5

��

��

��

��


Criti al issues in the design of NOWs

Criti al issues 8><>: Performan eAvailability/reliabilityQuality of servi e (QoS)

Main layers 8>>>><>>>>:Appli ations programmer interfa e (API)Communi ation librariesNetwork hardware( Network interfa e ontroller (NIC)Router

6

��

��

��

��


Improving performan e

Design issues8>>>>>>>><>>>>>>>>:

TopologyRouting algorithmFlow ontrol strategyDeadlo k handling strategySwit hing te hniqueRouter ar hite ture

7

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSome de�nitionsTopology: De�nes how the nodes are inter onne ted by hannels� Dire t networks, swit h-based networksRouting algorithm: Determines the path sele ted by a message torea h its destination� Deterministi routing, adaptive routingFlow ontrol: Establishes the handshaking proto ol between routersto ex hange information and de�nes the unit of information onwhi h ow ontrol is performed� Credit-based, stop & goSwit hing te hnique: Determines how and when bu�ers are reservedand swit hes are on�gured� Pa ket swit hing, ir uit swit hing, wormhole, virtual ut-through8

��

��

��

��


Popular topologies1. Dire t networks

(b) 3-ary 2-cube (c) 3-ary 3D-mesh(a) 2-ary 4-cube (hypercube)

9

��

��

��

��


2. Multistage inter onne tion networks1111

1110

1101

1100

1011

1010

1001

1000

0111

0110

0101

0100

0011

0010

0001

0000

1111

1110

1101

1100

1011

1010

1001

1000

0111

0110

0101

0100

0011

0010

0001

0000

1111

1110

1101

1100

1011

1010

1001

1000

0111

0110

0101

0100

0011

0010

0001

0000

1111

1110

1101

1100

1011

1010

1001

1000

0111

0110

0101

0100

0011

0010

0001

0000

Multistage butterfly network Omega network 10

��

��

��

��


3. Swit h-based irregular topologies

01

6

2

5 7

3 4

Bidirectional Links

1 2 0

3

6

5 7

4Switch

Processing Elements

Switch-Based Network Graph Representation 11

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasUni�ed ViewSome manufa turers developed swit hes that are suitable to imple-ment either dire t or indire t networks (Inmos C104, SGI SPIDER)We an view networks using point-to-point links as a set ofinter onne ted swit hes, ea h one onne ted to zero, one, or morenodes:� Dire t networks orrespond to the ase where ea h swit h is onne ted to a single node� Crossbar networks orrespond to the ase where there is a singleswit h onne ted to all the nodes� Multistage inter onne tion networks orrespond to the asewhere swit hes are arranged into several stages and the swit hesin intermediate stages are not onne ted to any pro essor� Swit h-based irregular networks orrespond to the general ase12

��

��

��

��


Appli ation to networks of workstationsSwit h-based inter onne ts like Autonet, Myrinet and ServerNethave been proposed to build networks of workstations with eitherregular or irregular topology.The irregularity provides:

� Wiring exibility.� S alability.� In remental expansion apability.

13

��

��

��

��


Drawba k: The irregularity makes deadlo k avoidan e and routingquite ompli ated.Simplest solution: Avoid deadlo k by eliminating all the y li dependen ies between hannels) Many messages are routed following non-minimal paths.! Higher message laten y! Waste of resour es! Lower throughput 14

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasThe up*/down* routing algorithm (Autonet)

4 2 5

3176

0"up" direction

Routing is distributed, imple-mented using table-lookup, andbased on an assignment of dire -tion to the operational links.Routing rule: a legal route musttraverse zero or more links in the\up" dire tion followed by zero ormore links in the \down" dire tion.

� Ea h y le has at least one link in the \up" dire tion and onelink in the \down" dire tion.� Cy li dependen ies are avoided: messages annot ross a linkin the \up" dire tion after one in the \down" dire tion. 15

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting eÆ ien y

4 2 5

3176

0"up" direction � From 7 to 0: OK� From 2 to 5: la k of adaptivity� From 4 to 1: non-minimal routing

The basi routing rule prevents from using minimal routing andadaptivity in most ases be ause of \down" to \up" on i ts.Probability of non-minimal routing in reases with network size. 16

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasA design methodology for adaptive routing algorithms

inter onne tionnetwork+deadlo k-freerouting fun tionnewmethodology)

physi al hannelsdupli ated or split intotwo virtual hannels(original and new)+extended routingfun tionKey for improved performan e: Allow y li dependen ies between hannels and implement fully adaptive minimal routing! Redu es ontention by in reasing routing adaptivity! Allows more messages to follow minimal paths 17

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasExtended routing fun tion

� Newly inje ted messages an use the new hannels without anyrestri tion. For performan e reasons, only minimal paths areallowed� Original hannels are used exa tly in the same way as in theoriginal routing fun tion� On e a message reserves one of the original hannels, it annotuse any of the new hannels again� When the routing table provides both kinds of hannels, givepreferen e to new hannels

The extended routing fun tion is deadlo k-free 18

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasImproving the eÆ ien y of the methodology� Idea: Fo us on minimal routing, even if adaptivity is redu ed� Restri t the transition from new hannels to original hannels� Improved adaptive routing fun tion:{ Newly inje ted messages an only use new hannels{ At intermediate swit hes, a higher priority is assigned to thenew hannels belonging to minimal paths{ If all the new hannels are busy, then an original hannelbelonging to a minimal path (if any) is sele ted{ If none exists, then the one that provides the shortest pathis used (this ensures deadlo k-freedom)� On e a message reserves an original hannel, it an no longerreserve a new one 19

��

��

��

��


Fully adaptive minimal routing algorithmXS �GRZQ

Escapechannel

Adaptivechannel

0LQLPXP�SDWK

20

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasPerforman e evaluationEvaluation of four routing s hemes:

� Basi up*/down* routing s heme (UD).� Up*/down* routing s heme using two virtual hannels perphysi al hannel (UD-2VC).� Adaptive routing s heme using two virtual hannels per physi al hannel (A-2VC).� Improved adaptive routing s heme using two virtual hannelsper physi al hannel (MA-2VC).

Performan e evaluation arried out by simulation. 21

��

��

��

��


Network model:� Topology generated randomly (8-port swit hes)� 4 nodes (pro essors) onne ted to ea h swit h� Two adja ent swit hes are onne ted by a single link� One routing ontrol unit per swit h (assigned in a round-robinfashion)� Message destination is randomly hosen among nodes� It takes one lo k y le to ompute the routing algorithm, totransfer one it from an input bu�er to an output bu�er, or totransfer one it a ross a physi al hannel 22

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting algorithm: simulation results (I)

25

30

35

40

45

50

55

60

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Ave

rage

Lat

ency

(C

ycle

s)

Traffic (Flits/Cycle/Node)

UDUD-2VC

A-2VCMA-2VC � Network size:16 swit hes� Message length:16 its

23

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting algorithm: simulation results (II)

30

35

40

45

50

55

60

0.1 0.2 0.3 0.4 0.5

Ave

rage

Lat

ency

(C

ycle

s)


’UD’’UD-2VC’

’A-2VC’’MA-2VC’ � Network size:32 swit hes� Message length:16 its

24

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting algorithm: simulation results (III)

30

40

50

60

70

80

0.05 0.1 0.15 0.2 0.25 0.3

Ave

rage

Lat

ency

(C

ycle

s)


UDUD-2VC


25

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting algorithm: simulation results (IV)

80

100

120

140

160

180

200

0.05 0.1 0.15 0.2 0.25 0.3

Ave

rage

Lat

ency

(C

ycle

s)


UDUD-2VC


26

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouting algorithm: simulation results (V)

300

400

500

600

700

800

900

0.05 0.1 0.15 0.2 0.25 0.3

Ave

rage

Lat

ency

(C

ycle

s)


UDUD-2VC


27

��

��

��

��


0.0

0.08

0.16

0.24

0.32

0.40

0.48

0.56

0.64

0.72

CHANNEL

UTILIZATION

0 1

2

3 4 5 6 7

9

10

11

12

13

1415

8 Up*/down* routing.Two virtual hannels.Network size: 16 swit hes.Message length: 16 its.Load: lose to saturation.28

��

��

��

��


0.72

0.64

0.56

0.48

0.40

0.32

0.24

0.16

0.08

0.0

NOITAZILITU

LENNAHC

0 1

2

3 4 5 6 7

9

10

11

12

13

1415

8 Fully adaptive routing.Two virtual hannels.Network size: 16 swit hes.Message length: 16 its.Load: same as for up*/down*29

��

��

��

��


0.0

0.08

0.16

0.24

0.32

0.40

0.48

0.56

0.64

0.72

CHANNEL

UTILIZATION

0 1

2

3 4 5 6 7

9

10

11

12

13

1415

8 Fully adaptive routing.Two virtual hannels.Network size: 16 swit hes.Message length: 16 its.Load: lose to saturation.30

��

��

��

��


Simulation Model Based on Appli ation Message Tra esIs the network in a NOW able to handle the traÆ supported by the network in a DSM without saturating?We gathered message tra es at the node ontroller level in asimulated DSM, then feeding a NOW with those tra es.Tra es were gathered from SPLASH2 appli ations:Appli ation Problem SizeBarnes-Hut 8K parti les, � = 1:0, 4 time stepsLU 512� 512 doubles, 8� 8 blo ksRadix 1M keys, 1K radix, max 1MWater 512 mole ules, 4 time steps 31

��

��

��

��


Main hara teristi s of the simulated systems:

DSM NOWNumber of pro essors 64 64Topology 8 x 8 mesh irregular (16 swit hes)Routing algorithm Deterministi Up/Down, MA-2v Link Width 16 bits 8 bitsLink Bandwidth 400 MB/se 200 MB/se 32

��

��

��

��


Number of messages during exe ution time0

20000

40000

60000

80000

100000

120000

140000

160000

1e+07 2e+07 3e+07 4e+07 5e+07 6e+07 7e+07 8e+07

Mes

sage

s

Time (cycles)

’Amount of messages’

Barnes-Hut

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

1e+07 1.5e+07 2e+07 2.5e+07 3e+07M

essa

ges

Time (cycles)

’Amount of messages’

Water 33

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSimulation results for appli ation tra es (Barnes-Hut)

0

10000

20000

30000

40000

50000

60000

70000

80000

1e+07 2e+07 3e+07 4e+07 5e+07 6e+07 7e+07 8e+07

Late

ncy

(Cyc

les)

Time (Cycles)

MA-2VCUD-2VC

UD

34

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasZoom of the �rst peak (Barnes-Hut)

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

1.8e+07 1.85e+07 1.9e+07 1.95e+07 2e+07

Late

ncy

(Cyc

les)

Time (Cycles)

MA-2VCUD-2VC

UD

35

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasZoom of the se ond peak (Barnes-Hut)

0

50000

100000

150000

200000

250000

3.7e+07 3.75e+07 3.8e+07 3.85e+07 3.9e+07

Late

ncy

(Cyc

les)

Time (Cycles)

MA-2VCUD-2VC

UD

36

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSimulation results for Water

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07

Late

ncy

(cyc

les)

Time (cycles)

’MA-2VC’’UD-2VC’

’UD’

Average message laten yduring Water exe ution0

50000

100000

150000

200000

250000

300000

350000

1.4e7 1.42e7 1.44e7 1.46e7 1.48e7 1.5e7 1.52e7 1.54e7 1.56e7La

tenc

y (c

ycle

s)

Time (cycles)

’MA-2VC’’UD-2VC’

’UD’

Zoom of the se ondsaturation point 37

��

��

��

��


How do irregular networks ompare with regular ones (I)?

30

35

40

45

50

55

60

65

70

0.1 0.2 0.3 0.4 0.5

Late

ncy

(cyc

les)

Traffic (flits/cycle/switch)

’UD’’UD-2vc’’MA-2vc’

’XY’’XY-2vc’

’XYadapt’

� Network size:16 swit hes� Message length:16 its38

��

��

��

��


How do irregular networks ompare with regular ones (II)?

300

350

400

450

500

550

600

650

700

0.1 0.2 0.3 0.4 0.5 0.6

Late

ncy

(cyc

les)




’XYadapt’


��

��

��

��


How do irregular networks ompare with regular ones (III)?

35

40

45

50

55

60

65

0.05 0.1 0.15 0.2 0.25 0.3

Late

ncy

(cyc

les)




’XYadapt’


��

��

��

��


How do irregular networks ompare with regular ones (IV)?

300

400

500

600

700

800

0.05 0.1 0.15 0.2 0.25 0.3 0.35

Late

ncy

(cyc

les)




’XYadapt’


��

��

��

��


How do irregular networks ompare with regular ones (V)?

40

50

60

70

80

0.05 0.1 0.15 0.2 0.25

Late

ncy

(cyc

les)

Traffic (flits/cycles/switch)



’XYadapt’


��

��

��

��


How do irregular networks ompare with regular ones (VI)?

300

400

500

600

700

800

900

0.05 0.1 0.15 0.2 0.25

Late

ncy

(cyc

les)




’XYadapt’


��

��

��

��


Flow ontrol in NOWs

Links onne ting swit hes are usually long and may have verydi�erent lengths ) Channel delay is high and may redu e lo kfrequen y.Solution: Use hannel pipeliningChannel width is small ) It is not interesting to use dedi ated linesfor ow ontrol signals.Solution: Use ontrol its and Stop & Go ow ontrol44

��

��

��

��


\Stop" & \Go" proto olGO (40%)

STOP (66%)Designed to minimize the traÆ of ontrol itsInput bu�ers have two watermarks:\Stop" and \Go", that divide thebu�er into three partsTwo ontrol its: \Stop" and \Go"

45

��

��

��

��


Virtual Channels

No dedi ated ow ontrol lines ) ontrol its are required tomanage virtual hannelsIf it-by- it multiplexing is used, 50% of the bandwidth is wasteddue to the transmission of ontrol itsHow to perform hannel multiplexing eÆ iently?Solution: Transmit blo ks of its

46

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasBlo k multiplexing

A B A B A B A B

2n-1

2n

A B

n

Message Size: n flits

Average Latency: 2n Average Latency: 1.5n2n

(a) (b)

The virtual hannel that owns the link sends its until its queueempties or it re eives a \Stop" ontrol itBlo k multiplexing minimizes the use of ontrol its and may alsoredu e average message laten y 47

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol proto ol

� Before transmitting data its, a \Sele t n" ontrol it is sent,where n is the virtual hannel number

� Data its are sent until the message blo ks or it has been ompletely transferred

� If the input bu�er �lls over the \Stop" mark, a \Stop n" ontrol it is sent. When a \Stop n" ontrol it is re eived, thetransmitter swit hes to the next ready virtual hannel

� If the input bu�er empties below the \Go" mark, a \Go n" ontrol it is sent. When a \Go n" ontrol it is re eived, thetransmitter simply re ords the status hange 48

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasPerforman e evaluationEvaluation of four ow ontrol proto ols:

� Flit-by- it multiplexing with long wires (FT4 or FT8 dependingon wire delay)� Pipelined hannels with blo k multiplexing (PB4 or PB8 depend-ing on wire delay)� Ideal network ( it-by- it multiplexing with short wires) (FT1)� Blo k multiplexing with short wires (PB1)

Performan e evaluation arried out by simulation 49

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasNetwork model:

� Topology generated randomly� 4 nodes (pro essors) onne ted to ea h swit h� Two ontiguous nodes onne ted by a single link� 8-port swit hes� One routing ontrol unit per swit h (assigned in a round-robinfashion)� Fully adaptive routing� Message destination is randomly hosen among nodes 50

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (I)

30

40

50

60

70

80

90

0.2 0.4 0.6 0.8 1

Late

ncy

(cyc

les)


’FT4’’PB4’’FT1’’PB1’

� Network size:16 swit hes� Message length:16 its� Fly time:4 y les51

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (II)

100

150

200

250

300

350

400

0.2 0.4 0.6 0.8 1

Late

ncy

(cyc

les)


’FT4_64f’’PB4_64f’’FT1_64f’

’PB4_128f’’FT1_128f’

� Network size:16 swit hes� Message length:64 and 128 its� Fly time:4 y les52

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (III)

30

40

50

60

70

80

90

100

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

Late

ncy

(cyc

les)


’FT4’’PB4’’FT1’’PB1’

� Network size:64 swit hes� Message length:16 its� Fly time:4 y les53

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (IV)

100

150

200

250

300

350

0.05 0.1 0.15 0.2 0.25 0.3 0.35

Late

ncy

(cyc

les)


’FT4_64f’’PB4_64f’’FT1_64f’

’PB4_128f’’FT1_128f’

� Network size:64 swit hes.� Message length:64 and 128 its.� Fly time:4 y les54

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (V)

40

60

80

100

120

140

160

180

200

220

0.2 0.4 0.6 0.8 1

Late

ncy

(cyc

les)


’FT8’’PB8’’FT1’ � Network size:16 swit hes� Message length:16 its� Fly time:8 y les

55

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasFlow ontrol: simulation results (VI)

100

200

300

400

500

600

0.2 0.4 0.6 0.8 1

Late

ncy

(cyc

les)


’FT8_64f’’PB8_64f’’FT1_64f’

’PB8_128f’’FT1_128f’

� Network size:16 swit hes� Message length:64 and 128 its� Fly time:4 y les56

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling strategyUsually, routing algorithms avoid deadlo k by restri ting routingDeadlo k re overy s hemes allow true fully adaptive minimal routingwithout devoting virtual hannels to deadlo k avoidan eDeadlo k may arise) Me hanisms to dete t and handle deadlo ksare required: timeouts, improved dete tion me hanisms (Martinez etal, ICPP97; L�opez et al, HPCA 98)Disha is a deadlo k re overy te hnique that allows fully adaptiverouting and requires the minimum amount of dedi ated resour esto re over from deadlo ksEa h swit h must be equipped with one or two additional entralbu�ers, alled deadlo k bu�ers 57

��

��

��

��


Applying Disha to irregular networksImplementing Disha in an irregular network with a single deadlo kbu�er per swit h may be impossible. Example:

DeadlockBuffer

Switch B

Msg 2

Switch A

Buffer

Msg 1

Deadlock

Disha an be implemented with two deadlo k bu�ersUp*/down* routing an be used to route messages in the deadlo kbu�ers 58

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSwit h organization for Disha

LC

LC

LC

LC

LC

LC

LC

LC

Switch

Deadlock Buffers

InputChannels

OutputChannelsRouting and

Arbitration

“up”

“down”

59

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasPerforman e evaluationNetworks with one and two virtual hannels per physi al hannelhave been evaluatedThe performan e of Disha has been ompared with the performan eof the avoidan e-based fully adaptive routing (MA-2v algorithm)and the up*/down* algorithmDeadlo k dete tion me hanism: rude timeouts� Evaluated network sizes: 16, 32, and 64 swit hes� Message lengths onsidered: 16, 64, 128, and 256- it messages� Fly time equal to 4 y les ) Input bu�er size is 27 its 60

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (I)

35

40

45

50

55

0.05 0.1 0.15 0.2 0.25 0.3 0.35

Late

ncy

(cyc

les)


’TO 50’’TO 100’’TO 200’’TO 400’

’up-down’

� Network size:32 swit hes.� Message length:16 its.� Virtual hannels:161

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (II)

35

40

45

50

55

60

65

70

0.1 0.2 0.3 0.4 0.5 0.6 0.7

Late

ncy

(cyc

les)

Traffic (flits/cycle/node)

’TO 50’’TO 100’’TO 200’’TO 400’

’MA-2vc’’up-down’


��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (III)

160

180

200

220

240

260

280

300

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

Late

ncy

(cyc

les)


’TO 200’’TO 400’’TO 600’’TO 800’

’up-down’


��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (IV)

200

250

300

350

400

0.1 0.2 0.3 0.4 0.5 0.6 0.7

Late

ncy

(cyc

les)


’TO 200’’TO 400’’TO 600’’TO 800’

’MA-2vc’’up-down’


��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (V)

160

180

200

220

240

260

280

0.1 0.2 0.3 0.4 0.5 0.6

Late

ncy

(cyc

les)


’16 No Disha’’16 Disha’



� Timeout:400 y les.� Message length:128 its.� Virtual hannels:165

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDeadlo k handling: simulation results (VI)

150

200

250

300

350

400

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Late

ncy

(cyc

les)





� Timeout:400 y les.� Message length:128 its.� Virtual hannels:266

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSwit hing te hniqueDisadvantages of wormhole swit hing in a NOW:� Bu�ers must be very large to support hannel pipelining� Wire length is limited by bu�er apa ity

Advantages of virtual ut-through swit hing in a NOW:� Simpler ow ontrol (performed at pa ket level, not at it level)� Bu�er apa ity determined by pa ket size, independent of wirelengthTraditional disadvantages of virtual ut-through swit hing:� Large bu�er size ) Similar to that of wormhole in NOWs� Pa ketizing overhead ) Some messaging layers (FM) for NOWsin rease performan e by splitting messages into �xed-size pa ket67

��

��

��

��


Organization of a wormhole router with virtual hannels

Switch

Phy

sica

l Inp

ut C

hann

els

Phy

sica

l Out

put C

hann

els

Routing and

Arbitration UnitFlit Buffers

VC

CV

CC

VC

CV

CC

68

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasVirtual ut-through router with multiplexed rossbar

Switch

mux

mux

mux

mux

Phy

sica

l Inp

ut C

hann

els

Packet BuffersRouting and

Arbitration Unit

Phy

sica

l Out

put C

hann

els

69

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasVirtual ut-through router with partially multiplexed rossbar

Switch

Phy

sica

l Inp

ut C

hann

els

Packet Buffers

Phy

sica

l Out

put C

hann

els

Routing and

Arbitration Unit

70

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (I)

40

50

60

70

80

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Late

ncy

(cyc

les)


’MA-2vc’’BMA-2vc’’BFA-2vc’ � Network size:16 swit hes� Message length:16 its� Fly time:4 y les

71

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (II)

40

50

60

70

80

0.1 0.2 0.3 0.4 0.5 0.6 0.7

Late

ncy

(cyc

les)



72

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (III)

40

45

50

55

60

65

70

75

80

85

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

Late

ncy

(cyc

les)

Traffic (flits/cycles/switch)


73

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (IV)

150

200

250

300

350

400

450

0.2 0.4 0.6 0.8 1

Late

ncy

(cyc

les)



74

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (V)

200

250

300

350

400

450

500

550

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Late

ncy

(cyc

les)



75

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of swit hing te hnique (VI)

200

250

300

350

400

450

0.1 0.2 0.3 0.4 0.5

Late

ncy

(cyc

les)



76

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of rossbar size (I)

40

60

80

100

120

140

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Late

ncy

(cyc

les)


’Partially Multiplexed Crossbar’’Multiplexed Crossbar’ � Network size:32 swit hes� Message length:16 its� Fly time:4 y les

77

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasE�e t of rossbar size (II)

200

300

400

500

600

700

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Late

ncy

(cyc

les)


’Partially Multiplexed Crossbar’’Multiplexed Crossbar’ � Network size:32 swit hes� Message length:128 its� Fly time:4 y les

78

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRemarksIrregular topologies provide wiring exibility, s alability, and in re-mental expansion at the ost of higher laten y and lower throughputFully adaptive routing onsiderably outperforms deterministi andpartially adaptive routing on irregular topologies, approa hing theperforman e of regular topologies ex ept for large networksPipelined links drasti ally improve performan e over non-pipelinedlinks. EÆ ient ow ontrol is ru ial to minimize ontrol overheadDeadlo k re overy signi� antly improves performan e over deadlo kavoidan e, espe ially when virtual hannels are not implemented.However, it requires eÆ ient deadlo k dete tion te hniquesVirtual ut-through outperforms wormhole swit hing in a NOWenvironment, also featuring simpler ow ontrol, simpler routerar hite ture, and bu�er size independent of wire length 79

��

��

��

��


Improving availability/reliability

Design issues 8>>><>>>:Topology hange/failure dete tionChange/fault resolution8><>: Fault-tolerant routingStati re on�gurationDynami re on�guration

80

��

��

��

��


Network re on�guration (irregular topologies)

Messagerouting

Table update

Tree computation

New rootcomputation

On/offdetection

Swit hes are independent from ea h other and routing tables annotbe syn hronously updated 81

��

��

��

��


Problems asso iated with network re on�guration

'($'/2&.

Deadlock-FreeRouting Function 51

RECONFIGURATIONINTERACTION

,QLWLDO7RSRORJ\

)LQDO7RSRORJ\

Deadlock-FreeRouting Function 52

Traditional solution: stop network traÆ in the region beingre on�gured (stati re on�guration)

82

��

��

��

��


Cost of stati re on�guration0

12

3

0

50

100

150

200

DynamicStatic

Ave

rage

late

ncy

(ms)

Time (s) 83

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasGoalProvide support for dynami re on�guration while avoiding deadlo k

,QFRUUHFWJUDSK

6ZLWFKDFWLYDWLRQ�GHDFWLYDWLRQ

Reconfiguration

process

&RUUHFW�JUDSK

Change in

topology

'HDGORFNDYRLGDQFH

84

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasDiÆ ulties arising in dynami re on�gurationThere is no deadlo k-free sequen e of routing table updates) Perform a sequen e of partial routing table updatesBasi on epts for up*/down* routing

break

break

root

ab

c

d e

f 85

��

��

��

��


Removing nodes from the network

F

D

E

G H

86

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasAdding nodes to the network

D

E

K

I

H

G

F

J

L

M

Q R

O

N

P

S

T 87

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasAdding links to the network

D

E

K

I

H

G

F

J

L

M

Q R

O

N

P

S

T

U

V W

X

88

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasThe resulting a y li network graph

D

E

K

I

H

G

F

J

L

M

Q R

O

N

P

S

T

U

V W

X

89

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasIdentifying regions in the network

D

E

K

I

H

G

F

J

L

M

Q R

O

N

P

S

T

U

V W

X

90

��

��

��

��


Corre ting the inter-region graphD

L

T

U

V

D

L

T

U

V

91

��

��

��

��


Updating link dire tion assignmentsRoot node Break node

Break node Root nodeBreak node Root node

92

��

��

��

��


Moving a root node

Break nodeBreak node

Root node Root node

93

��

��

��

��


Moving a root node over a break nodeBreak node

Root node

94

��

��

��

��


Moving a break nodeBreak node

Root node

Break node

Root node

Break nodeRoot node

95

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasChanging link dire tion assignment

D

K

L

T

U

V

D

K

L

T

U

V

D

K

L

T

U

V

D

K

L

T

U

V

D

K

L

T

U

V

D

K

L

T

U

V

96

��

��

��

��


The resulting link dire tion assignment

D

E

K

I

H

G

F

J

L

M

Q R

O

N

P

S

T

U

V W

X

97

��

��

��

��


RemarksNetwork re on�guration is required when omponents are swit hedon/o�, links are remapped, and omponents failDynami network re on�guration allows routing table update with-out stopping network traÆ Dynami re on�guration is ru ial if the network is designed toprovide quality of servi e (QoS) guaranteesIn most ases, re on�guration only a�e ts a few routing tables.Two major ex eptions arise when the root node fails, and when twoisolated networks are linked together 98

��

��

��

��


Improving QoS

Design issues8>>>>>><>>>>>>:

Swit hing and ow ontrolAdmision ontrol/bandwidth allo ationLink/swit h s hedulingStarvation of best-e�ort traÆ Router ar hite ture

99

��

��

��

��


Goals of the MMR proje tDesign a single- hip Multimedia Router (MMR) for high-speed LANsthat supports a large number of multimedia onne tions whileallo ating the remaining bandwidth to best-e�ort traÆ The MMR should handle this hybrid traÆ eÆ iently, satisfyingthe QoS requirements of multimedia traÆ , minimizing the averagelaten y of best-e�ort traÆ , and maximizing link utilization whenthe network rea hes saturation

100

��

��

��

��


Appli ation requirements

� Very long data streams

� Large number of onne tions

� Jitter sensitive

� Laten y tolerant, espe ially during onne tion setup

� Short ontrol messages 101

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSwit hing and ow ontrolSwit hing te hniques that reserve resour es on the y, like wormholeswit hing, are not well suited be ause the time a message is blo kedwaiting for a busy resour e is not boundedControl messages and best-e�ort traÆ will bene�t from low-laten yswit hing te hniques like wormhole or VCT swit hing

� Long data streams are transmitted by using pipelined ir uitswit hing (PCS) by �rst establishing a onne tion from sour eto destination and then forwarding the data� Control messages and best-e�ort traÆ are transmitted by usingVCT swit hing

Flow ontrol prevents data losses while requiring only small bu�ers102

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRouter ar hite ture

Switch

Routing andArbitration Unit

Phit Buffers

VC Buf.

VC Buf.

VC Buf.

VC Buf.

Phy

sica

l Inp

ut C

hann

els

Phy

sica

l Out

put C

hann

els

Mux

Mux

Mux

Mux

Mux

Mux

Mux

Mux

Phit Buffers

Link/SwitchScheduler

103

��

��

��

��


Virtual hannel bu�er organization

dem

ux

Decoder

ControlWord

mux

RAM

RAM

RAM

RAM

AddressGenerator Link

Scheduler

From

Buffer

Buffer

Buffer

Buffer

104

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasLink/swit h s hedulingFlits/pa kets are syn hronously transmitted ( it y le)Links and swit h ports are syn hronously s heduled for the next it y le during the urrent it y leFlits are large enough to hide s heduling time. Fast hardwares hedulers based on status bit ve torsFlits/pa kets are s heduled a ording to their priority (e.g., EDF).For ea h priority level, ea h link s heduler supplies several andidatesamong those virtual hannels with ready itsThe swit h s heduler assigns swit h ports to links, starting withthose ports with a smaller number of requests. This strategymaximizes link and swit h utilization 105

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSele tion matrix and on i t ve tor

0

1

1

0

2

3

0

2

3

Out

put P

orts

1 2 3

Input Ports

Con

flict

Vec

tor

Can

dida

tes

Can

dida

tes

Leve

l 1Le

vel 2

55Mbps 24Mbps

55Mbps 12Mbps

64Kbps

5Mbps

20Mbps 10Mbps

2

0

2

0

1

1

0

2 106

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasSele tion matrix after arbitration steps

0 1 2 3 0 1 2 3Input Ports Input Ports

01230123

Out

put P

orts

01230123

Out

put P

orts

55Mbps 12Mbps

5Mbps

10Mbps

10Mbps

5Mbps

10Mbps

20

0

10

00000

01

0000000

10100

00X X0

0

1

0X

X

X

X

X

X

X

X

0

0 107

��

��

��

��


Performan e evaluationEvaluation with a mix of CBR and VBR traÆ VBR traÆ omposed of real MPEG-2 video sequen esFrame Size (bits)Video Sequen es Max. Min. AverageAyersro 535030 148755 232976Hook 454560 159622 272738Martin 444588 116094 199880Flower Garden 900139 308411 497126Mobile Calendar 970205 412845 600742Table Tennis 933043 260002 440547Football 590532 340246 441459 108

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasExample of VBR MPEG-2 video sequen e

0

5

10

15

20

25

30

35

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Mbi

ts/s

Time (miliseconds)

Flower Garden Sequence, Q=8

109

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasAverage rossbar utilization

55

65

75

85

95

55 65 75 85 95

Util

izat

ion

(%)

Offered load (%)

Average crossbar utilization for different % of VBR traffic

20 % VBR50 % VBR80 % VBR

110

��

��

��

��


Flit delay statisti sLoad (%) Max. Delay Ave. Delay Std. Dev.Total 59.98 29358 196.38 288.0070.20 200073 341.28 288.0081.13 26915769 283940 1074500784CBR 59.98 3299 127.50 288.0070.20 1913 135.26 288.0081.13 2969 148.99 289.00VBR 59.98 29358 253.76 288.0070.20 200073 516.50 288.0081.13 26915769 521314 1074500784111

��

��

��

��


Flit delay distribution for CBR traÆ Load (%) TH1 TH2 TH3 TH4 TH5 TH659.98 0.034 0.00056 0.00011 0 0 070.20 0.063 0.00075 0 0 0 0TH1 = 838.5 nanose , TH2 = 2 � TH1, ..., TH6 = 268 mi rose

Flit delay distribution for VBR traÆ Load (%) TH1 TH2 TH3 TH4 TH5 TH659.98 6.35 1.17 0.29 0.11 0.045 0.00470.20 19.50 6.81 2.09 0.64 0.20 0.07550% of the workload omposed of VBR traÆ 112

��

��

��

��

GAP Parallel Architectures GroupGrupo de Arquitecturas ParalelasRemarksSingle- hip routers supporting hundreds of onne tions are feasible.They an be used as building blo ks for ommer ial swit hesSmall �xed-size bu�ers an be eÆ iently implemented using inter-leaved RAM modules. Flow ontrol prevents bu�er over ow. Ifne essary, information will be dis arded at the sour e nodeSimple link/swit h s heduling strategies amenable for single- hipimplementation an be very eÆ ient. Syn hronous s heduling,status bit ve tors, and multiple andidates per link are the key designissuesSmall phit bu�ers allow fast pro essing of ontrol informationAdmission ontrol, dynami bandwidth allo ation, and starvation ofbest-e�ort traÆ are urrently under study 113

Parallel Architectures Group Grupo de Arquitecturas...

Documents

Transcript of Parallel Architectures Group Grupo de Arquitecturas...