Application Specific Massive Parallelism Systolic and Instruction Systolic Computing Heiko...

88
Application Specific Massive Parallelism Systolic and Instruction Systolic Computing Heiko Schröder, 2003 LOCOMAP
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    219
  • download

    0

Transcript of Application Specific Massive Parallelism Systolic and Instruction Systolic Computing Heiko...

Application Specific Massive ParallelismApplication Specific Massive Parallelism

Systolic and Instruction Systolic Computing

Heiko Schröder, 2003

LOCOMAP

Heiko Schröder, 2003 Parallel control structures 2

Flynn’s taxonomy ...Flynn’s taxonomy ...

• SA --- Systolic Array

• SIMD --- Single Instruction Multiple Data

• ISA --- Instruction Systolic Array

• MIMD --- Multiple Instruction Multiple Data

• SPMD --- Single Program MD

Heiko Schröder, 2003 Parallel control structures 3

parallel mergeparallel merge

initial situation:

1.) sort columns

(odd-even-transposition sort)

2.) sort rows

(odd-even-transposition sort)

sorted !!!!

x1 x2 x3 x4 x5 x6

x7

x17 x18

y1 y2 y3 y4 y5 y6

y7

y17 y18

...

...

...

...

Heiko Schröder, 2003 Parallel control structures 4

0-1 principle0-1 principle

• The 0-1 principle states that if all sequences of 0 and 1 are sorted properly than this is a correct sorter.

• The sorter must be based on moving data.

initially

0s

0s

1s

after verticalsort

0s

1s

after horizontalsort

0s

1s

Heiko Schröder, 2003 Parallel control structures 5

MIMD (clocked)MIMD (clocked)

min max Time: 2n

Heiko Schröder, 2003 Parallel control structures 6

systolic mergesystolic merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 7

systolic mergesystolic merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 8

systolic mergesystolic merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 9

systolic mergesystolic merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 10

systolic mergesystolic merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 11

1 3 3 45 5 6 74 4 3 29 8 8 7

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 12

systolic mergesystolic merge

1 3 3 45 5 6 74 4 3 29 8 8 7

Heiko Schröder, 2003 Parallel control structures 13

systolic mergesystolic merge

1 3 3 44 4 3 25 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 14

systolic mergesystolic merge

1 3 3 44 4 3 25 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 15

systolic mergesystolic merge

1 3 3 24 4 3 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 16

systolic mergesystolic merge

1 3 3 24 4 3 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 17

systolic mergesystolic merge

1 3 3 24 4 3 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 18

systolic mergesystolic merge

1 3 2 34 3 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 19

1 3 2 34 3 4 45 5 6 79 8 8 7

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 20

1 2 3 33 4 4 45 5 6 79 8 8 7

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 21

1 2 3 33 4 4 45 5 6 79 8 8 7

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 22

1 2 3 33 4 4 45 5 6 79 8 8 7

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 23

1 2 3 33 4 4 45 5 6 78 9 7 8

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 24

1 2 3 33 4 4 45 5 6 78 9 7 8

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 25

1 2 3 33 4 4 45 5 6 78 7 9 8

systolic mergesystolic merge

Heiko Schröder, 2003 Parallel control structures 26

systolic mergesystolic merge1 2 3 33 4 4 45 5 6 78 7 9 8

Heiko Schröder, 2003 Parallel control structures 27

• sorted !!!

systolic mergesystolic merge1 2 3 33 4 4 45 5 6 77 8 8 9

Time: 4n-3Period: nArea: 3n-2

Heiko Schröder, 2003 Parallel control structures 28

Systolic convolutionSystolic convolution

w6 w5 w4 w3 w2 w1* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

* x3 * x2 * x1x6 * x5 * x4

time

Result stream(s)

Heiko Schröder, 2003 Parallel control structures 29

Characteristics of SAsCharacteristics of SAs

Extremely high cost-performanceno flexibility -- long development time

Suitable for special signal processing tasks ???

Heiko Schröder, 2003 Parallel control structures 30

ISA mergeISA merge

x1x8

y1y8

...

...

Period: 4n-4Time: 6n-6

Heiko Schröder, 2003 Parallel control structures 31

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 32

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 33

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 34

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 35

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 36

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 37

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 38

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 39

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 40

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 41

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 42

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 43

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 44

ISA mergeISA merge

x1x8

y1y8

...

...

Heiko Schröder, 2003 Parallel control structures 45

ISA mergeISA merge

x1x8

y1y8

...

...

46Heiko Schröder, 2003 Parallel control structures 46

Principle of the ISA

Communication- Register

Heiko Schröder, 2003 Parallel control structures 47

C := CNC := CN

48Heiko Schröder, 2003 Parallel control structures 48

Interface Processors

Interface Processors Interface Processors NorthNorth

Interface Interface Processors WestProcessors West

ISA

. . . ..

. .

.

49Heiko Schröder, 2003 Parallel control structures 49

Architecture of Systola 1024

Interface processors

ISA

RAM NORTH

program memory

host computer bus

Controller

RAM WEST

Heiko Schröder, 2003 Parallel control structures 50

Hough transform on the ISAHough transform on the ISA

• good line detection method

shear

Fast tomographyfast cryptography

Heiko Schröder, 2003 Parallel control structures 51

robot visionrobot vision

projectorCCD CCD

• stereo vision

Heiko Schröder, 2003 Parallel control structures 52

1. measuring a set of parallel rays

Computerized Tomography: Parallel Projections

1.2.

2. for a number of different angles

3.

3. reconstruction of the 2D-picture

4.

4. 3D-reconstruction out of many 2D-pictures

High-Speed Implementation on the ISA High-Speed Implementation on the ISA

Heiko Schröder, 2003 Parallel control structures 53

Radon Transform

Image Domain

Radon Tr.Radon Tr.

Radon Domain

Inverse Radon Tr.Inverse Radon Tr.

High-Speed Implementation on the ISA High-Speed Implementation on the ISA

Heiko Schröder, 2003 Parallel control structures 54

g( , )x y

t

g( cos sin , )x yt t t

Implementation of Backprojection

Implementation of Backprojection

Heiko Schröder, 2003 Parallel control structures 55

ISA mergeISA merge

1 3 3 45 5 6 79 8 8 74 4 3 2

C:=min{C, CE}

C:=max{C, CW}

Period: 4n-4Time: 6n-6

Heiko Schröder, 2003 Parallel control structures 56

ISA mergeISA merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 57

ISA mergeISA merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 58

ISA mergeISA merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 59

ISA mergeISA merge

1 3 3 45 5 6 79 8 8 74 4 3 2

Heiko Schröder, 2003 Parallel control structures 60

ISA mergeISA merge

1 3 3 45 5 6 74 8 8 79 4 3 2

Heiko Schröder, 2003 Parallel control structures 61

ISA mergeISA merge

1 3 3 45 5 6 74 8 8 79 4 3 2

Heiko Schröder, 2003 Parallel control structures 62

ISA mergeISA merge

1 3 3 44 5 6 75 4 8 79 8 3 2

Heiko Schröder, 2003 Parallel control structures 63

ISA mergeISA merge

1 3 3 44 5 6 75 4 8 79 8 3 2

Heiko Schröder, 2003 Parallel control structures 64

ISA mergeISA merge

1 3 3 44 4 6 75 5 3 79 8 8 2

Heiko Schröder, 2003 Parallel control structures 65

ISA mergeISA merge

1 3 3 44 4 6 75 5 3 79 8 8 2

Heiko Schröder, 2003 Parallel control structures 66

ISA mergeISA merge

1 3 3 44 4 3 75 5 6 29 8 8 7

Heiko Schröder, 2003 Parallel control structures 67

ISA mergeISA merge

1 3 3 44 4 3 75 5 6 29 8 8 7

Heiko Schröder, 2003 Parallel control structures 68

ISA mergeISA merge

1 3 3 44 4 3 25 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 69

ISA mergeISA merge

1 3 3 44 4 3 25 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 70

ISA mergeISA merge

1 3 3 24 4 3 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 71

ISA mergeISA merge

1 3 3 24 4 3 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 72

ISA mergeISA merge

1 3 2 34 3 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 73

ISA mergeISA merge

1 3 2 34 3 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 74

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 75

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 76

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 79 8 8 7

Heiko Schröder, 2003 Parallel control structures 77

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 78 9 7 8

Heiko Schröder, 2003 Parallel control structures 78

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 78 9 7 8

Heiko Schröder, 2003 Parallel control structures 79

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 78 7 9 8

Heiko Schröder, 2003 Parallel control structures 80

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 78 7 9 8

Heiko Schröder, 2003 Parallel control structures 81

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 77 8 8 9

Heiko Schröder, 2003 Parallel control structures 82

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 77 8 8 9

Heiko Schröder, 2003 Parallel control structures 83

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 77 8 8 9

Heiko Schröder, 2003 Parallel control structures 84

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 77 8 8 9

Heiko Schröder, 2003 Parallel control structures 85

ISA mergeISA merge

1 2 3 33 4 4 45 5 6 77 8 8 9

Heiko Schröder, 2003 Parallel control structures 86

Use of the ISAUse of the ISA

Areas of application for ISA:automatic optical quality control

real time signal processingcomputer graphics /visualizationlinear equationsCryptography --> Tele-medicine ?Bio-informatics

Special features:fast aggregate functions (sum, carry)fast local communicationno local memorytypical improvement over PC: Factor 20-30

Heiko Schröder, 2003 Parallel control structures 87

Future of massively parallel architectures

Future of massively parallel architectures

Amdahl’s Law:Amdahl’s Law:

Special purpose --- general purpose ?

Flexibility:Flexibility:

SIMD --- ISA --- MIMD

Physical laws:Physical laws:

Mesh/torus --- hypercube et.al.

Hybrid-networks !

Reconfigurable tori !

Optical computing !

??

??