11

Introduction to Parallel ProcessingIntroduction to Parallel Processing

CS147CS147

Tung Sze MingTung Sze Ming

22

Topics CoveredTopics Covered

• An Overview of Parallel ProcessingAn Overview of Parallel Processing

• Parallelism in Uniprocessor SystemsParallelism in Uniprocessor Systems

• Parallelism in Multiprocessor SystemsParallelism in Multiprocessor Systems– Flynn’s ClassificationFlynn’s Classification– System TopologiesSystem Topologies

33

An Overview of Parallel ProcessingAn Overview of Parallel Processing

• What is parallel processing?What is parallel processing? Parallel processing is a method Parallel processing is a method

to improve computer system to improve computer system performance by executing more performance by executing more than one instructions at the than one instructions at the same time.same time.

44

MisconceptionMisconception

• When speaking of parallel When speaking of parallel processing, most people would think processing, most people would think of multi-processor systemsof multi-processor systems

55

Parallelism in Uniprocessor SystemsParallelism in Uniprocessor Systems

• It is possible to achieve parallelism with a It is possible to achieve parallelism with a uniprocessor system.uniprocessor system. Some examples are the instruction pipeline, Some examples are the instruction pipeline,

arithmetic pipeline, I/O processor. arithmetic pipeline, I/O processor.

Only if the system processes two different Only if the system processes two different instructions simultaneously can it be instructions simultaneously can it be considered parallelconsidered parallel

A system that performs different operations on A system that performs different operations on the same instruction is not considered parallel.the same instruction is not considered parallel.

66

Parallelism in a Uniprocessor SystemParallelism in a Uniprocessor System

• A reconfigurable arithmetic pipeline is an A reconfigurable arithmetic pipeline is an example of parallelism in a uniprocessor system.example of parallelism in a uniprocessor system.

Each stage of a reconfigurable arithmetic Each stage of a reconfigurable arithmetic pipeline has a multiplexer at its input. The pipeline has a multiplexer at its input. The multiplexer may pass input data, or the data multiplexer may pass input data, or the data output from other stages, to the stage inputs. output from other stages, to the stage inputs. The control unit of the CPU sets the select The control unit of the CPU sets the select signals of the multiplexer to control the flow of signals of the multiplexer to control the flow of data, thus configuring the pipeline.data, thus configuring the pipeline.

77

A Reconfigurable Pipeline With Data Flow for A Reconfigurable Pipeline With Data Flow for the Computation the Computation A[i] A[i] B[i] * C[i] + D[i] B[i] * C[i] + D[i]

0

1MUX2 3

S1 S0

0

1MUX2 3

S1 S0

0

1MUX2 3

S1 S0

0

1MUX2 3

S1 S0

*L

AT

CH

+L

AT

CH

|L

AT

CH

To

memory

and

registers

Data Inputs

0 0 x x 0 1 1 1

88

Although arithmetic pipelines Although arithmetic pipelines can perform many iterations of the can perform many iterations of the same operation in parallel, they same operation in parallel, they cannot perform different operations cannot perform different operations simultaneously. To perform different simultaneously. To perform different arithmetic operations in parallel, a arithmetic operations in parallel, a CPU must include a vectored CPU must include a vectored arithmetic unit. arithmetic unit.

99

Vector Arithmetic UnitVector Arithmetic Unit

A vector arithmetic unit contains A vector arithmetic unit contains multiple functional units that perform addition, multiple functional units that perform addition, subtraction, and other functions. The control subtraction, and other functions. The control unit routes input values to the different unit routes input values to the different functional units to allow the CPU to execute functional units to allow the CPU to execute multiple instructions simultaneously.multiple instructions simultaneously.

1010

A Vectored Arithmetic UnitA Vectored Arithmetic Unit

DataInput

Connections

DataInput

Connections

*

+

-

%

Data

Inputs

AB+C

DE-F

1111

Parallelism in Multiprocessor SystemsParallelism in Multiprocessor Systems

• Parallel processing systems achieve Parallel processing systems achieve parallelism by having more than one parallelism by having more than one processor performing tasks processor performing tasks simultaneously. Since multiprocessor simultaneously. Since multiprocessor systems are more complicated than systems are more complicated than uniprocessor systems, there are uniprocessor systems, there are many different ways to organize the many different ways to organize the processors and memoryprocessors and memory

1212

Flynn’s ClassificationFlynn’s Classification

• SISD: Single instruction with single dataSISD: Single instruction with single data

• SIMD: Single instruction with multiple dataSIMD: Single instruction with multiple data

• MISD: Multiple instruction with single dataMISD: Multiple instruction with single data

• MIMD: Multiple instruction with multiple MIMD: Multiple instruction with multiple datadata

1313

TopologiesTopologies

TTopology of a multiprocessor system opology of a multiprocessor system refers to therefers to the pattern of connections pattern of connections between its processors. Various factors, between its processors. Various factors, typically involving a cost-performance typically involving a cost-performance tradeoff, determine which topology a tradeoff, determine which topology a computer designer will select for a computer designer will select for a multiprocessor system.multiprocessor system.

1414

Types of TopologyTypes of Topology

• Shared Bus Shared Bus TopologyTopology

• Ring Ring TopologyTopology

• Tree Tree TopologyTopology

• Mesh Mesh TopologyTopology

• Hypercube Hypercube TopologyTopology

• Completely connected Completely connected TopologyTopology

1515

Shared Bus TopologyShared Bus Topology

• processors communicate with each processors communicate with each other exclusively through this bus. other exclusively through this bus. However, the bus can only handle However, the bus can only handle only one data transmission at a time. only one data transmission at a time. In most shared busses, processors In most shared busses, processors directly communicate with their own directly communicate with their own local memory.local memory.

1616

Ring TopologyRing Topology

• The ring topology uses direct The ring topology uses direct connections between processors connections between processors instead of a shared bus. This allows instead of a shared bus. This allows all communication links to be active all communication links to be active simultaneously. Data may have to simultaneously. Data may have to travel through several processors to travel through several processors to reach its destinationreach its destination

1717

Tree TopologyTree Topology

• Like the ring, it uses direct Like the ring, it uses direct connections between processors; connections between processors; each having three connections. each having three connections. There is only one unique path There is only one unique path between any pair of processorsbetween any pair of processors

1818

Mesh TopologyMesh Topology

• In the mesh topology, every In the mesh topology, every processor connects to the processors processor connects to the processors above and below it, and to its right above and below it, and to its right and left.and left.

1919

Hypercube TopologyHypercube Topology

• The hypercube is a multidimensional The hypercube is a multidimensional mesh topology. Each processor mesh topology. Each processor connects to all other processors connects to all other processors whose binary values differ by one bit.whose binary values differ by one bit.

2020

Completely connected TopologyCompletely connected Topology

• In the most extreme connection In the most extreme connection scheme, the processors are completely scheme, the processors are completely connected. Every processor has (n-1) connected. Every processor has (n-1) connections, one to each of the other connections, one to each of the other processors. This increases the processors. This increases the complexity of the processors as the complexity of the processors as the system grows, but offers maximum system grows, but offers maximum communication capabilities.communication capabilities.

2121

• THE ENDTHE END