Parallel Computi ng Techniques
-
Upload
nadine-brown -
Category
Documents
-
view
21 -
download
0
description
Transcript of Parallel Computi ng Techniques
ParallelParallel ComputiComputing ng TechniquesTechniques
ParallelParallel ComputiComputing ng TechniquesTechniques
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
SuperComputersSuperComputers
http://www.research.ibm.com/bluehttp://www.research.ibm.com/bluegene/gene/
SuperComputersSuperComputers
http://www.research.ibm.com/bluehttp://www.research.ibm.com/bluegene/gene/ http://ungrid.unal.edu.co/ScDeadProjects.html
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
How to reduce processing How to reduce processing time?time?
1. SuperComputers1. SuperComputers (10(101 1 - 10- 1044 µP, µP, 10101 1 - 10- 102 2
m$)m$)2. Clusters2. Clusters
(10(101 1 - 10- 1033 µP, µP, 50.000 - 1050.000 - 101 1
m$)m$)3. Grids3. Grids
(10(101 1 – 10– 106..n6..n µP, µP, ~0$)~0$)
How to reduce processing How to reduce processing time?time?
1. SuperComputers1. SuperComputers (10(101 1 - 10- 1044 µP, µP, 10101 1 - 10- 102 2
m$)m$)2. Clusters2. Clusters
(10(101 1 - 10- 1033 µP, µP, 50.000 - 1050.000 - 101 1
m$)m$)3. Grids3. Grids
(10(101 1 – 10– 106..n6..n µP, µP, ~0$)~0$)
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
Multitasking Multitasking !=!= MultiprocessingMultiprocessing
MultitaskingMultitasking: 1 CPU: 1 CPU
Multitasking Multitasking !=!= MultiprocessingMultiprocessing
MultitaskingMultitasking: 1 CPU: 1 CPU
t
CPU
~10ms
T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3 ...
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
Multitasking Multitasking !=!= MultiprocessingMultiprocessing
Multiprocessing Multiprocessing : 2+ CPUs: 2+ CPUs
Multitasking Multitasking !=!= MultiprocessingMultiprocessing
Multiprocessing Multiprocessing : 2+ CPUs: 2+ CPUs
t
CPU1T1 T3 T1 T3 T1 T3 T1 T3 T1 T3 T1 T3
T2 T4 T2 T4 T2 T4 T2 T4 T2 T4 T2 T4CPU2
...
...
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
Parallel ComputParallel Computinging: 2+ : 2+ CPUsCPUs
Parallel ComputParallel Computinging: 2+ : 2+ CPUsCPUs
t
CPU1T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1
T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1CPU2
...
...
.
.
.
T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1CPU3...
11. . IntroductIntroduct
ionion
11. . IntroductIntroduct
ionion
Distributed ComputDistributed Computinging: : 2+ Machines2+ Machines
Distributed ComputDistributed Computinging: : 2+ Machines2+ Machines
NET(LAN, MAN, WAN)
T1
T1
T2
T1
T2
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
Flynn's Taxonomy Flynn's Taxonomy A classification of A classification of computer architectures computer architectures based on the number of based on the number of streams of instructions streams of instructions and data:and data:
SISD / MISD / SIMD / MIMDSISD / MISD / SIMD / MIMD
Flynn's Taxonomy Flynn's Taxonomy A classification of A classification of computer architectures computer architectures based on the number of based on the number of streams of instructions streams of instructions and data:and data:
SISD / MISD / SIMD / MIMDSISD / MISD / SIMD / MIMD
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
SISDSISDSingle Instruction Stream/Single Data Single Instruction Stream/Single Data StreamStream. sequential computer: PC (CPU+RAM). sequential computer: PC (CPU+RAM)
MISDMISDMultiple Instruction Stream/Single Multiple Instruction Stream/Single Data StreamData Stream. unusual. unusual
SISDSISDSingle Instruction Stream/Single Data Single Instruction Stream/Single Data StreamStream. sequential computer: PC (CPU+RAM). sequential computer: PC (CPU+RAM)
MISDMISDMultiple Instruction Stream/Single Multiple Instruction Stream/Single Data StreamData Stream. unusual. unusual
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
SIMDSIMDSingle Instruction Stream/Multiple Single Instruction Stream/Multiple Data StreamData Stream. array processor or “vector . array processor or “vector processor” processor” (Westinghouse, (Westinghouse, Fujitsy, NEC, Cray)Fujitsy, NEC, Cray). all CPUs execute same instruction at . all CPUs execute same instruction at same timesame time. parallel done by different data. parallel done by different data
MIMDMIMDMultiple Instruction Stream/Multiple Multiple Instruction Stream/Multiple Data StreamData Stream. . each cpu has own program and dataeach cpu has own program and data. . syncronization by communicationsyncronization by communication
SIMDSIMDSingle Instruction Stream/Multiple Single Instruction Stream/Multiple Data StreamData Stream. array processor or “vector . array processor or “vector processor” processor” (Westinghouse, (Westinghouse, Fujitsy, NEC, Cray)Fujitsy, NEC, Cray). all CPUs execute same instruction at . all CPUs execute same instruction at same timesame time. parallel done by different data. parallel done by different data
MIMDMIMDMultiple Instruction Stream/Multiple Multiple Instruction Stream/Multiple Data StreamData Stream. . each cpu has own program and dataeach cpu has own program and data. . syncronization by communicationsyncronization by communication
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
Amdahl's LawAmdahl's Law ( (Gene Gene AmdahlAmdahl/IBM)/IBM)
““If it takes one man one If it takes one man one minute to dig a post-hole minute to dig a post-hole then sixty men can dig it then sixty men can dig it in one secondin one second””
Amdahl's LawAmdahl's Law ( (Gene Gene AmdahlAmdahl/IBM)/IBM)
““If it takes one man one If it takes one man one minute to dig a post-hole minute to dig a post-hole then sixty men can dig it then sixty men can dig it in one secondin one second””
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
S S = =
S S = =
1111
F + (1-F + (1-F)/NF)/N
F + (1-F + (1-F)/NF)/N
S : speedupS : speedupF : sequential factionF : sequential factionN : number of N : number of processorsprocessors
S : speedupS : speedupF : sequential factionF : sequential factionN : number of N : number of processorsprocessors
22. . Parallel Parallel MachinesMachines
22. . Parallel Parallel MachinesMachines
if N if N infinite infiniteS = 1 / FS = 1 / F
e.g. e.g. if F = 20% if F = 20%
SSmaxmax = 5 = 5
if N if N infinite infiniteS = 1 / FS = 1 / F
e.g. e.g. if F = 20% if F = 20%
SSmaxmax = 5 = 5
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
33. . ClustersClusters33. . ClustersClusters
adm
LANLAN
33. . ClustersClusters33. . ClustersClusters
Distributed System:Distributed System:
Homogeneus Homogeneus platform (HW/SW)platform (HW/SW)
Dedicated machinesDedicated machines Local area networksLocal area networks Centralized Centralized
administrationadministration
Distributed System:Distributed System:
Homogeneus Homogeneus platform (HW/SW)platform (HW/SW)
Dedicated machinesDedicated machines Local area networksLocal area networks Centralized Centralized
administrationadministration
33. . ClustersClusters33. . ClustersClusters
Gigabit ethernet Gigabit ethernet cards/switches cards/switches (60-90 MB/s)(60-90 MB/s)
RAM ~10GB/s (~10RAM ~10GB/s (~1022 faster)faster)
Latency (50-100 µs)Latency (50-100 µs)(~5 µs w/special cards)(~5 µs w/special cards)
Gigabit ethernet Gigabit ethernet cards/switches cards/switches (60-90 MB/s)(60-90 MB/s)
RAM ~10GB/s (~10RAM ~10GB/s (~1022 faster)faster)
Latency (50-100 µs)Latency (50-100 µs)(~5 µs w/special cards)(~5 µs w/special cards)
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
adm
LAN-MAN-WANLAN-MAN-WAN
44. . Computational Computational GridsGrids
44. . Computational Computational GridsGrids
44. . Computational Computational GridsGrids
44. . Computational Computational GridsGrids
Distributed System:Distributed System:
Heterogeneus Heterogeneus platform (HW/SW)platform (HW/SW)
General purpose General purpose machinesmachines
LAN/MAN/WANLAN/MAN/WAN Multiple admin Multiple admin
domainsdomains(e.g. UN Bogotá (e.g. UN Bogotá campus/UN)campus/UN)
Distributed System:Distributed System:
Heterogeneus Heterogeneus platform (HW/SW)platform (HW/SW)
General purpose General purpose machinesmachines
LAN/MAN/WANLAN/MAN/WAN Multiple admin Multiple admin
domainsdomains(e.g. UN Bogotá (e.g. UN Bogotá campus/UN)campus/UN)
44. . Computational Computational GridsGrids
44. . Computational Computational GridsGrids
Specific Purpose Specific Purpose GridsGrids(seti@HOME, ...)(seti@HOME, ...)
General Purpose General Purpose GridsGrids(unGrid, ...)(unGrid, ...)
Specific Purpose Specific Purpose GridsGrids(seti@HOME, ...)(seti@HOME, ...)
General Purpose General Purpose GridsGrids(unGrid, ...)(unGrid, ...)
10-100 ethernet 10-100 ethernet cards/switches cards/switches (1-10 MB/s)(1-10 MB/s)
Latency (10-200 ms)Latency (10-200 ms)
10-100 ethernet 10-100 ethernet cards/switches cards/switches (1-10 MB/s)(1-10 MB/s)
Latency (10-200 ms)Latency (10-200 ms)
44. . Computational Computational GridsGrids
44. . Computational Computational GridsGrids
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
55. . unGridunGrid55. . unGridunGrid
National University of ColombiaNational University of Colombia:: comprises several campuses comprises several campuses across the countryacross the country
The Bogotá campus is The Bogotá campus is interconnected by a FDDI based interconnected by a FDDI based net, where several LAN and VLAN net, where several LAN and VLAN coexistcoexist
Around Around 55000000 computers computers interconnected by this netinterconnected by this net
Free Free CPU cycles are in the 60%-CPU cycles are in the 60%-90% range90% range
National University of ColombiaNational University of Colombia:: comprises several campuses comprises several campuses across the countryacross the country
The Bogotá campus is The Bogotá campus is interconnected by a FDDI based interconnected by a FDDI based net, where several LAN and VLAN net, where several LAN and VLAN coexistcoexist
Around Around 55000000 computers computers interconnected by this netinterconnected by this net
Free Free CPU cycles are in the 60%-CPU cycles are in the 60%-90% range90% range
The unGrid Project attempts to The unGrid Project attempts to construct a virtualconstruct a virtual supercomputer supercomputer that takes profit from these CPU that takes profit from these CPU idle cyclesidle cycles
The unGrid Project attempts to The unGrid Project attempts to construct a virtualconstruct a virtual supercomputer supercomputer that takes profit from these CPU that takes profit from these CPU idle cyclesidle cycles
knowledge grid
information grid
computation/data grid
55. . unGridunGrid55. . unGridunGrid
Factors to consider when Factors to consider when participating in a grid:participating in a grid:
. Privacy. Privacy
. Integrity. Integrity
. Transparency. Transparency
. Configuration/Uninstallation. Configuration/Uninstallation
. Advantages. Advantages
Factors to consider when Factors to consider when participating in a grid:participating in a grid:
. Privacy. Privacy
. Integrity. Integrity
. Transparency. Transparency
. Configuration/Uninstallation. Configuration/Uninstallation
. Advantages. Advantages
55. . unGridunGrid55. . unGridunGrid
55. . unGridunGrid55. . unGridunGrid
JavaSpaces
55. . unGridunGrid55. . unGridunGrid
Master-Worker Pattern
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
1. Introduction1. Introduction2. 2. Parallel Parallel MachinesMachines
3. 3. ClustersClusters4. 4. Computational Computational
GridsGrids5. 5. unGridunGrid
6. 6. Questions & Questions & AnswersAnswers
66. . Questions & Questions & AnswersAnswers
66. . Questions & Questions & AnswersAnswers
?? ??
ungrid.unal.ungrid.unal.edu.coedu.co
ungrid.unal.ungrid.unal.edu.coedu.co
i n g r e s a ri n g r e s a r i n s t a l e u n G r i n s t a l e u n G r
i di d u t i l i c e u n G r u t i l i c e u n G r
i di d
i n g r e s a ri n g r e s a r i n s t a l e u n G r i n s t a l e u n G r
i di d u t i l i c e u n G r u t i l i c e u n G r
i di d