Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous...
Transcript of Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous...
![Page 1: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/1.jpg)
Farms,Pipes,StreamsandReforestationType-DirectedParallelisation
DavidCastro,KevinHammond andSusmit SarkarUniversityofStAndrews,UK
IFIPWorkingGroup2.11Meeting, Bloomington, Indiana,23/8/16
![Page 2: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/2.jpg)
RePhrase Project:RefactoringParallelHeterogeneousSoftware– aSoftwareEngineeringApproach(ICT-644235),2015-2018,€3.6Mbudget
8Partners,6EuropeancountriesUK,Spain, Italy,Austria,Hungary,Israel
CoordinatedbyKevinHammond, StAndrews
0
![Page 3: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/3.jpg)
ParaFormance Project: ParallelPatterns forHeterogeneousMulticoreSystems(ICT-288570),2015-2018,£537Kbudget
Goal:FormationofHigh-Growth Company ofScaleby2023
Coordinated byKevinHammond, StAndrews
![Page 4: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/4.jpg)
TheProblem
• Weneedtochoosethebestparallelabstractions• Algorithmicskeletons [Cole1989]implementpatterns
• Weneedaformalwaytoreasonaboutparallelstructure§ Correctness oftransformations§ Reasoningaboutperformance
4
f1f1
f1
f2f2
f2
f1f1
f1
f2
![Page 5: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/5.jpg)
ExampleSkeleton:ParallelTaskFarms
§ TaskFarmsuseafixednumberofworkers§ Eachworkerappliesthesameoperation(f)§ f isappliedtoeachoftheinputsinastream.
5
f
f…
f
…,x10,x9,x8
x6
x7
x5
f x3
f x2
f x4
fx1,fx0,…
![Page 6: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/6.jpg)
ExampleSkeleton:ParallelPipeline
§ Parallelpipelinescomposetwooperations(f andg)§ overtheelementsofaninputstream§ f and g areruninparallel
6
f g…x9x8
g(fx1)g(fx0)
fx5fx4
g(fx2)fx6
fx3x7
![Page 7: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/7.jpg)
7
Example
![Page 8: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/8.jpg)
ImageMerge
8
Imagemergingcomposestwooperations,merge andmark
Possibleimplementationsinclude:
![Page 9: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/9.jpg)
ChoosinganImplementation
9
DecoratethefunctiontypewithIM(n,m)
where
Nowthetypesystemautomaticallyselects
Wecanguarantee thatthisisfunctionallyequivalent toimgMerge
![Page 10: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/10.jpg)
Inferringparallelstructures
10
Wecanleaveholesinthetypes,e.g.
IM(n,m)=_||FARMm_
replaces_withthesimplestpossiblestructures
IM(n,m)=mincost(_||FARMm_)
replaces_withtheleastcoststructures.
Wecanchoosetheprovablyleastcostskeleton
![Page 11: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/11.jpg)
11
BasicSemantics
![Page 12: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/12.jpg)
SyntaxofSkeletons
12
funT liftsanatomicfunctiontoacollectiontypeTdc representsdivideandconquerovercollectionTfb introducesfeedback
![Page 13: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/13.jpg)
SkeletonDenotational Semantics
13
Basesemantics,S(⍴ isaglobalenvironmentoffunctiondefns)
Liftedtoastreamingform,PovercollectiontypeT
![Page 14: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/14.jpg)
Morphisms forDivide-and-Conquer
14
Catamorphism (fold)
Anamorphism (unfold)
![Page 15: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/15.jpg)
Morphisms forStreams
15
Givenabifunctor,G,mapsovercollectionTare
![Page 16: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/16.jpg)
Iteration
16
Easytodefineusingthefix-pointcombinator,Yf=f(Yf)
![Page 17: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/17.jpg)
17
GeneralisingRecursionPatterns
![Page 18: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/18.jpg)
Hylomorphisms
18
map,cata andana arejustspecialcasesofhylomorphisms
Hylomorphisms aregeneralrecursionpatterns
ForhyloF fg μF recursivecalltreeg howinputsaresplitf howresultsarecombined
![Page 19: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/19.jpg)
Example:Quicksort
19
or
![Page 20: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/20.jpg)
AlltheWorld’saHylomorphism!
20
![Page 21: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/21.jpg)
21
StructureinTypes
![Page 22: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/22.jpg)
IntroducingParallelPatterns
§ Thetypesystemusesastructure-equivalencerelationthatdescribeswhentwoprogramsareextensionallyequivalent.
§ Thetype-checkingalgorithmneedstodecidethesestructure-equivalences.
§ Thetype-checkingalgorithmalsoneedstounifystructures,modulothisstructure-equivalencerelation.
ProgramStructure Targetparallelstructure
Sequentialnormalisedstructure
Sequentialnormalisedstructure
?
![Page 23: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/23.jpg)
SyntaxofStructuredTypes
23
![Page 24: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/24.jpg)
Structure-AnnotatedTypeRules
24
![Page 25: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/25.jpg)
Convertibility
25
Plussomeotherrulesderivedfromthehylomorphism laws.Weusethistoproduceaconfluentrewritingsystem.
![Page 26: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/26.jpg)
ParallelismErasure
26
Rewriterulesderivedfromconvertibility
Repeatedtoproduceaconfluentrewritingsystem
![Page 27: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/27.jpg)
Normalisation
27
Therewriterulesarederivedfrombasiclaws
Usedtodefineanormalisation procedure
Wecannowproveequivalenceoftwoparalleltermsby:i) erasingparallelismusingerase,ii) normalising usingnorm,andiii) testingforequivalence
…
![Page 28: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/28.jpg)
28
Example
![Page 29: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/29.jpg)
QuickSort Revisited
29
Startwithasequentialversion
Tocreateaparalleldivide-and-conquerversion,weneedtodecide
Thisiseasilydoneusingasimpleparallelismerasure
![Page 30: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/30.jpg)
Inferrring MoreComplexParallelStructure
30
Nowconsideramorecomplexstructure
ParallelismerasureontheRHSgives
Normalisation oftheLHS(usingHYLO-SPLITetc)gives
Normalisation oftheRHSgives
![Page 31: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/31.jpg)
Inferrring MoreComplexParallelStructure(2)
31
Weneedtounify thenormalised forms
and
![Page 32: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/32.jpg)
Inferrring MoreComplexParallelStructure(3)
32
Substitutingbackgivesusthedesiredparallelform
Wecanthenuseequivalencetogivetheactualprogram
![Page 33: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/33.jpg)
Costs
33
For1000listsof30,000,000elements
![Page 34: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/34.jpg)
Predictedv.ActualSpeedup
ImageConvolutionof500imagesontitanic,a 2.4GHz24-core,AMDOpteron6176architecture,runningCentosLinux2.6.18-274.e15.Dashedlinesarepredictions.
34
7.3 Image convolution
Image convolution is widely used in image processing applications. The convo-lution algorithm is the composition of two functions read and process. Thefunction read reads an image from a file, and process processes the image.
tread, tproc : TimingIC, BIC : SkelTyIC = Pipe (Farm (Func {ti=tread})) (Farm (Func {ti=tproc}))BIC = bestInst titanic IC
imageConv1 : Par BIC FilePath ImgimageConv1 = skel [readI, processI]
imageConv2 : Auto titanic FilePath ImgimageConv2 = bestSkel [readI, processI] [tread, tproc]
1 2 4 6 8 10 12 14 16 18 20 22
12
4
6
8
10
12
14
16
18
20
22
n2
Workers
Speedup
Pipe (Farm 6 Func) (Farm n2
Func)
Farm n2
(Seq Func Func)
Farm n2
(Pipe Func Func)
(a) titanic
1 4 8 16 24 32 38 46 54 62
14
8
12
16
20
24
28
32
36
40
44
n2
Workers
Farm n2
(Seq Func Func)
Pipe (Farm 12 Func) (Farm n2
Func)
Farm n2
(Pipe Func Func)
(b) lovelace
Fig. 6. Di↵erent Parallel Structures for Image Convolution, 500 Images 1024 * 1024
8 Related Work
Work has also gone into boiling down skeletons into a small set (included in theskeletons we consider) which can express a variety of patterns [17].
Scaife et al [33] present the design and implementation of a parallelisingcompiler that automatically extracts parallelism for Standard ML. They exploitparallelism in the familiarmap and fold HOFs by using nested parallel skeletons.
Refactoring. Roughly, rewriting systems consist of a set of objects with somerelations on how to transform those objects. Rewriting rules for transformingdi↵erent parallel skeletons into other kinds of parallelism have been used forthe implementation of di↵erent refactoring techniques [1, 2, 11]. In this work weprovide a formalisation of rewriting rules for parallel skeletons that allow us toensure that no incorrect parallel structure is introduced.
1 2 4 6 8 10 12 14 16 18 20 22
12
4
6
8
10
12
14
16
18
n Workers
Speedup
Farm n Func
N = 1024N = 2048
(a) Matrix Multiplication
1 2 3 4 5 6 7 8 9 10 11 12
2
3
4
5
6
n2
Workers
Speedup
Farm n (Pipe Func Func)
(b) Image Merge
1 2 4 6 8 10 12 14 16 18 20 22
12
4
6
8
10
12
14
16
18
20
n2
Workers
Speedup
Pipe (Farm n
1
Func) (Farm n
2
Func)
n1
= 1n1
= 2n1
= 4n1
= 6n1
= 8
(c) Image Conv
Fig. 5. Speedups vs predictions (titanic).
![Page 35: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/35.jpg)
35
Conclusion
![Page 36: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/36.jpg)
Conclusions
• First-ever treatment ofparallelismthatreflects parallelstructure intypes
• Severaladvantages toexposingparallelstructure intypes• clearseparation between thestructureandthefunctionality• documentation ofhowaprogramwasparallelized• easytochangetheparallelstructure ofaprogramwithoutmodifyingthe
functionalbehaviour
• Reasoningaboutcostsofdifferent parallelstructure isverypowerful§ Automatically findsuitableparallelstructures§ Compile-time informationabouttherun-time behaviour§ Automatically rewriteprogramstominimizecosts
36
![Page 37: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/37.jpg)
FutureWork
• Otherpatterns,e.g.stencilandbulksynchronousparallelism
• Moredetailedcostmodels(seee.g.Hammondetal,2016)
• DynamicAnalysisisalsopossible
• Allow(certainkindsof)sideeffectsintheworkers
• Implementback-ends.Runourstructuredprograms!
• Largercasestudies
37
![Page 38: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/38.jpg)
PaperAvailableonRequest
38
ToappearinICFP2016
![Page 39: Farms, Pipes, Streams and Reforestation · RePhraseProject: Refactoring Parallel Heterogeneous Software – a Software Engineering Approach (ICT-644235), 2015-2018, €3.6M budget](https://reader033.fdocuments.net/reader033/viewer/2022042909/5f39e3dd4bd7257e9b54df65/html5/thumbnails/39.jpg)
THANKYOU!
http://rephrase-ict.eu
@rephrase_eu
http://paraphrase-ict.eu
39