PhD Lunchtime Seminars “are weekly lunch meetings aimed at bringing together the students,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of PhD Lunchtime Seminars “are weekly lunch meetings aimed at bringing together the students,...
PhD Lunchtime SeminarsPhD Lunchtime Seminars
“are weekly lunch meetings aimed at bringing together the students, researchers and
professors in our and other departments to discuss our work”
http://www.di.unipi.it/~nicotra/wiki/index.php/http://www.di.unipi.it/~nicotra/wiki/index.php/Main_PageMain_Page
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Particular[ly] sharp[y]Particular[ly] sharp[y]Particular[ly] sharp[y]Particular[ly] sharp[y]
Cristian DittamoCristian DittamoCristian DittamoCristian Dittamo
didididi
AgendaAgendaAgendaAgenda
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
What are the problems?What are the problems?What are the problems?What are the problems?
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Hardware evolutionHardware evolutionHardware evolutionHardware evolution
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Parallel programming approachesParallel programming approachesParallel programming approachesParallel programming approaches
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Parallel languageParallel language HPF, High performance Fortran HPF, High performance Fortran
• shared memory, SPMD, shared memory, SPMD, data parallel programmingdata parallel programming POP-C++POP-C++
• an extension of C++, integrating distributed objects, several an extension of C++, integrating distributed objects, several remote method invocations semantics, and resource remote method invocations semantics, and resource requirementsrequirements
Parallel compilerParallel compiler SUIF, Stanford UniversitySUIF, Stanford University, , Automatically translates sequential Automatically translates sequential
C/Fortran programs into parallel C code , shared & distributed memoryC/Fortran programs into parallel C code , shared & distributed memory Parallel librariesParallel libraries
MPIMPI• Message passing, SPMD/MIMDMessage passing, SPMD/MIMD
OpenMPOpenMP• Shared memory, SPMD, Intel C++ compiler, GNU gcc v.4Shared memory, SPMD, Intel C++ compiler, GNU gcc v.4
SkeletonSkeleton programmers qualitatively express parallelism programmers qualitatively express parallelism
• at the source level, instantiating and composing a set of at the source level, instantiating and composing a set of pre-defined pre-defined parallelism exploitation patterns/skeletonsparallelism exploitation patterns/skeletons
Example – MPI (1/2)Example – MPI (1/2)Example – MPI (1/2)Example – MPI (1/2)
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
#include "mpi.h"#include "mpi.h"#include <stdio.h>#include <stdio.h>#include <math.h>#include <math.h>double f(double); double f(double); double f(double a ) { return (4.0 / (1.0 + a*a)); }double f(double a ) { return (4.0 / (1.0 + a*a)); }
int main(int argc,char *argv[]) {int main(int argc,char *argv[]) {// ... variable declaration …// ... variable declaration … char processor_name[MPI_MAX_PROCESSOR_NAME];char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Init(&argc,&argv); MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); MPI_Comm_rank(MPI_COMM_WORLD,&myid);
n = atoi(argv[1]);n = atoi(argv[1]); fprintf(stdout,"Process %d of %d is on %s\n", fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name);myid, numprocs, processor_name); fflush(stdout);fflush(stdout);
MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);
h = 1.0 / (double) n;h = 1.0 / (double) n; sum = 0.0;sum = 0.0; for (i = myid + 1; i <= n; i += numprocs) for (i = myid + 1; i <= n; i += numprocs) { {
x = h * ((double)i - 0.5);x = h * ((double)i - 0.5);sum += f(x);sum += f(x); }} mypi = h * sum;mypi = h * sum;
Example – MPI (2/2)Example – MPI (2/2)Example – MPI (2/2)Example – MPI (2/2)
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);
if (myid == 0) {if (myid == 0) {printf("pi is approximately %.16f, Error is %.16f\n", printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); pi, fabs(pi - PI25DT));fflush(stdout);fflush(stdout); }}
MPI_Finalize(); MPI_Finalize(); return 0;return 0;
}}
Example – OpenMP (1/1)Example – OpenMP (1/1)Example – OpenMP (1/1)Example – OpenMP (1/1)
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
#include <omp.h> #include <omp.h> #include <stdio.h>#include <stdio.h>#include <math.h>#include <math.h>double f(double); double f(double);
double f(double a ) { return (4.0 / (1.0 + a*a)); }double f(double a ) { return (4.0 / (1.0 + a*a)); }
int main(int argc,char *argv[]) {int main(int argc,char *argv[]) {int n, i;int n, i;double pi, h, sum, x;double pi, h, sum, x;
n = atoi(argv[1]);n = atoi(argv[1]); fprintf(stdout,"Process %d of %d is on %s\n", fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name);myid, numprocs, processor_name); fflush(stdout);fflush(stdout);
h = 1.0 / n;h = 1.0 / n;sum = 0.0;sum = 0.0;
#pragma omp parallel for private(x) reduction(+:sum)#pragma omp parallel for private(x) reduction(+:sum)
for (i = 1; i <= n; i++) {for (i = 1; i <= n; i++) {x = h * (i - 0.5);x = h * (i - 0.5);
sum += (4.0 / (1.0 + x*x));sum += (4.0 / (1.0 + x*x));}}
pi = h * sum;pi = h * sum;printf("pi is approximately %.16f, Error is %.16f\n", printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); pi, fabs(pi - PI25DT));fflush(stdout);fflush(stdout);
}}
AgendaAgendaAgendaAgenda
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
A possible solutionA possible solutionA possible solutionA possible solution
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Particular Particular [a]C#[a]C#
Code Bricks
ParallelParallelmodelsmodels
Virtual machine Virtual machine [ CLR / Mono / SSCLI (Rotor) ][ CLR / Mono / SSCLI (Rotor) ]
multi-coremulti-coremulti-multi-
processorprocessormulti-multi-
computercomputeruni-uni-
processorprocessor
CLIFileCLIFile
Operating System Operating System [ Windows, Linux ][ Windows, Linux ]
Without Virtual MachineWithout Virtual MachineWithout Virtual MachineWithout Virtual Machine
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
app1
app2 app3
A new layerA new layerA new layerA new layer
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
app1app2
app3
.NET.NET.NET.NET
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Common Language InterfaceCommon Language InterfaceCommon Language InterfaceCommon Language Interface
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Ref. ECMA 335
Virtual Execution SystemVirtual Execution SystemVirtual Execution SystemVirtual Execution System
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Ref. ECMA 335
Compiling source codeCompiling source codeCompiling source codeCompiling source code
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Ref. “Applied .NET Framework programming”, J.RichterRef. “Applied .NET Framework programming”, J.Richter
Compiler for: C#, Visual Basic, JScript, Alice, APL, COBOL, Component Pascal, Eiffel, Fortran, Haskell, Mercury, ML, Mondrian, Oberon, Perl, Python, RPG, Scheme, and Smalltal
ExecutionExecutionExecutionExecution
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
native code
Ref. “Applied .NET Framework programming”, J.RichterRef. “Applied .NET Framework programming”, J.Richter
How I am madeHow I am madeHow I am madeHow I am made
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Reflection is the ability of a program of access to a Reflection is the ability of a program of access to a description of itselfdescription of itself
A system may support reflection at different levels:A system may support reflection at different levels: from simple information on types (C++ RTTI) to reflecting the from simple information on types (C++ RTTI) to reflecting the
entire structure of the programentire structure of the program another dimension of reflection is if a program is allowed to another dimension of reflection is if a program is allowed to
read or change itselfread or change itself
CLR supports an extensible model of reflection at type-CLR supports an extensible model of reflection at type-system level system level
CLI files contain definition of types annotated with their CLI files contain definition of types annotated with their description (metadata)description (metadata)
ReflectionReflectionReflectionReflection
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
CLI = Data + MetadataCLI = Data + Metadata
Metadata are static and cannot be changed at runtime Metadata are static and cannot be changed at runtime thus the only overhead is in terms of spacethus the only overhead is in terms of space
Metadata are crucial to support dynamic loading as well Metadata are crucial to support dynamic loading as well as other core services (i.e. remoting, serialization, as other core services (i.e. remoting, serialization, reflection, and so on)reflection, and so on)
A program can access metadata using the reflection A program can access metadata using the reflection APIAPI
Custom annotationsCustom annotationsCustom annotationsCustom annotations
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
CLR (and C#) allows to extend metadata with custom CLR (and C#) allows to extend metadata with custom informationinformation
The abstraction provided are The abstraction provided are custom attributescustom attributes
Each element of the type system can be labeled with Each element of the type system can be labeled with attributesattributes
These attributes are attached to metadata and can be These attributes are attached to metadata and can be searched using Reflection APIsearched using Reflection API
Programmer can annotate a program with these Programmer can annotate a program with these information and another program can exploit that to information and another program can exploit that to manage itmanage it
Ex: Web servicesEx: Web services
CodeBricksCodeBricksCodeBricksCodeBricks
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Library that provides:Library that provides:
a technique suitable to express staged computations, targeted a technique suitable to express staged computations, targeted to modern execution environment ( JVM / CLR );to modern execution environment ( JVM / CLR );
• Code fragments are introduced in programs as first class values, that can Code fragments are introduced in programs as first class values, that can be composed by means of an operator called be composed by means of an operator called BindBind
meta-programming support in the runtime environmentmeta-programming support in the runtime environment
an execution model for program based on the mechanism an execution model for program based on the mechanism
CLIFile reader:CLIFile reader: provide an abstract view of the IL code as an arrayprovide an abstract view of the IL code as an array
Ref. “Multi-stage and Meta-programming Support in Strongly Typed Ref. “Multi-stage and Meta-programming Support in Strongly Typed Execution Engines” A.Cisternino, tesi di dottoratoExecution Engines” A.Cisternino, tesi di dottorato
[a]C#[a]C#[a]C#[a]C#
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
[a]C# extends the original language by allowing the use [a]C# extends the original language by allowing the use of custom attributes inside of custom attributes inside a method bodya method body
How can a tool retrieve annotations from an assembly?How can a tool retrieve annotations from an assembly?
The [a]C# run-time provides operations to manipulate The [a]C# run-time provides operations to manipulate the CIL instructions within the scope of annotations:the CIL instructions within the scope of annotations:
ExtrusionExtrusion: is used to extrude the annotation by generating a : is used to extrude the annotation by generating a new method whose body and arguments are respectively the new method whose body and arguments are respectively the annotated code and the free annotated code and the free variables of the annotation;variables of the annotation;
InjectionInjection: is used to insert code immediately before and after an : is used to insert code immediately before and after an annotation;annotation;
ReplacementReplacement: is used to replace the annotated code with the : is used to replace the annotated code with the specified codespecified code
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Example of [a]C# codeExample of [a]C# codeExample of [a]C# codeExample of [a]C# code
public void MyMethod {public void MyMethod { [Annotation1] [Annotation1] {{ int int ii = 0, = 0, jj = 1; = 1; [Annotation2][Annotation2] { {
int int zz = 2;= 2; ii = = jj + + zz;; }} }}}}
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
[a]C#[a]C#acscacsc
[a]C#[a]C#acscacsc
.custom instance void Annotation1::.ctor
.custom instance void Annotation2::.ctor
.locals init (int32 V0, int32 V1, int32 V2)nopldc.i4.0call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.0stloc V0ldc.i4.0stloc V1ldc.i4.1call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.2stloc V2ldloc V1ldloc V2Addstloc V0ldc.i4.1call void [acsrun]ACS.Annotation::End(int32)nopldc.i4.0call void [acsrun]ACS.Annotation::End(int32)nopret
public void MyMethod public void MyMethod {{ [Annotation1] [Annotation1] {{ int int ii = 0, = 0, jj = 1; = 1;
[Annotation2][Annotation2] { {
int int zz = 2;= 2; ii = = jj + + zz;; }} }}}}
IL code analysisIL code analysis Find out annotationsFind out annotations
RewritingRewriting
Two versionTwo version
• Multi-threadedMulti-threaded
• Multi-processMulti-process
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
.custom instance void Annotation1::.ctor
.custom instance void Annotation2::.ctor
.locals init (int32 V0, int32 V1, int32 V2)nopldc.i4.0call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.0stloc V0ldc.i4.0stloc V1ldc.i4.1call void [acsrun]ACS.Annotation::Begin(int32)nopldc.i4.2stloc V2ldloc V1ldloc V2Addstloc V0ldc.i4.1call void [acsrun]ACS.Annotation::End(int32)nopldc.i4.0call void [acsrun]ACS.Annotation::End(int32)nopret
.method public instance void MyMethod()nopldarg.0callvirt instance void Master0()ret
.method public instance void Master0()
.locals init (int32 V0, int32 V1, class state0 V2)nopldc.i4.0stloc V0ldc.i4.0stloc V1newobj instance void state0::.ctor()stloc.s V2ldloc.s V2Ldloc V0stfld float64 state0::s1ldloc.s V2Ldloc V1stfld float64 state0::s2ldarg.0Ldftn instance void Worker_0(object)...ret
ParticularParticular
ArchitectureArchitectureArchitectureArchitecture
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
File.acsacsc
File.exe
ParallelAnnotation.dll
Analyser
File.exe
ParallelVersion.dll
Example of parallelization Example of parallelization Example of parallelization Example of parallelization
public void Mandelbrot (Complex z1, Complex z2, int xsteps, int ysteps) {public void Mandelbrot (Complex z1, Complex z2, int xsteps, int ysteps) {
// variables declaration and initialization// variables declaration and initialization
[ Parallel ] [ Parallel ] {{
int block int block = = ystep / ystep / number_of_workernumber_of_worker;;
for (int count = 0; count < number_of_worker; count++) {for (int count = 0; count < number_of_worker; count++) {
int start = int start = block * count;block * count;
[ Process ] [ Process ] {{
for (int i = 0; i < xsteps ; i++) {for (int i = 0; i < xsteps ; i++) {
for (int j = start ; j < start + block ; j++) {for (int j = start ; j < start + block ; j++) {
// Draw the Mandelbrot fractal// Draw the Mandelbrot fractal
}}
}}
}}
}}}}
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Example of parallelization Example of parallelization Example of parallelization Example of parallelization
C:>C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\acsc.exe /out:.\bin\Debug\Mandelbrot.exe /r:C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\CParallelAnnotation\bin\Debug\CParallelAnnotation.dll MandelGraph.acs Program.cs Form1.cs Form1.Designer.cs CQueue.cs Dialog.cs
acsc
Mandelbrot
Parallel - processesParallel - processes
Mandelbrot.exe
SequentialSequential
++
Parallel - threadsParallel - threads
MandelbrotParallel.dll
Rem_MandelbrotParallelServer0.exe+Rem_MandelbrotParallelServer0.config
Rem_MandelbrotParallelServer1.exe+Rem_MandelbrotParallelServer1.config
MandelbrotParallel.dll
@echo offclsecho ---------------------------------------------echo Compile Mandelbrot fractal renderer projectecho ---------------------------------------------cd C:\Partizione_D\Projects\MandelbrotPar\Mandelbrotcall make2.batecho ...DONEecho.echo ---------------------------------------------echo Generate Parallel version using threadsecho ---------------------------------------------cd C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\Analyser\bin\DebugAnalyser.exe -a C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\Mandelbrot.exe -m genMandel -t thread echo ...DONEecho.echo ---------------------------------------------echo Verify assemblyecho ---------------------------------------------peverify C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\MandelbrotParallel.dllecho ...DONE
@echo offclsecho ---------------------------------------------echo Compile Mandelbrot fractal renderer projectecho ---------------------------------------------cd C:\Partizione_D\Projects\MandelbrotPar\Mandelbrotcall make2.batecho DONEecho.echo ---------------------------------------------echo Generate Parallel version using processesecho ---------------------------------------------cd C:\Partizione_D\Projects\codebricks\ACS\test\CParallel\Analyser\bin\DebugAnalyser.exe -a C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\Mandelbrot.exe -m genMandel -t process hosts.txtecho DONEecho.echo ---------------------------------------------echo Verify assemblyecho ---------------------------------------------peverify C:\Partizione_D\Projects\MandelbrotPar\Mandelbrot\bin\Debug\MandelbrotParallel.dllecho DONE
Analyser
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
ConclusionConclusionConclusionConclusion
Parallel Code generation of a annotated Parallel Code generation of a annotated sequential programssequential programs
Programmer drivenProgrammer driven
Good resultGood result
Advantages:Advantages:
Cross platformsCross platforms
Trasnformations at binary levelTrasnformations at binary level
DebuggingDebugging
AgendaAgendaAgendaAgenda
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
Future worksFuture worksFuture worksFuture works
Dittamo Cristian, Università di Pisa, 27 marzo 2007Dittamo Cristian, Università di Pisa, 27 marzo 2007
ImplementationImplementation schedulerscheduler
more parallel modelsmore parallel models
communication communication
synchronizationsynchronization
Formal specificationFormal specification bytecode rewritingbytecode rewriting