A Polymorphic Type System for Bulk Synchronous Parallel ML

Frédéric Gava and Frédéric Loulergue

Laboratory of Algorithms, Complexity and Logic

Paris, France

F. Loulergue PaCT 2003 2

Overview

Introduction Pure functional bulk synchronous

parallel programming Problems with nesting Overview of the system Conclusion and future work

Introduction

F. Loulergue PaCT 2003 4

Bulk Synchronous Parallelism + Functional Programming = BSML

Bulk Synchronous Parallelism : Scalability Portability Simple cost model

Functional Programming : High level features (higher order

functions, pattern matching, concrete types, etc.)

Programs proofs Safety of the environment

F. Loulergue PaCT 2003 6

Bulk Synchronous Parallelism

T(s) = (max0i<p wi) + hg + l

F. Loulergue PaCT 2003 7

The BSMLlib Library Bulk Synchronous Parallel ML library

for the Objective Caml language operations on a parallel data

structureAbtract type: par

access to BSP parameters:

bsp_p: unit int

bsp_p() = number of processes

F. Loulergue PaCT 2003 8

Creation of Parallel Vectors mkpar: (int )par

(mkpar f )

Cost: (max0i<p wi)

f (p-1)…(f 1)(f 0)

F. Loulergue PaCT 2003 9

Pointwise Parallel Application apply: ( ) par par par

apply

=

fp-1…f1f0

vp-1…v1v0

fp-1 vp-1…f1 v1f0 v0

F. Loulergue PaCT 2003 10

Communication Operation: Put

type option = None | Some of

put: (int option) par(int option) par

put =

Cost: (max0i<p wi) + hg + l

NoneNoneSome v4Some v1

NoneNoneSome v3None

NoneSome v5NoneNone

NoneNoneSome v2None3210

NoneNoneNoneNone

NoneNoneSome v5None

Some v4Some v3NoneSome v2

Some v1NoneNoneNone3210

F. Loulergue PaCT 2003 11

Global Conditional

if vec at n then … else …if

at n then e1 else e2= e1

Cost: (p-1)g + l

… true fp-1…b1b0

n

F. Loulergue PaCT 2003 12

Implementations of BSMLlib BSMLlib v 0.1 :

O. Ballereau, G. Hains, F. Loulergue Ocaml + BSPlib End of 1999

BSMLlib v 0.2 : Frédéric Gava, Xavier Leroy, Frédéric

Loulergue Ocaml + MPI Available (http://bsmllib.free.fr)

F. Loulergue PaCT 2003 13

Examples

let replicate x = mkpar(fun pid->x) (* bcast: int->’a par->’a par *)

let bcast n vec =

let tosend=mkpar(fun i v dst ->

if i=n then Some v else None) in

let recv=put(apply tosend vec) in

apply (replicate noSome)

(apply recv (replicate n))

Problems with the nesting of parallel vectors

F. Loulergue PaCT 2003 15

Example (1)

let example1 =

mkpar(fun pid->bcast pid vec)

(* example1: par par *)

F. Loulergue PaCT 2003 16

Example (2)

let example2 =

mkpar(fun pid->

let this=mkpar(fun i->i)

in pid)

(* example2: int par *)

F. Loulergue PaCT 2003 17

Example (3)(* fst: ’a *’b ->’a *) let fst = fun (a,b) -> a

two usual values: fst(1,2) two parallel values: fst(mkpar(fun i->i),mkpar(fun i->i)) parallel and usual: fst (mkpar(fun i->i),1)

usual and parallel: fst (1, mkpar(fun i->i))

F. Loulergue PaCT 2003 18

Example (4)let v1 x=mkpar(fun pid -> pid)

and v2 x=put(mkpar(fun i s-> 1+s))in

let c1 x = ((v1 x),1)

and c2 x = ((v2 x),2) in

mkpar(fun pid ->

if pid<(nproc/2)

then snd (c1())

else snd (c2()))

Overview of the type system

F. Loulergue PaCT 2003 20

Types

F. Loulergue PaCT 2003 21

Constraints

F. Loulergue PaCT 2003 22

Types of some predefined expressions

F. Loulergue PaCT 2003 23

Typing judgments

in the type environment E (which binds variables to types)

the expression e has the type [/C].

F. Loulergue PaCT 2003 24

Some Rules

F. Loulergue PaCT 2003 25

Conclusion and Future Work Conclusion

A Polymorphic Type to avoid the nesting of parallel vectors of BSML

Implementation for a mini-BSML Future Work

Sum types Imperative features Full implementation