Evolving the ML Module System Derek Dreyer Toyota Technological Institute at Chicago April 15, 2004.

Evolving the ML Module SystemEvolving the ML Module System

Derek Dreyer

Toyota Technological Institute at Chicago

April 15, 2004

2

Data AbstractionData Abstraction

• Should be able to restrict how much clients of a program module know about its implementation

• Enforcement of program invariants

• Protect clients from implementation changes

3

Object-Oriented Approach (Java)Object-Oriented Approach (Java)

• Classes/objects encapsulate code with data

• Private fields/methods are inaccessible to clients

• Semantics of data abstraction tied up with other features of OOP– Inheritance, subtyping, dynamic dispatch

4

Module-Oriented Approach (ML)Module-Oriented Approach (ML)

• Modules are units of “core-language” code

• Interface of a module describes what other modules get to know about it

• Implementor-side data abstraction via sealing

• Client-side data abstraction via functors

5

Evolving the ML Module System Evolving the ML Module System

• Want to make the ML module system even better:– e.g. Add support for recursive modules

6

Evolving the ML Module System Evolving the ML Module System

• Want to make the ML module system even better:– e.g. Add support for recursive modules

• But where do we start?– Several variants of the ML module system:

• Standard ML, Objective Caml, Moscow ML, etc.

– Relationships/tradeoffs between them are unclear

7

The Goal of This WorkThe Goal of This Work

• Develop a unifying account of existing variations on the ML module system

• Build upon this foundation with support for new features (e.g. recursive modules)

8

OverviewOverview

• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work

9

IntSet ModuleIntSet Module

• Module implementing sets of integers:

module IntSet =

mod

type set = int list

val emptyset : set = []

fun insert (x:int,S:set) : set = x::S

fun member (x:int,S:set) : bool = ...

...

end

10

Using the IntSet ModuleUsing the IntSet Module

• Clients use “dot notation” to refer to components of the IntSet module:

module IntSet = mod ... end

val S : IntSet.set =

IntSet.insert(3, IntSet.emptyset)

11

Using the IntSet ModuleUsing the IntSet Module

• Clients use “dot notation” to refer to components of the IntSet module:

module IntSet = mod ... end

val S : int list =

IntSet.insert(3, IntSet.emptyset)

12

Abstract IntSet InterfaceAbstract IntSet Interface

• Hide definition of set type in interface of IntSet:

interface INT_SET =

iface

type set

val emptyset : set

fun insert : int * set -> set

fun member : int * set -> bool

... end

13

Data Abstraction via SealingData Abstraction via Sealing

• Seal the implementation with the interface:

module IntSet =

(mod ... end) :> INT_SET

• Clients of IntSet can’t see definition of IntSet.set

• IntSet.set is an abstract type

14

Modules Can Have EffectsModules Can Have Effects

• IntSet module was purely functional– Body just defined values and functions

• But modules can have side effects– E.g. creation of mutable state

15

Symbol Table ModuleSymbol Table Module

• When evaluated, generates a new symbol table:

module SymbolTable =

mod

val table = HashTable.create(...)

type symbol = int

fun string_to_symbol (x:string) = ...

fun symbol_to_string (n:symbol) = ...

end

16

Sealing the Symbol TableSealing the Symbol Table


(mod ... end) :>

iface

type symbol

fun string_to_symbol : string -> symbol

fun symbol_to_string : symbol -> string

...

end

17

Making Sets More GenericMaking Sets More Generic

• IntSet module only supports integer sets

• Implementation of sets basically the sameregardless of what the type of items in the set is

• Functors allow you to implement generic sets– Can be instantiated with different item types

– Similar to templates in C++, but more powerful

18

FunctorsFunctors

• A functor is a function from modules to modules:

Some Comparable

Item Type

Set Functor

Sets of

That Item

input output

19

Interface of Comparable ItemsInterface of Comparable Items

interface COMPARABLE =

iface

type item

fun compare : item * item -> bool

end

20

The Set FunctorThe Set Functor

module Set =

functor (Item : COMPARABLE) ->

mod

type set = Item.item list

fun member (x,S) =

...Item.compare(x,y)...

...

end

21

Applying the Set FunctorApplying the Set Functor

module IntItem =

mod

type item = int

fun compare(x,y) = Int.compare(x,y)

end

module IntSet = Set(IntItem)

22

Applying the Set FunctorApplying the Set Functor

module StringItem =

mod

type item = string

fun compare(x,y) = String.compare(x,y)

end

module StringSet = Set(StringItem)

23

Two Forms of Data AbstractionTwo Forms of Data Abstraction

• Can seal a module with an abstract interface– Implementor-side abstraction

• Can use functors to make a module more generic– Client-side abstraction

24

OverviewOverview


25

Type Components of ModulesType Components of Modules

• Modules in ML have type components

• Can “project out” type components of modules:– e.g. IntSet.set, SymbolTable.symbol.

• In all the examples so far, we’ve only projected types out of module variables (or names)

26

Module ExpressionsModule Expressions

• Examples of module expressions:

mod ... end

(mod ... end) :> (iface ... end)

Set(IntItem)

27

QuestionQuestion

• Why not be able to project types from arbitrary module expressions?

mod ... end

(mod ... end) :> (iface ... end)

Set(IntItem)

28

ExampleExample

interface I =

iface type t ... end

module A =

(mod type t = int ... end) :> I

module B =

(mod type t = float ... end) :> I

29

Non-Projectible ModuleNon-Projectible Module

• Suppose M = if button_is_selected() then A else B

30



module C = M

module D = M

31



module C = M

module D = M

• If M.t is a valid type, then C.t = M.t = D.t • But C.t might be int and D.t might be float!!• Unsound for M to be projectible

32

Projectible ModuleProjectible Module

• Suppose M = mod type t = int; val x = 3 end

module C = M

module D = M

• Fine if M is projectible, since C.t = M.t = D.t = int

33

PurityPurity

• “Impure” module expression: if button_is_selected() then A else B

• “Pure” module expression: mod type t = int; val x = 3 end

34

PurityPurity


• “Pure” module expression: mod type t = int; val x = ref 3 end

35

PurityPurity


• “Pure” module expression: mod type t = int; val x = ref 3 end

• Sound for M to be projectible , M is pure (w.r.t. type components)

36

PlanPlan

• Consider how purity and projectibility relate to:– Sealing– Functors– The way that sealing and functors interact

37

SealingSealing

• Suppose M = (mod ... end) :> (iface type t ... end)

module C = M

module D = M

• Sealing has no run-time effect, so M is pure• But if M is projectible, then C.t = M.t = D.t• This violates abstraction!

38

Big PictureBig Picture

if button_is_selected() then A else B

Impure Modules (all non-projectible)

Pure Modules

Projectible Non-projectible

A mod

type t = int val x = ref 3

end

B

M :> I

39

FunctorsFunctors

• To track purity in the presence of functors:– Need to know whether applying a functor will

unleash an effect or not

• Distinguish types of total and partial functors:

– F : I1 ! I2 , body of F is pure

– F : I1 ! I2 , body of F is impure

tot

par

40

Total Total ,, Applicative Applicative

• F : I1 ! I2, M : 1, F and M are pure

module C = F(M)

module D = F(M)

• F(M) is pure ) projectible

C.t = F(M).t = D.t

tot

41

Partial Partial ,, Generative Generative

• F : I1 ! I2, M : 1, F and M are pure

module C = F(M)

module D = F(M)

• F(M) is impure ) non-projectible

C.t D.t

par

42

Set Functor ExampleSet Functor Example

module Set = functor (Item : COMPARABLE) ->

(mod ... end :>

iface

type set

fun member : Item.item * set -> set

...

end)

• Body is pure, so Set functor is total

43

Set Functor ExampleSet Functor Example

module Set = functor (Item : COMPARABLE) ->

(mod ... end :>

iface

type set

fun member : Item.item * set -> set

...

end)

module IntSet1 = Set(IntItem)

module IntSet2 = Set(IntItem)

• IntSet1.set and IntSet2.set are compatible

44

SymbolTable ModuleSymbolTable Module


mod ... end :>

iface

type symbol



end

45

SymbolTable FunctorSymbolTable Functor

module SymbolTable = functor () ->

(mod ... end :>

iface

type symbol



end)

• Body is pure, so SymbolTable functor is total

46

SymbolTable FunctorSymbolTable Functor

module SymbolTable = functor () -> (mod ... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end)

module ST1 = SymbolTable()module ST2 = SymbolTable()

• But ST1.symbol and ST2.symbol are not compatible!

47

Pure vs. Impure SealingPure vs. Impure Sealing

if button_is_selected() then A else B

Impure Modules (all non-projectible)

Pure Modules

Projectible Non-projectible

A mod

type t = int val x = ref 3

end

B

M :> I

M :>> I Two forms

of sealing

48

SymbolTable Functor RevisitedSymbolTable Functor Revisited

module SymbolTable = functor () ->

(mod ... end :>>

iface

type symbol



end)

• Body is impure, so SymbolTable functor is partial

49

SummarySummary

• Analysis in terms of purity and projectibility

• Previous systems make “applicative” or “generative” a design choice, but we support both

• Previous systems just employ one or the other form of sealing and call it “sealing”

50

Unifying Previous SystemsUnifying Previous Systems

• Standard ML ‘97– Only has impure sealing, all functors are partial/generative

• Objective Caml / Leroy (1995)– Only has pure sealing, all functors are total/applicative

• Shao (1999)– Supports both total and partial functors

– Only has impure sealing, can’t write applicative Set functor

• Russo (2000)– Two languages, one like SML and one like O’Caml

– Moscow ML combines them, but language is unsound

51

OverviewOverview


52

Recursive ModulesRecursive Modules

• Existing proposals fall into two categories:– “Units” or “Mixin modules” meant to replace ML modules

(Flatt-Felleisen 98, Ancona-Zucca 96, Duggan-Sourelis 98)

– Extend ML module system with recursive module construct(Crary et al. 99, Russo 01)

• Issues provoked by recursive module construct:– Recursion over general expressions with effects (see

[Dreyer 04] for details)

– Interaction of recursion and data abstraction

53

Recursive Module ExampleRecursive Module Example

rec (X : iface

module A : IA

module B : IB

end.

mod

module A = MA :> IA

module B = MB :> IB

end

)

54

Abstract InterfaceAbstract Interface

iface

module A : iface

type t

...

end

module B : iface

type u

fun g : A.t -> u

end

end

X :

55

Recursive Module BodyRecursive Module Body

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

56

The Two The Two A.tA.t’s’s

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

But X.B.g’s argument type is X.A.t,which is an abstract type!

57

Moscow ML SolutionMoscow ML Solution

iface

module A : iface

type t = int

...

end

module B : iface

type u

fun g : A.t -> u

end

end

X :

58

Our SolutionOur Solution

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = ...

end

At this point, we know that A.t = int,so we know that X.A.t = int as well.

59

Our SolutionOur Solution

mod

module A = mod

type t = int

fun f(x) = ... X.B.g(3) ...

end :> IA

module B = mod ... end :> IB

end

At this point, A.t is abstract, so all we know is that X.A.t = A.t.

60

Current and Future WorkCurrent and Future Work

• Designed an ML dialect based on our analysis, following [Harper-Stone 97]– Currently implementing it in TILT compiler for SML

• Future Work:– Further extensions to ML module system

• E.g. type classes

– Generalizations of ML-style data abstraction

Thank you!Thank you!

Evolving the ML Module System Derek Dreyer Toyota Technological Institute at Chicago April 15, 2004.

Documents

Transcript of Evolving the ML Module System Derek Dreyer Toyota Technological Institute at Chicago April 15, 2004.